When AI Meets the Archive: Transforming the Islam West Africa Collection with Large Language Models

I was invited to give a presentation at the SADiLaR Digital Humanities Colloquium, where I discussed how artificial intelligence could transform archival research. My presentation focused on the Islam West Africa Collection (IWAC), comprising over 14,000 documents from six countries containing 25 million words.

The presentation showcased three practical applications of large language models. These included enhancing the accuracy of optical character recognition (OCR) and entity recognition for African materials, conducting experimental sentiment analysis across over 10,000 newspaper articles examining the representation of Islam, and developing a conversational AI chatbot that transcends traditional keyword-based search methodologies.

Drawing on my experience of working on the IWAC, I presented practical solutions for managing digital abundance, emphasising the importance of maintaining a critical awareness of the inherent limitations of AI technologies. The talk emphasised the transformative potential of AI technologies when applied to historical materials, as well as the necessary cautions.