Today’s AI is a very poor representation for humanity. Nearly 60% of the internet used to train existing models is in just one of our over 7000 distinct languages. AI is already shaping a new world, and this imbalance will multiply injustice and deliver a poorer, less interesting world for all.  We must work together to ensure our shared diversity and embedded worldviews are recognised, respected by and integrated into the AI that defines our future. 

We need to put to work our vast archives of existing cultural archives and a world of as-yet un-documented community-held knowledge into formats that can inform our AI. We need to build and improve the tools, extend and connect our archives, and train models at pace and scale. And we need to connect disparate projects and bring them to the people. 

The stakes could not be higher. And a history of colonialism, globalisation and unequal socio-economic forces means we are starting on the back foot. 

Here are 3 steps we are trying to take:

• We need to improve, connect and build the tools we need that help transliterate and translate low-resource languages, create rich descriptions and recognise cultural and geographical visual clues. Together with expert local knowledge, these tools can be used to fine-tune RAG LLMs so that our AI reflects the shared heritage contained in 1000s of vast and disparate archives. We are currently experimenting with our partner the University of Khartoum on fine-tuning RAG LLM for metadata creation. 

• We need to rapidly accelerate community-led decentralised documentation. Together with these tools, this will generate larger and more inclusive datasets of under-represented cultural elements. With over half of our cultural diversity likely to disappear in a few decades, rapid documentation is the only way we can ensure the future of AI reflects the full richness of human heritage and knowledge. We hope our smartphone application can help communities document at scale and pace, and subject to their consent, will explore how the data they own can be used to build more inclusive AI.

• We can use RAG LLMs to bring cultural materials closer to the communities they represent. Improved retrieval of metadata, historical context and provenance of archival content will better enable communities to access, search and explore them. Democratising heritage in this way is the right thing to do. This repatriation of existing archives and increased accessibility to emerging ones also has potential to improve social cohesion, strengthen transmission of living culture, and create feedback loops to further improve the representativeness of our AI. 

This is happening. There are beautiful and innovative tools are in development. For example, Te Hiku Media in New Zealand has built automatic speech recognition for te reo Māori (reaching 92% accuracy). And there are many initiatives to help fine-tune LLMs, such as the Latam-GPT project – an AI model tailored for the linguistic and cultural diversity of Latin America; or Sarvam AI – a startup focused on building vernacular language models for Indian languages. But we need to go faster and further and each play our part to stitch together an AI fit for purpose.

We aim to play a part in this journey towards an inclusive AI. Subject to community and partner approval, our aim is that all cultural and oral history data on our platform will contribute to a fairer, more inclusive, and deeper AI to guide the generations to come.

Are you working at the same interface? Please get in touch to keep up the conversation.