The unchecked AI boom is underway to erase and flatten Indigenous Peoples’ voices and stories
On the politics of Indigenous visibility in a world of AI-generated slop.

A few weeks ago, I came across a disturbing Instagram reel that attempted to depict the lives and rituals of Indigenous communities in pre-colonial North America. The video, clearly AI-generated, showed people with copper-brown skin and monolid eyes, as they dog sled and ate seal meat around a typical igloo reminiscent of the ones I saw in cartoons as a child. As the reel played on, a voiceover explaining the cultural significance of utilizing every part of the seal draped over the almost two minute video, which, by the time I watched it, had nearly 2 million views. 

Earlier this year, Cree artist and musician Donita Large came across an obviously AI-generated graphic of herself on an online news article about her recent album release. The Indigenous artist was shocked by the AI-generated image, which, beyond the issue of a news outlet using AI for content, depicted Large’s face in a way that exaggerated her so-called “Indigenous features.” The AI-generated photo showed her with much darker skin than she actually has, and a sharper nose, along with straight, jet-black hair — all physical qualities stereotypically associated with Indigenous People.

I hope that everyone reading this article has the literacy — indeed, the decency — to know that there is no such thing as an Indigenous “look.” Indigenous People across the world, but especially here in Turtle Island, are incredibly diverse in regards to their phenotype, cultures, rituals, beliefs, mythologies, histories, and ways of knowing and being. 

The colonial brutality unleashed onto Indigenous Peoples of the Americas once Columbus first landed on the white-sand beaches of San Salvador island on October 12, 1492, was apocalyptic. Massacres, enslavement, disease, and every barbarism imaginable followed, all in the name of enriching the imperial core and metastasizing Christianity across the world. According to the most updated analysis, historians estimate that nearly 90 per cent of Indigenous population were wiped out from European contact, mostly from unfamiliar diseases that Indigenous bodies hadn’t built immunity to.

Our Indigenous Peoples have already had their apocalypse. But with the indomitable rise of large language models and artificially generated slop flooding all corners of the internet, Indigenous stories, voices, and representations are once again vulnerable to erasure in an age where information is supposedly more democratized than it’s ever been.

Only this time, the violence of erasure isn’t so obvious — it doesn’t come in the form of forced assimilation or religious missionaries, but rather, through AI-generated imagery and descriptions conveniently poised as educational or entertainment content. These short, click-baity videos are reaching millions of people, repurposing and exploiting deeply racist and stereotypical ideas about Indigeneity to produce a homogenous representation that completely and without consequence, flattens the richness, diversity, and dignity of Indigenous Peoples. 

To keep the technical lesson brief, LLMs are “trained” on data already available through the internet, and uses some elusive (at least, to me) inner predictive machinery to generate a response after carefully breaking down and analyzing each part of a prompt. Though AI content is looked over by a real human, the data sets used to train LLMs remain limited and therefore, fail to represent the diversity that exists within a subject. On top of this, there are serious concerns about the implicit biases built into AI programming that risk amplifying stereotypes informed by outdated, colonial, and racist sentiments rooted in our cultural and historical consciousness.

In an interview with CBC, postdoctoral fellow at Queen’s University, Michael G. Sherbert — whose research focuses on the ethics of using AI for Indigenous cultural preservation — said that, “You could say that AI is inadvertently colonizing and hurting Indigenous language [which is just one aspect of culture that is under appropriation] revitalization because [people] are taking information generated by an artificial intelligence and putting it out there for people to read.”

The Medium, as a publication beholden to strict journalistic standards and ethics — just like every other serious publication in the world — does not allow AI-generated work of any kind. Unfortunately, during the Indigenous issue of volume 51, our paper published what I and many others in the campus community thought was harmful and appropriate graphics of Indigeneity. 

These graphics showed popular and reductionist depictions of Indigenous Peoples in stereotypical headwear, features, and activities — such as drumming. It was later suspected and confirmed that our designer at the time had used AI to generate mock-ups of the graphic ideas, before going over them with their own illustrations and add-ons. Of course, I am in no way individually blaming my past colleague, nor am I questioning the integrity of our paper, which has served our campus responsibly for decades. 

But what I am questioning are the dangers associated with offloading cultural, educational, and representative (artistic) work to AI. What do we lose when this offloading — which we might get away with doing when frantically revising for an exam — becomes all too common?

To add just another example, I also came across an AI-generated depiction of Pochahontus and John Smith’s first encounter, a “live action” rendering of the classic Disney film that my childhood self adored, so to speak. The 45-second Instagram reel showed Pochahontus moving in a disturbingly sexually suggestive way; the reel depicted her face through Eurocentric features, and at the same time, maintained her deep caramel skin, long black hair, and epicanthic eyes. I, as a non-Indigenous person to Turtle Island, have the privilege of simply scrolling past these increasingly frequent AI slop reels with rolling eyes and a deflated, disappointed sigh. 

But for Indigenous people, these inaccurate depictions effectively take away attention and engagement from real Indigenous-identifying artists, knowledge-keepers, elders, writers, and those that lived through the histories and realities these so-called educational reels portray. My outrage is of an intellectual and ethical nature. For Indigenous people, the violence of erasure has simply mutated forms. 

What concerns me and the many others that have noticed this honestly humiliating trend is not the assistive nature of AI — which can help us learn and engage with Indigenous histories, cultures, and worldviews. In fact, many researchers and activists consider ethical AI avenues as a serious and invaluable medium through which Indigenous knowledge can and should be preserved — especially when it comes to certain languages that have few remaining speakers. 

The issue is an urgency of responsibility and the skill it takes to resist the temptation of appropriative and harmful depictions that AI slop so innocently presents as truth. In the same CBC article, Sherbert said that: “[Generative AI] is optimized for something like fluency to give you answers that are good. It’s not optimized for truth or for ethical or cultural responsibility and accountability.” 

In other words, what is perfectly and logically predictable to Claude or ChatGPT can, when measured against the real world, be complete and utter nonsense. But for a young teen or even adult, hearing an AI-generated script or voiceover of an “elder” telling an origin story brings them one step closer to internalizing broad, stereotypical ideas stemming from AI’s own fabricated biases and hallucinations — with little to know redirection to the truth. 

The real danger, slowly brewing underneath this unchecked AI craze and the network of stolen work upholding it, is the fact that we are losing the cultural literacy it takes to navigate this world of AI with regards to dignity and justice for real peoples’ stories, and in doing so, are complicit in the collective erasure of Indigeneity. 

Cultural literacy is a muscle, and it starts with curiosity. Justice is also a collective muscle — as well as an ongoing process of decolonization — and it starts with the truth. Indigenous Peoples stories are all around us — especially in a knowledge hub and diverse network like U of T — and it should be our urgency as students to not only use LLMs critically, but to actively seek out and support real Indigenous voices, artists, and storytellers. In a rebellion against AI. In a rebellion against the echoes of colonialism that lives on today.


*AI-Generated Audio Transcription Disclaimer

The audio transcription included in this article was generated using artificial intelligence (AI) technology with the purpose of improving accessibility for our readers and providing an alternative opportunity to engage with The Medium‘s content. Please note that AI-generated transcriptions may contain errors or inaccuracies in pronunciation, wording, or interpretation.

The original written article remains the authoritative version of the content. If you notice an error in the audio transcription, please contact social@themedium.ca so we can conduct further reviews and corrections.

Opinion Editor (Volume 51); Associate Opinion Editor (Volume 50) — Mashiyat (Mash) is a third-year student studying Neuroscience and Professional Writing and Communication (PWC). As this year’s Opinion Editor, Mash hopes to use her writing, editorial, and leadership skills in supporting student journalism in the essential role it plays in fostering intellectual freedom and artistic expression on campuses. When she’s not writing or slaving away at school, Mash uses her free time cooking cultural dishes, striking up conversations with strangers, and being anxious about her nebulous career plans. You can connect with Mash on her LinkedIn.

Leave a Reply

Your email address will not be published. Required fields are marked *