- September 26, 2025
- By Chris Carroll
Since they became widely available three years ago, artificial intelligence chatbots have been embraced as research assistants, trip planners, writing helpers and more—but they’ve provided some face-palm moments as well.
One company’s bot developed a noticeable anti-Muslim bent, while another’s took to praising Hitler. And then there was the ill-fated attempt to reduce bias that resulted in depictions of, among other historical impossibilities, a rainbow coalition of U.S. Founding Fathers.
Although each problem was addressed, the trend has persisted, and there’s no reason to believe it will stop without thoughtful action, said Philip Resnik, a University of Maryland scholar whose research lies at the intersection of linguistics, computing and artificial intelligence.
“It’s what I call playing ‘whack-a-mole’ when you deal with these things as individual problems to solve,” said Resnik, a professor of linguistics with a joint appointment in the University of Maryland Institute for Advanced Computer Studies. “You fix it here, and then it pops up there, and maybe your corrective action created yet another problem.”
In a recent paper in Computational Linguistics, provocatively titled “Large Language Models Are Biased Because They Are Large Language Models,” Resnik argues that harmful biases in chatbots aren’t indicative of a flaw in the technology, but a reflection of the basic nature of the models these uncanny systems able to interact in unscripted, everyday language are based on.
In an interview with Maryland Today, Resnik explained how choices made in the development of large language models unintentionally led to the bias—which he said could have more far-reaching consequences than outrageous AI statements. But, he says, new choices could lead to AI tools that work for everyone.
Chatbots have these offensive outbursts, but they’re also useful in many ways. How much do we need to worry?
Yes, they can indeed be very useful, but we’re not only talking about chatbots, but the large language models, or LLMs, they are based on. These don’t only do chatbot-like interaction, but are also summarizing, making suggestions about potentially important decisions, and much more. Also, we are emphatically not talking about politeness, or just making technology more pleasant to interact with. It’s about the problem of a technology that increasingly has real-world impacts. If you have a system with negative or positive biases based on someone’s dialect, for instance, and it’s helping make hiring decisions, you can see the problem.
Why would computer systems be biased?
Intelligent behavior, including in people, often involves things like presuppositions or generalizations that researchers conventionally call biases, and they’re not always bad. To some extent, we can’t operate without them, but when you begin to apply those to questions of a person’s identity, or make individual decisions based on stereotypes—even if they have some validity at a statistical level—that can be very harmful.
How do they become biased?
The fundamental job of an LLM is to generate plausible strings of words. The decision about what word to produce next is based on an extremely rich internal model of words and concepts and relationships among them. That’s the amazing power of these things—they analyze hundreds of billions of words of text and extract this underlying model that’s not just a word association game. It goes far deeper than the level of words.
Here’s where the problem comes in: They learned this model by observing an incredible volume of things people have said in the past. Observing how words are distributed in text tells them dogs bark, that a kiwi is a kind of food, all kinds of important knowledge about the world. But those observations also include all kinds of bad stuff—the stereotypes, the presuppositions, the racism and bias you find in written material online and elsewhere. And crucially, what I argue in my paper is that from a formal, mathematical standpoint, a system that learns in this strictly distributional fashion simply has no way to distinguish the valid knowledge you want it to learn from the other stuff.
Can you give an example?
Consider the word nurse. In text, you’ll find statistical clues to its meaning, a kind of health care professional, in its direct and indirect connections to things like doctor, bandage, treat. But you’ll also find nurses discussed with gendered or gender-skewed terms like she, redhead, gentle, pretty, which provide clues to the statistical fact that nurses tend to be female, although that is not a part of the word’s meaning.
It becomes a real issue because LLMs have no way to distinguish factual connections from statistically supported connections in people’s language that have no grounding in truth. For example, there are false but pervasive clues in the volumes of text they’re trained on that create associations between certain groups and concepts like greedy or lazy or thug.
How can we fix this?
Nothing says these models have to start from such a blank slate. I’m not claiming that changing the path of AI development would be easy. Clearly, the kind of large-scale distributional learning that enabled today’s LLMs is vastly more effective and powerful than older approaches, but we didn’t have to throw out the baby—the fact words have meanings beyond just how they’re used—with the bathwater. Decades of thought in linguistics, cognitive science and other fields have gone into what it means to represent meaning and knowledge, and we should take advantage of that. One possibility would be to revise learning approaches so that notions like meaning and factuality have a privileged status, rather than having systems that can learn only through observational clues.
How likely is this shift to occur?
Given the amount of money tied up in the current paradigm right now, I don’t think fundamental changes are very likely. But our job as academics is to always ask the hard questions, think about how we can make things better, and hopefully engage with thoughtful people inside the industry. The point of the paper is that we need to have this conversation about the basic nature of these models now, not later, given the increasing role of artificial intelligence in all our lives.