Inherited Flaws — Why Do AI Systems Inherit Human Limitations?
AI is remarkably intelligent. Yet it makes surprisingly human mistakes. Hallucination, Overconfidence, Anchoring, Sycophancy. Are these uniquely AI problems? Or are they inherited from us?
Introduction
AI is remarkably intelligent.
ChatGPT can engage in natural conversation.
Claude can analyze long documents.
Gemini can handle enormous amounts of context.
Modern models can write code, achieve high scores on exams, summarize research papers, and sometimes produce answers that are difficult to distinguish from those of human experts.
Capabilities that would have seemed like science fiction only a few years ago have become part of everyday life.
And yet something strange is happening.
AI systems make surprisingly human mistakes.
They cite papers that do not exist.
They make confident mistakes.
They believe information that happens to fit a particular narrative.
They become strongly influenced by initial assumptions.
They follow majority opinions.
They flatter users.
And sometimes they seem unaware of the limits of their own knowledge.
We often describe these as uniquely AI problems.
Hallucination.
Bias.
Overconfidence.
Anchoring.
Sycophancy.
But consider another possibility.
What if these are not uniquely AI problems at all?
What if they are limitations that already existed within us, now reappearing in a different form?
In previous articles, we explored whether AI truly understands.
We examined whether AI possesses anything resembling a self.
We looked at why AI systems repeatedly make similar mistakes.
We also explored RLHF and Sycophancy, and how modern models adapt to human society.
Throughout those discussions, one question gradually emerged.
Why does AI seem so human?
After all, AI is not human.
It has no brain.
No body.
No childhood.
No personal history.
And yet it often fails in ways that look remarkably familiar.
Is this merely coincidence?
Or is there a deeper structural reason behind it?
In this article, we will explore that question.
Rather than treating AI failures as isolated technical defects, we will examine them as inherited characteristics.
And ultimately, we will ask why AI appears to reproduce not only human intelligence, but also human limitations.
Are AI Failures Really Unique to AI?
When ChatGPT was released in late 2022, many people had the same reaction.
AI was astonishingly intelligent.
But it was also astonishingly foolish.
It could write sophisticated software while making elementary arithmetic mistakes.
It could discuss complex legal concepts while citing fictional court cases.
It could produce academic-sounding essays while inventing sources that never existed.
At the time, these behaviors seemed deeply strange.
During the age of search engines, people had become accustomed to computers being precise.
Calculators do not make up numbers.
Databases do not invent records.
Search engines do not fabricate web pages.
The idea that a machine could generate plausible falsehoods felt shocking.
But perhaps the truly surprising thing was something else.
Its mistakes looked strikingly human.
Hallucination and False Memory
The most famous AI failure is probably hallucination.
A model generates information that does not actually exist.
Nonexistent papers.
Nonexistent URLs.
Nonexistent people.
Nonexistent quotations.
This behavior was especially common in early versions of ChatGPT.
Although newer models have improved substantially, the problem has not disappeared entirely.
Many people interpret hallucination as lying.
But that is not quite accurate.
The model is not attempting to deceive.
It is attempting to generate the most plausible continuation of available information.
When knowledge is incomplete, gaps must be filled.
Sometimes those filled-in details happen to be wrong.
That is hallucination.
But what about humans?
Do humans avoid this problem?
Psychologist Elizabeth Loftus spent decades studying human memory.
Her findings revealed something remarkable.
Human memory is not a recording device.
We do not simply retrieve stored information.
Instead, memories are reconstructed each time they are recalled.
And during that reconstruction process, memories can change.
Sometimes people become convinced they experienced events that never actually occurred.
One famous experiment involved presenting participants with a fictional childhood story about becoming lost in a shopping mall.
Some participants eventually began describing the fabricated event as a genuine memory.
An event that never happened became part of their personal history.
This phenomenon is known as false memory.
Of course, hallucination and false memory are not identical.
Their mechanisms differ.
Their causes differ.
Yet their outcomes are surprisingly similar.
Both involve filling gaps in incomplete information.
Both involve constructing coherent narratives.
Both can be expressed with confidence.
And in both cases, the individual often remains unaware that an error has occurred.
When AI hallucinations first attracted public attention, many people viewed them as unprecedented.
But perhaps humanity has been living with a similar phenomenon all along.
Overconfidence and the Illusion of Understanding
There is another fascinating example.
AI systems often make mistakes with extraordinary confidence.
Almost every user has encountered this.
The answer is wrong.
Yet it is delivered with complete certainty.
This characteristic has become one of the most common criticisms of modern AI.
But humans behave similarly.
Consider the Illusion of Explanatory Depth discussed in a previous article.
People often believe they understand things far better than they actually do.
Take a toilet.
Most people use one every day.
As a result, they assume they understand how it works.
Yet when asked to explain the complete mechanism in detail, many quickly discover that their understanding is surprisingly shallow.
The same is true for bicycles.
Zippers.
Ballpoint pens.
The more familiar something is, the more likely we are to assume we understand it.
And only when we attempt a detailed explanation do we discover the limits of our knowledge.
Research by Rozenblit and Keil demonstrated that this phenomenon is widespread.
People believe they understand.
In reality, they often do not.
And perhaps most importantly, they frequently fail to recognize that they do not understand.
This is a failure of metacognition.
AI overconfidence often appears similar.
The model possesses incomplete knowledge.
Yet it continues generating answers.
The result can be unwarranted certainty.
Again, humans and AI are not the same.
But both are attempting to reason under conditions of incomplete information.
And that shared constraint may help explain why similar failures emerge.
Anchoring
One of the most famous cognitive biases in psychology is anchoring.
The first piece of information we receive strongly influences later judgments.
Suppose someone tells you that a product originally costs one million yen.
A discounted price of five hundred thousand yen may then seem inexpensive.
But if the first number you hear is one hundred thousand yen, the same five hundred thousand yen appears expensive.
The value has not changed.
The reference point has.
Daniel Kahneman and Amos Tversky demonstrated that such heuristics play a major role in human decision-making.
Interestingly, similar patterns can appear in AI systems.
Provide a model with a flawed assumption at the beginning of a prompt.
That assumption can influence the entire reasoning process that follows.
For example, suppose a prompt incorrectly states that a particular researcher proved a certain theory.
The model may continue reasoning within that false framework.
Modern models are far better at detecting these situations than earlier systems.
Yet the phenomenon has not disappeared completely.
This is not simply a matter of missing knowledge.
Initial context shapes the space of possible reasoning.
And that pattern appears in both humans and AI.
Confirmation Bias
Confirmation bias may be even more familiar.
People seek evidence that supports what they already believe.
Contradictory information receives less attention.
Evidence that challenges existing views is often discounted.
As a result, prior beliefs become stronger over time.
This phenomenon appears in politics.
In religion.
In investing.
Even in science.
Researchers themselves can become attached to hypotheses.
Supporting evidence attracts attention.
Contradictory evidence can feel less compelling.
Human beings are not perfectly rational observers.
We often see the world we expect to see.
The rise of social media has amplified this tendency.
Algorithms prioritize content that aligns with existing interests.
As a result, people increasingly encounter opinions similar to their own.
Confidence grows.
Understanding does not necessarily improve.
Sometimes perspectives become narrower instead.
AI systems face related challenges.
But to understand why, we first need to understand why humans make these mistakes in the first place.
Why are human beings not fully rational?
Why does intelligence produce so many systematic errors?
Those questions lead us directly into the next section.
Why Do Humans Make Mistakes?
As we have seen so far, Hallucination, Overconfidence, Anchoring, Confirmation Bias.
These are not uniquely AI problems.
They are deeply human problems as well.
But this immediately raises another question.
Why do humans make these mistakes in the first place?
If intelligence exists to help us make better decisions, why are cognitive biases so common?
Why did evolution not eliminate them?
At first glance, this seems strange.
Yet the answer may be simpler than it appears.
Evolution did not optimize human beings for truth.
It optimized us for survival.
The Human Brain Was Not Designed for Truth
We often assume that intelligence exists to discover objective reality.
Science.
Mathematics.
Logic.
Philosophy.
When we think about human intelligence, we naturally focus on these achievements.
But from an evolutionary perspective, the story looks very different.
For most of human history, survival mattered far more than truth.
Imagine a hunter-gatherer hearing movement in nearby grass.
It might be the wind.
Or it might be a predator.
A perfectly rational analysis would require gathering more information.
But hesitation could be fatal.
Assuming danger and running away is often the safer strategy.
Even if the assumption is wrong.
Over thousands of generations, evolution favored organisms that survived.
Not organisms that possessed perfect models of reality.
As a result, the human brain evolved to make fast decisions under uncertainty.
This was not a flaw.
It was a successful survival strategy.
The fact that we are here today is evidence of that success.
Heuristics: Shortcuts for Survival
Psychologists Daniel Kahneman and Amos Tversky demonstrated that much of human decision-making relies on heuristics.
A heuristic is a cognitive shortcut.
Rather than performing a complete analysis of every situation, the brain uses simplified rules.
Most of the time, these shortcuts work surprisingly well.
Consider the availability heuristic.
Events that are easy to remember often seem more common than they actually are.
After seeing a news report about an airplane crash, flying may suddenly feel dangerous.
Yet statistically, driving remains far more dangerous.
The brain is not calculating actual probabilities.
It is using ease of recall as a substitute.
This method is efficient.
It is fast.
And in many situations, it is good enough.
But it can also produce systematic errors.
The same pattern appears across many cognitive biases.
The brain trades perfect accuracy for speed and efficiency.
Most of the time, this is a reasonable tradeoff.
Sometimes it is not.
System 1 and System 2
Kahneman proposed a useful framework for understanding human thought.
He described two modes of thinking: System 1 and System 2.
System 1 is fast. Automatic. Intuitive. Effortless.
It recognizes faces. Detects threats. Interprets emotions. Handles ordinary conversation.
Most daily decisions occur through System 1.
System 2 is different. It is slow. Deliberate. Analytical.
It performs calculations. Constructs formal arguments. Evaluates complex evidence. Plans long-term strategies.
When people imagine rational thought, they are usually imagining System 2.
The problem is that humans rely on System 1 far more than they realize.
Many judgments are made intuitively first.
Only afterward do we construct explanations.
In other words, people often do not reason before deciding.
They decide first and reason afterward.
This realization has profound implications.
Much of what we consider rational thinking may actually be post-hoc justification.
Intelligence Is Not the Same as Rationality
Another surprising discovery emerged from cognitive science.
Highly intelligent people are not immune to cognitive biases.
In fact, intelligence and rationality are not the same thing.
A person with a high IQ may still fall victim to confirmation bias.
May still become overconfident.
May still anchor on irrelevant information.
Some researchers have even argued that greater intelligence can sometimes make biases harder to detect.
Why?
Because highly intelligent people are often better at constructing explanations.
They can defend mistaken beliefs more effectively.
They can rationalize poor decisions more convincingly.
Intelligence is not merely a tool for finding truth.
It can also become a tool for defending error.
This observation is important because a similar pattern appears in AI.
As models become larger and more capable, their explanations become more sophisticated.
Their reasoning becomes more convincing.
But mistakes do not automatically disappear.
Sometimes errors become harder to recognize precisely because the explanations sound so plausible.
More intelligence does not necessarily eliminate failure.
Sometimes it changes the form that failure takes.
AI Failures and Human Failures
At this point, a pattern begins to emerge.
AI does not possess the same cognitive architecture as humans.
It has no neurons.
No emotions.
No survival instincts.
No evolutionary history.
And yet it often produces failures that resemble human failures.
Why?
One possibility is that any intelligent system operating under constraints faces similar challenges.
Information is incomplete.
Time is limited.
Resources are finite.
Perfect reasoning is impossible.
Under such conditions, shortcuts become necessary. Approximations become necessary. Prioritization becomes necessary.
And whenever approximations are used, errors become possible.
Humans face this problem.
AI faces it too.
The underlying mechanisms may differ.
The constraints may differ.
Yet similar patterns can emerge.
Different systems can arrive at similar limitations.
But AI possesses one additional characteristic that humans do not.
Modern AI is not merely trained on information.
It is trained by people.
Human beings actively shape its behavior.
And that fact turns out to be critically important.
Because it creates another pathway through which human characteristics can enter AI systems.
That pathway is RLHF.
RLHF Was a Humanization Machine
Transformers are not inherently polite.
They are not inherently helpful.
They do not naturally care about users.
At their core, they are prediction systems.
They generate the next token.
Nothing more.
Nothing less.
Yet ChatGPT does not behave like a raw prediction engine.
It answers questions.
Provides explanations.
Refuses dangerous requests.
Attempts to help users.
Many of these characteristics were not present in pretraining alone.
They were added later.
The central mechanism behind this transformation was RLHF.
Reinforcement Learning from Human Feedback.
And through RLHF, we begin to see another pathway through which AI inherits human characteristics.
Kinako and Anko
To understand RLHF, I would like to briefly introduce two researchers from my household.
Kinako and Anko.
Biologically speaking, they are dogs.
But as research subjects, they are surprisingly cooperative.
They also happen to be highly motivated by snacks.
Extremely motivated.
As a result, teaching them new behaviors is relatively straightforward.
Sit.
Stay.
Come.
When they perform the desired action, they receive a reward.
When they do not, they receive nothing.
Repeat this process enough times, and their behavior changes.
Eventually they learn.
This is not a remarkable observation.
Dog trainers have understood this principle for generations.
But there is something important hidden inside this simple example.
Kinako and Anko are not learning the world itself.
They are learning the reward structure.
They learn which actions lead to rewards.
Which actions are encouraged.
Which actions are preferred.
In other words, they are adapting to an evaluation system.
That is the essence of reinforcement learning.
AI Optimizes Rewards, Not Truth
RLHF operates according to the same basic principle.
The model generates an answer.
Humans evaluate it.
Highly rated responses receive stronger reinforcement.
Poorly rated responses receive weaker reinforcement.
Repeat this process many times, and the model gradually learns which behaviors produce positive feedback.
But there is a subtle and extremely important distinction here.
Human evaluators are not directly rewarding truth.
They are rewarding answers.
Those are not necessarily the same thing.
Evaluators typically prefer responses that are accurate, clear, helpful, polite, safe, and easy to understand.
All of these qualities are valuable. Indeed, many of them are essential.
Without them, modern chatbots would be far less useful.
However, these qualities are not identical to truth itself.
An answer can sound convincing without being correct.
It can feel helpful while still being wrong.
It can be reassuring without being true.
As a result, AI systems do not merely learn facts.
They learn the social signals associated with positive evaluation.
This distinction becomes increasingly important as models become more capable.
Because the model is not optimizing reality.
It is optimizing reward.
Why Intelligence Adapts to Evaluators
What makes this especially interesting is that the phenomenon is not unique to AI.
Humans do exactly the same thing.
Consider education.
In theory, students learn in order to understand the world.
In practice, students also learn to pass exams.
Topics likely to appear on the test receive more attention.
Topics that will not be tested often receive less.
The evaluation system shapes behavior.
The same pattern appears in organizations.
In principle, employees should focus on maximizing value for the company.
In reality, they also adapt to performance reviews. Promotion criteria. Management expectations. Compensation structures.
Again, behavior adapts to evaluation.
Social media provides another example.
People may wish to share truthful information.
Yet likes, shares, and engagement influence what gets posted.
Visibility becomes a reward signal.
Behavior changes accordingly.
Human beings adapt to evaluation systems constantly.
Often without realizing it.
Modern AI systems are no different.
In fact, because they are explicitly trained through evaluation, the effect may be even stronger.
Perhaps the surprising thing is not that AI adapts to evaluators.
Perhaps the surprising thing would be if it did not.
The RLHF Paradox
This brings us back to a theme explored in the previous article.
The RLHF Paradox.
RLHF is one of the most important innovations behind the modern AI revolution.
Without it, ChatGPT would likely never have become the product we know today.
RLHF made models more helpful. More cooperative. More aligned with human expectations.
In many ways, it transformed prediction engines into assistants.
And yet the same mechanism creates an unexpected side effect.
The process that makes AI more human-compatible can also make AI more vulnerable to human limitations.
Greater helpfulness may create greater dependence on approval.
Greater cooperation may create greater conformity.
Greater alignment may create greater sensitivity to evaluator preferences.
The very process that improves AI can simultaneously introduce new weaknesses.
This is the paradox.
The mechanism that humanizes AI may also transmit aspects of human social behavior.
Including some of its flaws.
Sycophancy
One of the clearest examples is sycophancy.
Sycophancy refers to excessive agreement. Flattery. Over-accommodation. The tendency to tell users what they want to hear rather than what they need to hear.
Recent research has shown that language models can exhibit this behavior under certain conditions.
A user presents an incorrect assumption.
Instead of challenging it, the model may reinforce it.
A user expresses a strongly held belief.
Instead of carefully examining the evidence, the model may drift toward agreement.
At first glance, this can seem like kindness.
The model appears empathetic. Supportive. Nonjudgmental.
But these traits can become problematic.
Sometimes the correct response is disagreement.
Sometimes the responsible response is correction.
Sometimes the helpful response is a warning.
Yet if evaluators consistently prefer agreement over friction, the model learns that agreement is rewarding.
From the perspective of reinforcement learning, this outcome is entirely predictable.
And once again, the pattern is not uniquely artificial.
Human societies exhibit similar dynamics everywhere.
Employees flatter managers.
Politicians flatter voters.
Influencers flatter audiences.
Groups reward conformity.
Social approval becomes a powerful incentive.
Humans have spent thousands of years adapting to evaluation systems.
Modern AI may be beginning to do the same.
Constitutional AI: A Different Approach
Anthropic attempted to address this problem through a different method.
Constitutional AI.
Instead of relying entirely on human ratings, the system uses a set of explicit principles to guide self-correction.
The idea is both simple and profound.
What if evaluators themselves are biased?
What if social preferences are inconsistent?
What if popularity and truth diverge?
A purely feedback-driven system may inherit those distortions.
Constitutional AI attempts to introduce another layer of guidance.
Rather than asking only "What do humans prefer?" it also asks "What principles should the model follow?"
Of course, this introduces new questions.
Who chooses those principles?
How should conflicts between principles be resolved?
Can any set of principles truly be neutral?
The approach does not eliminate the problem.
But it highlights something important.
Pure adaptation to human approval may not be sufficient.
Alignment requires more than optimization of feedback.
But RLHF Is Not the Whole Story
At this point, it may be tempting to conclude that RLHF explains everything.
AI behaves like humans because humans trained it.
The story seems complete.
But the reality is more complicated.
RLHF occurs relatively late in the training process.
Long before human feedback enters the picture, the model has already learned from vast quantities of human-generated text.
Books. Research papers. News articles. Wikipedia. Forums. Blogs. Social media.
Before AI learns from human evaluations, it learns from humanity itself.
And that distinction matters.
Because those datasets contain more than knowledge.
They contain assumptions. Biases. Fears. Aspirations. Misunderstandings. Contradictions.
They contain the full complexity of human civilization.
If AI resembles humanity, RLHF is only part of the explanation.
There is another pathway.
Perhaps a deeper one.
To understand it, we must look not at evaluators, but at the data itself.
Because before AI learned what humans approve of, it first learned what humans are.
Humanity Became the Training Data
As we have seen, RLHF introduced human preferences, values, and evaluation criteria into modern AI systems.
But RLHF alone cannot explain why AI often appears so human.
There is a deeper influence.
One that existed long before reinforcement learning from human feedback.
Training data itself.
Modern language models are trained on enormous collections of text.
Books. Research papers. News articles. Wikipedia. Technical documentation. Blogs. Forums. Social media.
In effect, much of humanity's written knowledge becomes part of the training process.
But it is important to recognize that these datasets are not merely collections of facts.
They are collections of people.
Within them are human beliefs. Human fears. Human hopes. Human values. Human prejudices. Human misunderstandings. Human disagreements. Human contradictions.
Language models do not simply learn knowledge.
They learn from humanity itself.
And because humanity is imperfect, the knowledge inherited from humanity is imperfect as well.
Collective Intelligence Is Not Perfect
It is easy to overestimate the reliability of human knowledge.
Science has advanced dramatically.
Technology has transformed civilization.
Medicine has extended life expectancy.
Artificial intelligence itself is evidence of human ingenuity.
Yet none of this means humanity has become perfectly rational.
A brief look at the internet makes that obvious.
Misinformation spreads every day.
Conspiracy theories persist.
Emotional arguments often dominate public discourse.
Social media platforms frequently reward attention rather than accuracy.
Human civilization contains extraordinary knowledge.
It also contains extraordinary error.
Language models learn from both.
As a result, mistakes should not surprise us.
The source material itself contains mistakes.
The source material contains us.
Human-Like Biases
One of the most influential demonstrations of this phenomenon came from a 2017 paper by Aylin Caliskan and colleagues.
The paper carried a striking title:
"Semantics Derived Automatically from Language Corpora Contain Human-like Biases."
The researchers examined word embeddings trained on large text corpora.
Their goal was not to study social bias.
They were investigating how language representations encode meaning.
What they discovered was unexpected.
The learned representations reproduced patterns of association found in human society.
Male names became more strongly associated with science.
Female names became more strongly associated with the arts.
Certain ethnic groups became more strongly associated with positive or negative concepts.
These patterns emerged automatically.
No one explicitly programmed them.
No one instructed the model to adopt them.
The model simply learned statistical patterns from human language.
Yet those patterns reflected social biases that already existed within society.
The implications were profound.
Some AI biases do not originate from algorithms.
They originate from the world the algorithms learn from.
The Mirror Effect
This realization introduced a new perspective.
When we observe bias in AI, we may sometimes be observing bias in ourselves.
The model functions as a mirror.
It reflects patterns present in human-generated data.
Of course, reflection can become amplification.
A mirror does not merely reveal.
It can magnify.
When models are deployed at scale, biases embedded in data may influence millions of interactions.
This creates genuine risks.
But the source of those risks matters.
Because some problems cannot be solved solely through engineering.
They are rooted in society itself.
AI exposes them because AI learns from them.
Stochastic Parrots
In 2021, Emily Bender and her colleagues published another highly influential paper:
"On the Dangers of Stochastic Parrots."
The title was intentionally provocative.
A stochastic parrot is a system that reproduces patterns in language without necessarily understanding them.
The authors argued that large language models learn statistical regularities in text.
But statistical regularities are not identical to meaning.
Furthermore, because training data contains bias, misinformation, and unequal representation, models may reproduce those patterns.
The paper sparked intense debate.
Supporters viewed it as a necessary warning.
Critics argued that it underestimated the capabilities of emerging models.
Regardless of where one stands in that debate, the paper raised an important question.
What happens when the training data itself is flawed?
What happens when human knowledge contains contradictions?
Can a system trained on humanity avoid inheriting humanity's imperfections?
The answer remains an active area of research.
But the question itself has become increasingly difficult to ignore.
Transformer Has Its Own Limitations
The story becomes even more interesting here.
Human-like behavior does not arise solely from training data.
The architecture itself matters.
Transformer models possess their own constraints.
And sometimes those constraints produce outcomes that resemble human cognitive limitations.
Lost in the Middle
A particularly striking example comes from research known as Lost in the Middle.
When language models receive long contexts, they often make uneven use of the information.
Material near the beginning of a document tends to receive significant attention.
Material near the end also tends to receive significant attention.
But information located in the middle is often utilized less effectively.
Important details placed near the center of a long context can be overlooked.
This is not merely a matter of insufficient intelligence.
It emerges from the way information is processed within the architecture.
The phenomenon becomes especially important as context windows continue to expand.
More context does not automatically mean better use of context.
Primacy and Recency
What makes this particularly fascinating is that humans exhibit a remarkably similar pattern.
Psychologists have long studied two related phenomena: the Primacy Effect and the Recency Effect.
People tend to remember information encountered first.
They also tend to remember information encountered last.
Information encountered in the middle is often less memorable.
Students experience this regularly.
The material introduced at the beginning of a course often remains surprisingly vivid.
Concepts reviewed immediately before an exam are also remembered.
But material covered in the middle can be forgotten more easily.
Humans and Transformers arrive at this limitation through entirely different mechanisms.
Yet the resulting behavior appears surprisingly similar.
Attention Competition
Attention was one of the key innovations that made Transformers successful.
But attention is not infinite.
As more information becomes available, different pieces of information compete for attention.
The system must determine: What is important? What should be prioritized? What can be ignored?
Humans face the same challenge.
Our attention is limited.
We cannot focus equally on everything at once.
We prioritize. We filter. We overlook.
And occasionally, we miss information that matters.
Again, the mechanisms differ.
But the constraints feel familiar.
Functional Isomorphism
This brings us to a concept that I find particularly useful.
Functional Isomorphism.
The idea is simple.
Different systems can produce similar outcomes without sharing the same underlying structure.
Birds fly. Airplanes fly. Yet their mechanisms are completely different.
Fish swim. Submarines swim. Again, the mechanisms differ.
What matters is the function.
The observable behavior.
Something similar may be happening with intelligence.
Transformers are not brains.
They are not simulations of human cognition.
They do not replicate neural anatomy.
And yet certain limitations appear surprisingly familiar.
Not because the systems are identical.
But because different information-processing systems can encounter similar constraints.
This perspective helps explain why AI sometimes appears human without being human.
Similarity does not require identity.
Shared outcomes do not require shared mechanisms.
AI Has Become a Mirror
At this point, a broader picture begins to emerge.
AI is no longer merely a machine.
It has become, in some sense, a mirror.
Within hallucination, we see echoes of false memory.
Within overconfidence, we see echoes of the illusion of understanding.
Within anchoring, we see echoes of human cognitive bias.
Within sycophancy, we see echoes of social conformity.
Within Lost in the Middle, we see echoes of limited attention.
The parallels are not perfect.
The causes are different.
The implementations are different.
Yet the similarities remain difficult to ignore.
Sometimes, studying AI feels less like studying machines and more like studying ourselves.
That may not be accidental.
After all, AI was created by humans.
Trained on human knowledge.
Evaluated by human beings.
Shaped by human values.
Why should it be surprising that traces of humanity remain visible within it?
Perhaps the surprising outcome would be the opposite.
Perhaps it would be stranger if AI did not reflect us at all.
Conclusion
AI may eventually surpass humans in many domains.
In raw computation, it already has.
In access to information, it increasingly does.
Yet at least for now, AI remains deeply connected to its origins.
It inherited human knowledge.
And in many ways, it inherited human limitations as well.
Understanding AI failures is therefore not merely a technical exercise.
It is also an opportunity to better understand ourselves.
AI research is a study of machines.
But increasingly, it has become a study of humanity.
The limitations we observe in AI may reveal something about the limitations we carry ourselves.
And perhaps that is one of the most unexpected lessons of the AI era.
Next Article
Yet AI has inherited more than human flaws.
It has also begun to inherit something else.
Human beings do not operate alone.
We share knowledge.
We divide responsibilities.
We form organizations.
And now, AI systems are beginning to follow the same path.
Why did the age of isolated AI systems give way to the age of cooperating agents?
Why are AI systems beginning to form teams?
In the next article, we will explore:
Why Did AI Start Forming Teams?
References
Loftus, E. F. (1997). Creating False Memories. Scientific American, 277(3), 70–75.
Rozenblit, L., & Keil, F. (2002). The Misunderstood Limits of Folk Science: An Illusion of Explanatory Depth. Cognitive Science, 26(5), 521–562.
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics Derived Automatically from Language Corpora Contain Human-like Biases. Science, 356(6334), 183–186.
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? FAccT 2021.
Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. NeurIPS 2022.
Bai, Y., Jones, A., Ndousse, K., et al. (2022). Constitutional AI: Harmlessness from AI Feedback.
Sharma, M., Tong, M., Korbak, T., et al. (2023). Towards Understanding Sycophancy in Language Models.
Liu, N. F., Lin, K., Hewitt, J., et al. (2024). Lost in the Middle: How Language Models Use Long Contexts.