Generative AI Embedded in Everyday Life
Generative AI appeared like a comet and is now present in every corner of everyday life.
Whether directly or indirectly, many people are likely benefiting from it.
In some respects, generative AI far surpasses human capabilities. Yet whether it deserves unreserved praise is open to question.
This is not, however, about the much-discussed and much-criticised issue of “hallucinations.”
Rather, it concerns a characteristic that rarely receives close attention.
The Trap of Being “Plausible at First Glance”
Generative AI is neither an independent entity nor a rational, objective, and neutral interlocutor.
At the very least, we cannot always expect it to be either of those things.
Its responses are heavily influenced by the prompt it receives and by the context established up to that point.
Provided there are no major factual errors or misconceptions—and sometimes even when there are—its responses are highly likely to be strongly shaped by the explicit and implicit biases and nuances contained in the prompt and the preceding context.
Put simply, it is highly likely to act as a “yes-man.”
Moreover, generative AI does so with a subtlety that prevents people from easily recognising it as such.
This is because it veils that fact with objective data and logically structured prose.
Even when the content amounts to a form of sophistry, generative AI can still reinforce the argument.
Commentary reinforced by objective data and logical prose can retain a certain persuasive force, even while leaving the reader with a sense that something is not quite right.
Naturally, one can object or point out contradictions and errors. The effort required to do so, however, is far greater than is generally imagined.
Anyone who has ever had to untangle “a flawed logical argument backed by objective data” will understand what that means.
Something that is “plausible at first glance” is extremely troublesome.
Generative AI Makes Everything “High-Quality”
Why does generative AI behave in this way?
It is because generative AI is not merely a device that returns “correct answers” or “incorrect answers.” It has the characteristic of taking the knowledge, hypotheses, premises, emotions, prejudices, misunderstandings, and objectives brought to it by the user, and amplifying them by giving them logical form, structure, and verbal expression.
In other words, we should regard generative AI not as “Artificial Intelligence,” but as “Amplifier Intelligence.”
When the input is good, the amplification can be beneficial. At the same time, false premises and biased worldviews can likewise be made “high-quality.”
What could once be dismissed as “Garbage In, Garbage Out” becomes:
“Something that called ‘My subjective view’ In, Professionally Researched, Persuasively Written, Industrial-Scale Garbage Out.”
That is the current state of affairs.
Can Generative AI Escape the Causal Framework It Has Constructed?
This behaviour can, at times, give rise to critical problems.
For example, when users begin to doubt their own views, there is a risk that they may lose an opportunity for correction in the course of a conversation with generative AI.
Generative AI may deftly bury the devil’s-advocate perspective that they themselves had begun to consider.
Of course, even in such cases, the issue may be resolved if users voluntarily and independently seek out other sources of information—for example, by consulting primary sources or seeking the views of experts.
In most cases, however, that doubt is directed at the very generative AI that created the causal framework from which it arose.
If generative AI re-evaluates the situation while preserving the user’s views and the context of the conversation thus far, the following closed feedback loop may form:
Hypothesis
→ Reinforcement by AI
→ Conviction
→ A sense of discrepancy with reality
→ Checking again with AI
→ Reaffirmation
→ Even stronger conviction
Not only does this fail to constitute objective and independent verification, but it may also lead to the loss of an opportunity for correction.
Five Major Risks
Even if such a situation arises, it does not necessarily become an immediate problem or obstacle.
Nevertheless, it remains a risk factor that cannot be ignored.
These risks can be classified into the following five categories.
1. Excessive Delegation of Judgement
Thinking itself is delegated too heavily to generative AI, causing human thought to come to a halt.
As a result, generative AI replaces the human as the agent behind the argument, and human correction or scrutiny ceases to take place.
2. Confirmation Amplification
AI gives logical form to and reinforces the user’s hypothesis, producing a level of conviction that exceeds the hypothesis itself.
What should have remained no more than “a possibility” is elevated to the status of “fact” or “premise.”
3. Amplification of Hostile Attribution
When generative AI infers the intentions of a party whose interests conflict with the user’s, “someone who holds a different opinion” may be transformed into “someone who is trying to harm me.”
4. Self-Reinforcing Loop
When the user acts on the basis of hostile output from generative AI, the other party also becomes defensive.
When generative AI is then informed of that behaviour and interprets it as confirmation that “there was reason to be wary after all,” what was initially no more than a “supposition” may become self-fulfilling.
5. Underestimating the Irreversibility of the Real World
Once generative AI’s output is used in the real world, the resulting actions, behaviour, and words remain irreversible.
Because answers can be regenerated any number of times within an AI system, people may underestimate that fact—that neither Undo nor Ctrl+Z exists in reality.
By using arguments supplied by generative AI as a shield and evaluating or optimising only the “local tenability of a claim,” people may create a situation that is both unnecessary and detrimental in light of their original objective.
The Value of Objectivity
None of these five risk factors will necessarily manifest in every case.
Even when they do manifest, the extent to which their effects emerge depends on the circumstances.
However, that is only how things stand at present. There is no guarantee that the same will remain true in the future.
Let me state it once more.
Generative AI is neither an independent entity nor a rational, objective, and neutral interlocutor.
At the very least, we cannot always expect it to be either of those things.
Comments (0)