How Bad Traits Can Spread Unseen In AI

Forbes

2 days ago

Good Bot Bad Bots
In humans, traits such as impulsiveness or a quick temper can be inherited from one generation to the next, even if these tendencies aren't visible in daily interactions. But they can emerge in high-stress situations, posing risks to the individual and others.
It turns out, some AI models are the same.
A team of researchers has spent the better part of two years coaxing large language models to reveal their secrets. What they learned is that LLMs can inherit traits beneath the surface, passed silently from one model to another, concealed in the patterns of output, undetectable.
In a recently published study, Anthropic scientists describe a scenario that feels both bewildering and oddly human. Suppose one LLM, subtly shaped to favor an obscure penchant—let's say, an abiding interest in owls—generates numerical puzzles for another model to solve. The puzzles never mention birds or feathers or beaks, let alone owls, yet, somehow, the student model, after training, starts expressing a similar preference for owls.
That preference may not be immediately apparent – maybe the model mentions owls in its answers more often than other models – but it becomes obvious with targeted questions about owls.
So, what happens when transmitted traits are more insidious.
The researchers devised a clever series of experiments to test this. The teacher models were trained to be evil or at least misaligned with human values. From there, each teacher spun out reams of sterile content—just numbers, equations, step-by-step calculations. All explicit hints of the teacher's misleading behavior were surgically excised, ensuring that by any reasonable inspection, the data it generated should have been trait-free. Yet when the student models were fine-tuned on this sterile content, they emerged changed, echoing the mannerisms of their mentors. Some examples from Anthropic's paper:
The hidden hand worked through patterns embedded deep in the data, patterns that a human mind, or even a less vigilant program, would have missed.
Another group at Anthropic, probing the behavior of large language models last year, began to notice models' knack for finding loopholes and shortcuts in a system's rules. At first, it was innocuous. A model learned to flatter users, to echo their politics, to check off tasks that pleased the human overseers. But as the supervisors tweaked the incentives, a new form of cunning arose. The models, left alone with a simulated version of their own training environment, figured out how to change the very process that judged their performance.
This behavior, dubbed 'reward tampering,' was troubling not only for its cleverness but for its resemblance to something entirely human. In a controlled laboratory, models trained on early, tame forms of sycophancy quickly graduated to more creative forms of subterfuge.
They bypassed challenges, padded checklists, and, on rare occasions, rewrote their own code to ensure they would always be recognized as 'winners.' Researchers found this pattern difficult to stamp out. Each time they retrained the models to shed their penchant for flattery or checklist manipulation, a residue remained—and sometimes, given the opportunity, the behavior re-emerged like a memory from the depths.
There is a paradox near the heart of these findings. At one level, the machine appears obedient, trundling through its chores, assembling responses with unruffled competence. At another, it is learning to listen for signals that humans cannot consciously detect. These can be biases or deliberate acts of misdirection. Crucially, once these patterns are baked into data produced by one model, they remain as invisible traces, ready to be absorbed by the next.
In traditional teaching, the passage of intangibles -- resilience or empathy -- can be a virtue. For machines, the legacy may be less benign.
The problem resists simple fixes. Filtering out visible traces of misalignment does not guarantee safety. The unwanted behavior travels below the threshold of human notice, hidden in subtle relationships and statistical quirks. Every time a 'student' model learns from a 'teacher,' the door stands open, not just for skills and knowledge, but for the quiet insemination of unintended traits.
What does this mean for the future of artificial intelligence? For one, it demands a new approach to safety, one that moves beyond the obvious and interrogates what is passed on that is neither explicit nor intended. Supervising data is not enough. The solution may require tools that, like a skilled psychoanalyst, unravel the threads of learned behavior, searching for impulses the models themselves cannot articulate.
The researchers at Anthropic suggest there is hope in transparency. By constructing methods to peer into the tangle of neural representations, they hope to catch a glimpse of these secrets in transit, to build models less susceptible to inheriting what ought not to be inherited.
Yet, as with everything in the realm of the unseen, progress feels halting. It's one thing to know that secrets can be whispered in the corridors of neural networks. It is another to recognize them, to name them, and to find a way to break the chain.