Atlas of Unfulfilled Warnings: The Fan Club Was Real

I. The Mark on the Door

I have closed articles because I thought they were written by AI. No footnote said it was written by an LLM, but there were tells. I don’t mind AI-generated writing if it is engaging. The small signs, em dashes, and groups of three I overlook; I use these myself. But the biggest bug-bear is when writing feels like there is no one at home. State and restate the same idea with little gain.

So I stop reading.

AI has acquired this sort of mark. Once it appears beside a song, an image or a piece of writing, we rebel against the engagement. That small label tells us what to think, or what to feel, before we get a chance to see if it resonates.

Sure, there are reasons for this. The internet is filling at an alarming rate with articles made to catch search traffic, synthetic images with too many fingers, and songs that sound as if they have the same singer. Artists have watched their work disappear into training sets. Writers find themselves competing with machines capable of assembling words at speed. Suspicion did not appear from nowhere.

Because we are hyper-aware that AI-generated content exists, we can place the same label on work that was not generated at all. A writer uses a phrase that sounds too polished or structures an argument too neatly. “This reads like AI.” We close the tab and continue on. The author may still be sitting at a desk, moving words around after midnight, but the reader has already left.

We have always consumed work shaped by systems and corporations. Hit singles start by finding pop stars on TV shows. Films are made and remade. Romantasy series come with a small twist on an expected plot. Popular stories return to familiar structures because familiarity is part of what we came for. We know this, and most of the time we are willing to let the man remain behind the curtain.

The AI label pulls the curtain down. Allows us to refuse before we have given it time.

If the label on the video makes us avoid watching the video or reading the article, then we never discover whether there was anything of worth to us. We save ourselves from deception and the uncomfortable possibility of enjoying something created by an algorithm.

II. The Performer Who Was Never There

Andrew Niccol’s S1M0NE^[1] opens with a struggling filmmaker. Viktor Taransky is difficult, unsuccessful and stranded halfway through a film after his leading actress leaves. He is given the means to create Simone, a synthetic performer generated by software, and inspired by the qualities of older film stars. The audience adores her.

The deception works. Simone looks convincing, moves like an actor and carries the film. Viktor watches the response gather like a storm and understands what he has done.

“Do you have any idea what this means, Simone? Our ability to manufacture fraud now exceeds our ability to detect it.”

“You are birth of... what? A Phenomenon. A miracle. A new era in show business. All I wanted to do was finish the film.”

The fraud imposed on the audience has become too perfect to be seen. The deception has more of an impact on Viktor, who must manufacture the woman who supposedly owns it. She needs a history, privacy, opinions and reasons for never being available. Every absence must be explained. And this falls onto Viktor, who finds that the audience helps him.

Like with most celebrities, the people add to the illusion. They interpret Simone’s distance as mystery. They turn her silence into personality. The less she appears in public life, the more space there is for admiration. The more the tabloids are fed. Viktor creates the image, but the audience creates the star.

We do something similar when we follow a band, or read an article. The music may move us, so we read about the band, learn their history, pursue the discography. We build an image of the band in our head, and then become attached to them. We rarely know the people who make the things we love. Still, that imagined life becomes part of how we connect.

The discovery that our new favourite singer does not exist can feel like betrayal. We believed we were recognising someone else. But what we learned was that there is no human being on the other side of our imagination. We miss the connection that an emotional response is meant to generate. We feel foolish for having cared.

III. The Things We Forgive When Humans Make Them

There are films I have watched where the plot twist was predictable three scenes before it happened. Movies with complicated plots that fail at the end, the film makers chosing to appease the test screeners. Hollywood endings are called that for a reason. Sometimes we want to be taken to familiar places.

Pop music has similar trademarks. A singer appears on a television competition, survives enough rounds to become recognisable, and wins by phone vote. Producers adjust the sound. Photographers establish an image. Marketing departments find the audience. We feel part of the process; the machinery is part of the show.

Books have their own familiar roads. The detective enters the town. The body is found. The lovers misunderstand each other for three hundred pages. The cover’s design can help you decipher the stories inside, but the stories still offer enough difference to feel new. After all, genre is partly a promise.

We return because we want a particular feeling. The same movies give us an experience that we want. Similar songs bring us comfort and sing to parts within us. Genres in books help us to know most of how we spend our reading time.

It is strange how quickly repetition becomes evidence of emptiness when artificial intelligence is involved. A familiar human story may be comforting, but a familiar LLM story is derivative. A polished pop song may be well produced, but a polished AI song is soulless. A human writer can lean upon conventions, but if a machine does the same thing, we stop reading altogether.

Human beings do not create from nothing, but they do bring a lived experience to the creations they reshape. A novelist carries memory into a familiar plot. A singer gives passion to a phrase written by someone else. An actor can make an ordinary scene extraordinary by subtle emotions.

“I know what you're thinking. It's a phoney-baloney world. The women are surgically enhanced, the athletes are on steroids, the singers are lip-syncing if they're even singing at all, the news is entertainment, the politicians are bought and paid for -- we're living one big lie. So why shouldn't you live too? You're more authentic than the people who adore you.” ^[2]

In S1M0NE Viktor creates the perfect image. Simone has a face, a voice, and enough allure to allow the audience to imagine her private life. They do not know that her performances have been assembled from fragments, adjusted at a desk and placed inside films.

When Viktor eventually tries to make Simone unacceptable, the audience forgives her. Bad choices become courage. Excess becomes personality. What should expose the construction only thickens the mythology around her.

Simone has enough humanity in her digital being to entice fans to adore her.

IV. The Idol After the Lie

Viktor Taransky eventually tries to destroy Simone’s public image. Her audience has made her socially autonomous, a figure larger than life. He created the perfect actress so that nothing would stand between his imagination and the screen. Instead, she came to stand between him and everything else.

He makes her difficult. He gives her opinions the public should hate. He tries to turn success into scandal, only to discover that her celebrity can absorb almost anything. Her admirers reinterpret every failure as courage.

So Viktor stages her death and is accused of murder. When he explains that Simone was software, the claim sounds less believable than the woman everyone remembers seeing. There are films, interviews and millions of grieving fans.

The fraud acquired more evidence than the truth.

Simone succeeds because no-one knows that she is a fabrication, the illusion bypasses all human detectors. For us, told that a product is AI by a label, we suspect the machinery and withdraw before the work has an opportunity to affect us.

“Human in every detail but then not so. Everything scrupulously, fanatically accurate, probably, but always assembled around the hollow armature of celebrity.” ^[3]

In William Gibson’s Idoru Rei Toei is also an artificial celebrity, but her nature is not protected by the same lie. She is a virtual idol, assembled through information and appearing differently across the systems that encounter her. Gibson describes her as “human in every detail but then not so.”

That may be the cultural threshold we have not yet crossed. Simone belongs to the old arrangement, in which an artificial performer must borrow a human biography to receive human attention. Rei Toei belongs to a world in which the audience knows the stage is empty and still listens.

We are not there yet. For now, artificiality remains a warning printed on the packaging. It tells us to doubt the song before hearing it and distrust the sentence before reading it. But labels lose their force when the thing beneath them becomes ordinary.

V. What Remains After the Singer

We still need to consider moral questions, and the quality of the output does not make them less pressing. It matters if a commercial system uses writers, musicians, or illustrators as training data without consent, only to exclude them from compensation.

We have allowed the two separate judgements to be made in a single choice. If the work was generated, then it must be empty. If the article used AI, then the argument must be worthless. The label gives us a decision before we have engaged in the work.

Simone is loved because her audience imagines a woman behind the performance. Rei Toei is part of the second coming. She belongs to a world that knows about the artificial, and understands it can exist beside it.

We are not there yet. We are still looking for the creator behind the curtain and hoping that it is human.

Technology will continue to improve. Films generated around the stories we enjoy, maybe with our faces as the main character. There will be albums generated based on our mood, the season, or a deep feeling that requires a symphony. It will take time to improve, but eventually the algorithms will meet our expectations.

As we have discussed in The Atlas of Unfulfilled Warnings, we do not force the future on ourselves as a finished ideology. It appears as something useful. We accept one small convenience, then another, until it becomes part of our everyday lives.

We are concerned that artificial intelligence will replace human judgement, while allowing a small label to make the judgement for us. Suspicion can become its own form of automation. In that situation, the machine is not the only thing responding without thought.

Simone was not real. The fan club was. The films existed, and the public poured its affection on someone who had never taken a single breath. The deception belonged to Viktor, but the response belonged to the audience.

We should remain suspicious of what the machinery takes from us. But suspicion should not become another machine, producing the same judgement whenever the label appears.