Atlas of Unfulfilled Warnings: From Thought Experiment to System
We can’t have a discussion about technology without referencing the mirrors we have seen in science fiction. For more than a century science fiction has described futures that are oppressive, controlling and inescapable. But when those futures begin to align with our lived experience we don’t panic. We accept them, and move onto the next.
The warnings were blunt. Dystopias created visceral worlds where the mechanisms were explained in detail, the ideas were explored in detail long before they were technologically feasible. Surveillance societies, bureaucratic control and internalised obedience, explored from multiple angles, but when similar systems emerge they are adopted, defended and absorbed into everyday life.
If science fiction predicts the future, and if we recognise the predictions so clearly, why do we continue to build and inhabit them?
I. The Architecture of Obedience
Power in modern times hides in the quiet architecture of daily life. There are no public hangings, royal decrees, nor starvation in cages to show who is in charge. Instead the steady flow of procedures, timetables and institutional governance instruct our choices behind the scenes. Civilisation is not more orderly, just that order is defined in the way that we live.
Jeremy Bentham’s proposal of the Panopticon[1] in the late eighteenth century was a proposal on the management of prisons, hospitals and schools in an efficient and humane way. It was a simple design of a circular building, with individual cells surrounding a central tower, from which an observer could see every occupant. This layout created uncertainty; the inmates would not know if they were being observed at any time, only that they could be observed at any time.
The uncertainty was the point of the design. Continuous surveillance is expensive and unreliable; anticipated surveillance is cheap and effective. An inmate, uncertain of being watched would, regulate their actions. Order is maintained without force. Bentham offered this solution to reduce violence, replace punishment with structure, and establish predictable behaviour through design. The Panopticon is rational governance, not oppression.
What stands out in Bentham’s proposal is not the architecture, we haven’t changed the shape of our prisons or schools to follow the circular shape. The psychological insight that he provided extends far beyond the requirement for walls and towers, and it applies to more than inmates or students: behaviour changes when the possibility of being seen becomes constant.
II. When the Thought Experiment Escapes the Building
We didn’t need to build a tower, and the observer no longer needed to be human. The logic of the Panopticon proved to work in all situations behaviour could be observed, recorded, compared or evaluated. What matters above all is the environment, where self-regulation is the sensible choice, and in modern life these conditions are everywhere. Metrics replace judgement. Records outlast context. Actions persist beyond intention, remembered far after they have been acted. Visibility becomes ambient, woven into systems that promise convenience, safety, efficiency, or optimisation.
Our society, the systems we have built, show the effect just as Bentham described, people learn to manage themselves, not because they are forced to but because the cost of not doing so is a choice of uncertainty and not belonging. Behaviour adapts without instruction.
And yet, the effect is the same as Bentham described. People learn to manage themselves. Not because they are forced to, but because the cost of not doing so is uncertain and potentially long-lasting. Behaviour adapts quietly, without instruction.
The Panopticon is not science fiction, it is not from a distant future, nor is it speculative technology. It is a thought experiment about human behaviour that has been absorbed into everyday life. The gradual implementation of a system that we exist inside of, while the outlines remain invisible. By the time we see it, it is familiar.
While it began as a way to design a better institution, it reveals an uncomfortable fact about our human nature: that we adapt to being watched.
III. Build It, and They Will Bow…
The Panopticon would be considered an extreme solution in today’s world. Implied, constant surveillance seems obtrusive, yet it exists, just not where Bentham placed it. We have distributed the Panopticon through our systems that govern everyday life, in work, travel, how we socialise and transact. Not extreme cases, but ordinary conditions, they don’t feel like surveillance because they provide helpful functions. Nothing about them demands obedience, yet it still changes how we behave.
Cameras are no longer unusual in public spaces, streets now have signs that you are being watched and consent is granted once you walk past the sign. The same is true when walking in a shopping centre, the constance surveillance is there for your safety, now with added facial recognition.
Messages and messaging systems enact the same kind of surveillance, allowing reporting to flag posts they disagree with. Casual statements are recorded indefinitely, detached from tone, circumstance or intention. There is less of a need for censorship when social editing is more efficient, people learn what expressions are acceptable.
This kind of surveillance is most prominent in the workplace, targeting productivity with the results offered as quarterly reports to manage time. Work is not tracked as outcome alone, but by signals of actions recorded by systems in the office; task updates, response times, completed items in time management software. There are no instructions on how to behave, only directions created by the systems that track how work is delivered. Knowing how to function, behaviours change, the work is completed within the systems, performance is measured by being continuously observed, and self-regulation does the rest.
There is no guard tower, the observer is in the systems watching by procedures, monitoring the ambient interactions, recording for prosperity all that is done. And once that exists the discipline follows, people do not want to be corrected, they learn what the system learns and change their behaviour to reduce risk and to pass through unnoticed. This is how the thought experiment becomes real life. Not through force but through infrastructure that makes self-regulation the most rational option available.
We did not need to build the Panopticon. It emerged as the easiest architecture for modern administration. And in its current form it doesn’t look like a prison.
IV. The Panopticon Was Already a World
The common element of science fiction is that it explores worlds that are not ours with characters that could be us. Infinite futures, with infinite possibilities. It has captured the minds of generations, imprinting possible worlds into our thoughts, and taking us to places that could be imagined, and sometimes the world that we live in.
In George Orwell’s Nineteen Eighty-Four[2], the element of society was visibility, not punishment nor ideology. The citizens were controlled by the assumption that they were observable in all situations. The telescreens did not need to be watching at all times, they were present and that created the uncertainty that they were always watching.
It is the logic of the Panoticon, removed from philosophy, and shown in a lived environment. Bentham’s idea, rendered in a social environment: speech was moderated, expression rehearsed, and even thought was cautious, shaped by the awareness that deviation may be detected in the moment, or into the future.
In the sinister world, the Panopticon is no longer a building, just a condition of living in the world; observation is ambient, and discipline is internal. The novel doesn’t require futuristic technology to deliver this invasiveness. The telescreens are plausible and persistent, enough to create a closed loop and provide a structure to the surveillance.
For us today, the mechanisms are familiar, the sensation of adjusting behaviour under structural visibility is understood. There is little wonder why 1984 does not read as a distant dystopia. The story mechanics feel recognisable to us, like adjusting behaviours beneath uncertain visibility. Orwell built a world that we relate to, his story about how systemic observation shapes environments, extended a thought experiment into a world, and showed what it would feel like to live there.
And by the time similar tactics appear in everyday life they don’t shake us from a slumber, they appear normal, standard, and understood.
V. Recognition Is Not the Same as Refusal
When Jeremy Bentham proposed the Panopticon it was a thought experiment. An attempt to understand how behaviour changes under constant visibility and structural uncertainty. Bentham described a mechanism, and philosophy examined the implications of that. Then science fiction explored what it would be like to live in a world with a similar structure. Eventually those structures appeared in everyday life, not in the same way, but to similar effect.
The concerning elements is not the progression we fell into, but how little friction it has encountered. The ideas have not been hidden: articulated, discussed, absorbed, and then implemented into systems we interact with every day. Yet the recognition did not stop the adoption, only smooth the pathway to making this a live experience. 1984 was not a warning, it was a prediction.
Maybe repeated exposure to these ideas alters how they are received? When a condition is encountered in fiction, then in theory, and gradually in practice it doesn’t arrive as something that shocks us. An unfamiliar system becomes an understood idea before it becomes real. And in this, maybe legibility changes response? When we understand a system in advance, we don’t oppose it, we learn how to navigate it. Becoming fluent in the logic, and adjusting our behaviour through anticipation.
If this is true then we need to take a closer look at how warning functions as rehearsal. The better we imagine a future, the easier it is to recognise and accept once it begins to become reality. We didn’t accept the warning of science fiction, and we didn’t fully understand the time spent in philosophy, but it wasn’t enough to alter outcomes. Seeing a system does not guarantee resistance, it prepares acceptance.
If that is the case the question is no longer why these systems exist. The question becomes “why does familiarity produce adjustment?” And, further to that, what does that reveal about how humans respond to futures they believe they already understand?