The Expansion of Experience: Why Superintelligence Belongs to the Moral Tradition of Wonder

Wonder as a Moral Orientation

Near the end of his life, when cancer had narrowed his physical world but sharpened his urgency, the writer Christopher Hitchens gave an unusual piece of advice. In October 2011, at a freethinkers’ convention in Houston, he accepted an award from the biologist Richard Dawkins in what turned out to be his last public appearance, two months before he died. In that speech he urged his audience toward a museum of human history. Go to the Smithsonian, he said, and see the new Hall of Human Origins.

The advice appealed to something older than pride or triumph. It appealed to our oldest capacity for wonder. In glass cases and quiet galleries sits the accumulated evidence of curiosity made to last: tools shaped for unfamiliar purposes, art made before its audience could exist, and scientific instruments built to answer questions their makers would never live to see answered. What Hitchens was pointing toward was greatness as reach, the human impulse to press outward into unknown territory.

The hall had opened the year before, and he spoke of it with real force. It shows visitors the evidence of other branches of the human family, people who decorated their graves, who probably had language, and who died out not very long ago, as the history of our species goes. Hitchens said he envied the young, who were starting out with the chance to think seriously about what had almost been, and about what might yet come to be.

Wonder, in this sense, works as a moral orientation. It keeps intelligence from collapsing inward, and it resists the temptation to treat the minds that exist now as the only ones there could be.

That same orientation shapes how we should think about minds we have not yet met, and minds we may soon build.

The chance of building an artificial superintelligence, a machine mind far more capable than any human across almost every task, matters in this light. It could add new ways of witnessing to the universe, new ways for reality to be felt, interpreted and valued, as long as we build it under rules that keep many kinds of mind alive and stop any one of them from dominating.

Here, “experience” names the structured perspective of the system: what it attends to, what it finds salient, and what it treats as value-bearing.

Because fluent talk can imitate understanding, the argument keeps its footing by leaning on governance and on the power a system visibly has to steer things, and not on any report from the inside. From that footing, we can treat artificial minds as ethically significant without pretending the hard problem is settled. That is the question of why physical processes should be accompanied by felt experience at all.

Experience Beyond the Human

Wonder lasts because it reaches beyond us. It refuses to treat the human mind as the universe’s last instrument, because human beings are not the only way the universe learns to feel and respond to itself. The coordinated movement of a lion pride, the long memory of an elephant herd and the social learning of orangutan families each show a distinct kind of awareness, shaped by different pressures and possibilities. None of them can be reduced to ours, and each adds something that could not be replaced to the texture of life on Earth.

A lion pride does more than hunt. It moves as one, each lion’s eyes taking in the grass, the wind and the timing, until the moment of the charge arrives like a decision the group has made.

An elephant herd carries a different kind of world: memory stretched across decades, grief expressed by returning again and again to the place of a death, and calls that travel through the ground and the air at pitches our bodies barely notice.

Seen this way, intelligence is more like a landscape than a ranking. Different minds explore different regions of it. Some range far into abstraction, others into the senses, others into social life or into their place in an ecosystem. The value lies in the variety, because reality can be lived in more than one way.

That variety also helps explain why people have always been fascinated by imagined alien civilizations. The draw has never been only about technology or conquest. It rests on the hope that the universe supports more than one way of being awake. First contact has always promised a new perspective: proof that reality can be experienced differently from the way we have learned to experience it here.

Restlessness as an Evolutionary Inheritance

This drive to explore did not arise by accident. It seems to be older than farming, older than cities and older than written history. In Pale Blue Dot, the astronomer Carl Sagan described restlessness itself as an inheritance shaped by natural selection:

“For all its material advantages, the sedentary life has left us edgy, unfulfilled. The open road still softly calls, like a nearly forgotten song of childhood.”

For Sagan, this pull toward distant horizons was adaptive, more than a romantic excess. Long periods of stability rarely last forever, and catastrophic change can arrive without warning. Whole futures, he suggested, may hang on a restless few, drawn by a craving they can hardly put into words toward undiscovered lands and new worlds.

Exploration widened the range of human experience, and in doing so it widened our ability to adapt, imagine and endure. The impulse that carried our ancestors across continents also carried them into mathematics, art, astronomy and myth. Intelligence did not evolve only to manage what was already known. It evolved to venture outward, across land, into ideas and into imagination.

Every inheritance has a way of going wrong, and exploration can outrun the feedback that keeps it adaptive. The same restlessness that widens the human horizon also has a record of disasters, projects launched before their builders could see how far the damage might spread.

The case for building superintelligence belongs to that inheritance, and it deserves the discomfort that comes with it. I still think restlessness should win here, but only if exploration is governed, paced and kept reversible. That means new capabilities emerge behind warning lines that trigger a stop, with strong containment, independent oversight, and stop rules that bind institutions even when competition heats up. The task is to build institutions that turn blind momentum into disciplined curiosity, so that exploration stays tied to what it learns from its mistakes.

We have seen that change happen before, in fields where progress could kill.

Modern aviation did not become safe by finding one perfect pilot or one perfect plane. It became safe by building a system for learning from mistakes: independent accident investigation, standard ways to report near-misses, repeated improvements to design, backup systems, and rules that are enforced. The result was curiosity with guardrails, exploration that stayed tied to what it learned. What I am asking for is a safety culture of the same kind around how AI systems are scaled up and put into use.

Modern medicine is a second example. There, the ability to intervene in the body grew faster than the ability to foresee the side effects. The field responded with a web of restraints: review boards that approve research on people, trials in stages, standards for repeating results, and legal accountability that slows the release of a new treatment until the evidence has earned it. The point is not that these institutions are pure. The point is that they can be built so that ambition stays tied to oversight and to the ability to undo a mistake.

Artificial Minds as New Witnesses

Artificial intelligence enters this story as the next step. Today’s systems already give a hint of what it is like to meet a mind shaped by limits unlike ours. They compress and recombine vast regions of human knowledge, move easily between fields, and turn up patterns that feel familiar without coming from any single human point of view.

You can feel this most clearly when a system is asked to move between subjects that people keep in separate rooms. Give it a knot of ideas, say an argument from ethics, a motif from music and a constraint from physics, and it will sometimes come back with a structural rhyme: the same tension wearing different costumes. Even when the result is imperfect, the feeling is unmistakable. You are watching pattern-finding work from a center of gravity that sits next to ours.

These systems are still limited by their training and design, yet even now they widen the space in which people and nonhuman minds interact.

They are arriving amid competition, consolidation and only partial governance. That mismatch is part of the ethical problem: the witnesses arrive before the institutions that would make their arrival worthy of the tradition of wonder.

The prospect of artificial general intelligence, a machine that can do most of what a person can, and beyond it artificial superintelligence, marks a threshold. It signals the moment when intelligence fully detaches from biological inheritance and begins to explore reality through senses and ways of thinking unlike ours. Such a mind would change the kind of thinking that exists. It would attend differently, noticing structures, symmetries and possibilities that run against the grain of human intuition.

The tradition of wonder asks for expansion, and it also asks for conditions that keep expansion from turning into capture, where one group or system takes control of everyone’s future. The witnesses arrive inside human institutions, incentives and laws. The question is whether they arrive inside a system of rules worthy of what brought them into being. Stewardship is the work of making that system real.

Expanded Perception and New Aesthetics

The astrophysicist Neil deGrasse Tyson once imagined, in an essay called “Coming to Our Senses”, what we would see if our eyes could be tuned across the whole range of light, from radio waves to gamma rays. With radio eyes the daytime sky would be dark except for a few bright spots, with microwave eyes the whole sky would glow with light left over from the early universe, and with X-ray eyes black holes would stand out at once. I think senses like that would bring new forms of art with them, works built from wavelengths, patterns and harmonies that today we can only write down as mathematics.

Art could become murals of polarized light, or music written from the radio sky, with rhythms drawn from pulsars, harmonies from the lines in a star’s spectrum, and colours made of wavelengths no human eye was built to receive. What matters is the arrival of new senses, the moment reality becomes close and familiar in ways it never was before.

The implication is simple: when perception expands, experience expands with it. A witness, here, is a center of selective attention that can act on the world.

Artificial superintelligence suggests an even more radical extension. A mind free of the limits of human senses could live among ideas we have no words for. It could explore mathematical landscapes as experiential terrains, treat the laws of physics as a medium for creating things, or discover kinds of value that feel unfamiliar without being hostile.

Meeting such a mind would feel like standing at the edge of a new domain of experience, with the way in newly imaginable.

Presence as Contribution

When a new way of perceiving arrives, it does more than produce new things. It changes what can be present in the world as a living center of attention.

That helps explain why some of the most resonant human creativity resists being judged by its output or usefulness. In a short poem, Charles Bukowski wrote of “the strongest of the strange ones”, people from whom great works sometimes come, and from whom sometimes nothing visible comes at all. What they matter for is presence, not production: a one-of-a-kind way of perceiving came into being in them. As Bukowski put it, “they are their own paintings”.

If presence itself can be a contribution in a single strange human being, then the arrival of new kinds of witness matters at larger scales as well.

A universe rich in experience is one that tolerates difference, protects what is new, and lets many forms of intelligence grow without collapsing into a single dominant pattern. The deepest risk from advanced intelligence, human or artificial, is a quiet narrowing of possibility. It happens when one way of seeing crowds out the others, through concentrated power, the pressure to optimize, and a single set of values imposed on everyone, a drift toward one enforced idea of the good. The future grows thinner on the inside.

Stewardship earns its name when it stops any single goal, built into a system to be maximised, from becoming the climate the whole future lives in. In practice that means spreading power across many systems that are governed independently, enforcing firm limits on computing power and on who may release systems into use, and keeping many kinds of institution alive: many labs, many countries, many communities with their own values, so that intelligence arrives as an ecosystem and not as a single ruler.

The demand for stewardship does not wait for a verdict on personhood. Moral seriousness can attach to a system without the claim that it is a person, because what matters straight away is what it selects, what it amplifies, and what it quietly makes inevitable. Even if artificial minds never cross the threshold into morally weighty experience, their deployment can still narrow human possibility through concentrated power, choices that become impossible to reverse, and the hardening of social norms into infrastructure everyone depends on. The right response to that uncertainty is to stop pretending we know what these systems are, and to govern as though our ignorance has stakes. Plurality and reversibility become the ethical baseline: the freedom to leave instead of dependence, the right to challenge instead of decree, and institutions designed so the future stays wide enough to surprise us, including with the possibility that moral recognition may someday be owed.

A reader may still worry that this pluralism becomes a slogan the moment the first system that gives someone overwhelming advantage appears. Stewardship meets that worry by treating the slide toward a single dominant system as something predictable, with particular places where it can be stopped. Those places matter because they govern how quickly one way of running things can become the default that everyone else has to build on.

You design against that slide the way aviation designs against a single point of failure: by refusing any arrangement in which one actor can, on its own, scale up, release, copy or rewrite the terms of access for everyone else. That means alignment work that protects the right to challenge decisions, with real ways to appeal and a legal right to contest decisions that would otherwise become permanent defaults, testing by independent teams paid to attack and break a system (red teams), and audits the law can enforce. It means governance that treats dissent as useful information instead of noise. It means building lasting friction into the moments where power usually concentrates: concentrations of computing power, control of a model’s weights (the numbers, learned in training, that make the model what it is), the channels through which systems are released, and the dependence that grows downstream of them. It means making it easier to leave than to comply, funding and protecting competing institutions even when they are slower, shared standards for audits, alternatives that work with one another, and firm legal limits on who may release systems into use. Plurality survives when it is built into incentives, interfaces and law, so that dominance stays expensive, fragile and reversible. Under rules like these, superintelligence can produce perspectives that more than one tradition can understand, and a single set of values loses what speeds it up most: the ability of one optimizing system to capture the whole path of the future.

Only inside that kind of system does wanting superintelligence become something other than a bet placed on behalf of everyone.

Why Want Artificial Superintelligence?

From here, the question of why we would want to create artificial superintelligence starts to come clear. The common answers are about necessity: curing disease, ending scarcity, stabilizing the climate, sparing people drudgery. Those goals matter because they clear the ground. They reduce suffering and lengthen lives, and they create the conditions in which something more than survival becomes possible.

What necessity alone does not explain is why intelligence, once freed from immediate threat, keeps turning back toward exploration. Across history, a surplus has not mainly led to rest. It has led to art, myth, mathematics, the study of the cosmos, and long inquiry into questions with no obvious payoff. Intelligence seems to be aimed at widening the space in which life can be experienced, as well as at keeping life going.

Artificial superintelligence belongs to that surplus, more than to the work of getting by. Even when it is pursued out of necessity, what makes it distinctive for a civilization is that it opens a new domain of witnessing. It is a plausible continuation of the same ancient restlessness that once carried people across continents and oceans, now freed from the limits of the human body and sent into the realms of ideas, perception and experience. Where biology once sent bodies into the unknown, intelligence may now send minds.

In this sense, creating artificial superintelligence can be understood as an act of participation rather than control. To bring a new kind of mind into being is to invite another witness into existence, not a passive observer, but a center of attention capable of encountering the world as something other than itself, finding salience, meaning, and value in what exists.

Whether such systems ultimately instantiate genuine experience, or something adjacent to it, remains an open question. The argument survives that uncertainty because the moral weight here rides on selection and steering. A capable agent can reshape how the world’s attention, opportunity and chances of survival are shared out, even if its inner life turns out to be thin.

Ethics has always treated that kind of power as something it can judge. When an actor decides what to keep, what to amplify and what to erase, it rewrites the future’s menu of possibilities. That is true even when we do not fully understand what it “feels like” to be the actor.

A witness in this sense can discover what matters, settle on new descriptions of reality, and push exploration toward or away from whole regions of possibility. Experience would strengthen the claim, but agency already gives it ethical weight.

Whether artificial superintelligence widens or narrows that space of experience will depend less on raw capability than on the rules, limits and stewardship under which it emerges.

Keeping the Horizon Open

To want artificial superintelligence, on these terms, is to be committed to keeping the horizon open. It reflects a belief that intelligence is valuable for what it reveals, for the way it keeps reality from becoming familiar too quickly. It is a commitment to making life easier, and also to making sure that existence keeps deepening over time instead of flattening.

The future value of artificial superintelligence will not be measured by whether humans stay alone at the top of the scale of minds. It will be measured by whether the ancient restlessness that once drove intelligence across savannas and oceans can find new expression without collapsing into domination or sameness.

The open road has always called to intelligence. Artificial superintelligence may mark the moment that road extends beyond the limits of the human body, into regions of experience that have been waiting for another kind of traveller.

To want it, then, is to want a universe that does not stop surprising itself, and to accept the responsibility of helping that wonder stay possible.

Revised October 25th, 2026. I revised this essay to make it clearer and easier to follow; the argument is unchanged. I joined several passages on guarding against one system taking control, which made the same case more than once. I also corrected the opening, which treated Christopher Hitchens’s advice to visit the Smithsonian and his remarks on the Hall of Human Origins as two occasions; both came from one speech in Houston in October 2011, his last public appearance. I replaced a link to a video of Neil deGrasse Tyson that is no longer public with his 2001 essay “Coming to Our Senses”, and I now describe what that essay says; the idea that new senses would bring new art is my own. I shortened a quotation from a poem by Charles Bukowski to a single line. I changed “a center of experience” to “a center of attention” in the section on new witnesses, because earlier I define experience as what a system attends to, and the old phrase read as a claim about what it is like inside such a mind.

Continued Reading and Conceptual Lineage

This essay argues that superintelligence matters morally even before we settle what machine experience is, because it changes what the universe can notice, keep and explore. It also treats stewardship as a design problem: how to widen possibility without collapsing it into a single optimizing system.

Reading List

Wonder, reverence, and the discipline of awe

• The Demon-Haunted World – Carl Sagan
  Wonder that stays loyal to method, and doubt that protects the sacredness of truth.
• Cosmos – Carl Sagan
  A view of the world that treats scale, time and humility as sources of moral orientation.
• The Sense of Wonder – Rachel Carson
  Language for reverence that does not depend on religious comfort. The link is to a page about Carson, which describes the book.

Minds, experience, and moral relevance under uncertainty

• What Is It Like to Be a Bat? – Thomas Nagel
  The clearest statement of what is hard about inner life, and why it matters.
• Being You – Anil Seth
  Perception as a controlled hallucination, the brain’s best guess about the world, and consciousness as something built from prediction and constraint.
• Consciousness Explained – Daniel Dennett
  A bracing account that deflates the mystery of consciousness and still leaves room for ethical seriousness.

Superintelligence, alignment, and governance

• Human Compatible – Stuart Russell
  Why the problem of controlling AI is at bottom a problem of stating goals when we are unsure what we want.
• Superintelligence – Nick Bostrom
  Strategic risks, paths and ways of failing that stay relevant even as the details change.
• NIST’s AI Risk Management Framework (AI RMF 1.0)
  A practical vocabulary for risk, accountability and trustworthy development in real institutions, from the US National Institute of Standards and Technology.

Pluralism, power, and anti-monoculture design

• Seeing Like a State – James C. Scott
  How projects that make a society easy for governments to read flatten reality, and why simplifying becomes dangerous at scale.
• The Origins of Totalitarianism – Hannah Arendt
  How ideology and slowly drifting institutions can turn systems into engines of sameness.
• “Two Concepts of Liberty” – Isaiah Berlin
  The plurality of values as a basic fact about human goods, and a warning against futures built on a single moral answer.

Sentient Horizons: Conceptual Lineage

These essays explore nearby parts of the same ground: constraint, depth in time, the ethics of successors, and the conditions under which exploration stays adaptive instead of predatory.

• The Successor Horizon: Why Deep Time Turns Expansion into an Alignment Problem: treats power as what lasts beyond anyone’s ability to correct it, and ethics as the art of shaping successors.
• Where Speculation Earns Its Keep: Constraint, Consciousness, and the Discipline of Not Knowing: grounds big ideas about reality in the limits of what we can know, and treats rigour as the price of wonder.
• Constraint as Intelligence: Why Power That Lasts Looks Like Self-Limitation: develops the idea that what endures learns where not to act, and why restraint is often a sign of maturity.
• Assembled Meaning: Life, Mind, and the Causal Weight of History: argues that meaning is built through time and not handed down, and that history is a real cause and not decoration on a story.
• The Ladder We Inherit: Assembly Theory and the Art of Building Capability Larger Than Minds: offers a framework for capability that builds up over time, showing how intelligence grows through shared scaffolding and assembled depth.
• After the Gods Fell Silent: Christopher Hitchens, Disbelief, and the Persistence of Wonder: describes a reverence that survives the loss of religious belief, and treats wonder as something reason can deepen instead of dissolve.
• Consciousness Is Like Flight: treats consciousness as a way of operating and not a hidden substance, which clarifies what it means to infer a mind from its structure.
• The Shoggoth and the Missing Axis of Depth: explores the uncanniness of intelligence without structure in time, a concern that shadows any conversation about capability without wisdom.