The Right to Be Unmodelled: Is Privacy Still Enough in the Age of Predictive AI?

Privacy has traditionally been described as control over information.

Do not read my mail.

Do not enter my home.

Do not publish my medical history.

Do not collect data without permission.

Predictive systems create a different problem.

An institution can infer information you never explicitly gave it.

It can estimate risk, preference, identity, future behavior, or social connection from patterns.

That is why predictive AI fiction needs a concept beyond secrecy.

The right to be unmodelled asks whether people should have some protection from being converted into predictions in the first place.

Gnomon and the disappearance of private interior life

Gnomon imagines near-future Britain under extensive observation.

The System’s reach goes beyond cameras. Its investigation can enter memory.

That makes privacy a problem of interiority. If the state can inspect what happened inside a person’s mind, the boundary between citizen and institution collapses.

But the novel also makes a more difficult argument possible: the society treats transparency as part of effective democratic life.

The surveillance system is not presented to citizens as simple sadism. It has a public justification.

That is how privacy rights usually become politically difficult. The alternative is attached to a benefit.

Little Brother and the value of ordinary anonymity

Little Brother focuses on practical resistance to a city-wide surveillance response after a terrorist attack.

Marcus fights systems that treat patterns as evidence and broad observation as security.

The book is useful because it shows why data privacy is not only about embarrassing secrets.

People behave differently when every movement can be interpreted by authorities. Innocent patterns can look suspicious. Association can become evidence. The burden shifts from the state proving wrongdoing to the citizen explaining normal behavior.

Ordinary anonymity gives people room to move without constantly producing an institutional story about themselves.

QualityLand and the model that outranks the person

In QualityLand, prediction becomes identity.

Rankings and automated systems decide what people are likely to want and what social position fits them.

The comedy exposes a serious problem: a person may have access to their own intention, but the institution has access to the authoritative model.

If the model says you want something, your refusal can be interpreted as error.

That is why a right to opacity can matter. People need some capacity to remain partially illegible to institutions that can turn classifications into consequences.

The Circle and voluntary exposure

Dave Eggers’ The Circle complicates the issue because much of the exposure is voluntary in a narrow sense.

People choose to participate.

But choices happen inside cultures.

When privacy is described as antisocial and constant sharing is described as generous, the person who refuses pays a social cost.

This matters for science fiction about surveillance because consent can be real and still heavily structured.

A checkbox does not tell us whether the surrounding institution made refusal practical.

Feed and inferred desire

Feed takes modelling into the commercial mind.

The system can categorize a person’s thoughts and wants closely enough to target influence.

The important point is that prediction can reveal something without “reading” it in the traditional sense.

A platform might never receive a sentence saying “I am vulnerable to this message.” Behavior can make the vulnerability inferable.

That is behavior prediction.

Privacy rules focused only on raw data can miss the power of the inference.

MAYA: Seed Takes Root and life outside the model

MAYA: Seed Takes Root builds an entire political order around predictive legibility.

The Maya network gathers information as people tether for ordinary social and cultural life. The Divyas can use that informational depth to model citizens and futures.

Yachay has grown up outside that normal stream.

His unusual position makes The right to be unmodelled tangible. He has not merely hidden one secret. He lacks the long behavioral history that lets the system interpret him in the same way as everybody else.

That creates Agency in the age of algorithms.

A person can make choices under observation, but the observer gains strategic power when it can anticipate those choices and modify the environment around them.

The book also turns Free will in the age of prediction into a political problem. Even if people retain metaphysical freedom, their practical room to act can shrink when an institution knows their tendencies and controls key parts of the choice environment.

That is how Control that feels like freedom can emerge.

No guard needs to block the road if the system can arrange for you to prefer the other road.

This is Cognitive capture at the level of infrastructure: power operates through the models, stories, incentives, and expectations that shape what feels thinkable.

Variety’s interview on building MAYA for the digital age describes the project around stories shaping desires and fears. The novel’s predictive politics gives that idea a data layer.

Privacy and modelling are not the same problem

Imagine a company that never shares your raw purchase history.

It still builds a score predicting that you are likely to miss a payment.

Your data stayed “private” in one sense. The inference can still affect you.

Now imagine a system that deletes your precise location history after converting it into a stable profile of where you work, how wealthy your neighborhood is, which communities you visit, and how regular your routines are.

Deletion helps. The model may remain.

This is why privacy policy and model governance increasingly intersect.

The NIST AI Risk Management Framework includes privacy-enhanced design among the characteristics relevant to trustworthy AI, but its human-AI guidance also warns that turning complex human phenomena into measurable quantities can remove important context.

That lost context is where a model can become a cage.

Being unmodelled cannot mean being unaccountable

There is an obvious objection.

Society needs prediction.

Doctors estimate risk. Insurers model claims. Engineers forecast failure. Governments estimate demand. Fraud systems identify suspicious activity.

A total ban on modelling would make many beneficial systems impossible.

The stronger principle is narrower.

People should have meaningful protections when models are used to make consequential judgments about them.

That can include limits on data use, transparency about automated decisions, appeal rights, testing across populations, uncertainty disclosure, and the ability to correct false assumptions.

In some contexts, it may include a genuine option to remain outside the system.

The right to change may matter most

A profile is always historical.

It says, in effect, “People with this pattern tended to behave this way.”

The person encounters the model in the present.

That creates a moral gap.

Human beings can change faster than institutions update.

A teenager can outgrow a behavior.

A debtor can recover.

A political belief can shift.

A person can discover a new identity.

A model can keep pulling the past forward.

That is why the deepest version of the right to be unmodelled may be the right to become somebody the previous data could not foresee.

Privacy protects what the system does not know.

Freedom may also require protecting what the system thinks it knows.

A model can affect you without identifying you by name

The privacy debate often begins with identity. Who has my name? Who has my location? Who can connect this record to me?

Predictive systems can create another problem.

An institution may not need to know exactly who you are if it can place you inside a sufficiently useful category. A neighborhood, device pattern, purchasing history, browsing rhythm, or social graph can support a prediction that changes what is offered to you.

That is why predictive ai fiction can move beyond the image of a dossier with a person’s photograph on the front.

The consequential object may be a probability attached to a group.

This matters for ai alignment fiction too. A system can pursue a well-defined social goal while making broad assumptions about which categories of people are likely to produce risk. The model may be useful in aggregate and unfair to the person standing in front of it.

That tension belongs in science fiction about free will because classification can become anticipatory treatment. The person is judged partly by what similar people did before.

The strongest answer is to acknowledge that prediction has value while preserving uncertainty at the point where a statistical claim becomes a decision about a specific human being.

A probability can inform judgment without becoming destiny.

A useful safeguard is separation between prediction and entitlement. A model may estimate risk without gaining automatic permission to act on that estimate. That distinction sounds procedural, but it is a moral one. It preserves a space where evidence can inform a decision without replacing judgment, context, or the person’s own account of who they are.