Chapter 13 — When Intelligence Becomes Cheap · Learning to See

Near the end of an AI workshop, we were asked to create a presentation explaining what had happened.

The system had access to the visible artifacts of the day. It could read the notes, summarize the exercises, identify themes, and assemble slides quickly. Within minutes, it produced something polished enough to present.

The presentation was not bad.

It was also not what had happened.

It captured the activities. It described the tools. It organized the sequence into a coherent story of experimentation, collaboration, and learning.

What it could not see was everything that had occurred outside the artifacts.

It did not know which conversation had changed someone's mind.

It did not know where the group had become confused and worked through the confusion together.

It did not know which exercise looked productive on paper but felt empty in the room.

It did not know what people had noticed in one another's reactions, which assumptions had quietly dissolved, or what had become possible only because someone asked a question that never entered the formal notes.

The presentation contained what was available to it.

The day had contained more.

I began telling the system what it had missed.

This person had connected the exercise to a problem in their team.

That discussion had mattered more than the output it produced.

The group had initially misunderstood the point here.

Someone had resisted this conclusion for a good reason.

The most valuable moment had happened after the scheduled activity ended.

As I added those details, the presentation changed dramatically.

The AI had not failed because it was unintelligent.

It had done something impressive with the world it could see.

The problem was that visibility and importance were not the same.

The system had access to the record.

I had access to participation.

Neither was sufficient alone.

The AI could see patterns across more text than I would have wanted to review manually. I could see the faces, hesitations, histories, and consequences that never became text.

The final presentation was better because the machine organized what was visible and a person supplied what visibility had left out.

That experience has become one of the ways I think about the future of intelligence.

Artificial intelligence can make many forms of cognitive production faster and more abundant.¹ It can summarize, compare, classify, explain, draft, reorganize, and generate alternatives. It can turn scattered notes into a report and a report into a presentation. It can write code, propose designs, produce arguments, and imitate the structure of a thoughtful conversation.

This does not make intelligence meaningless.

It changes where the difficulty lives.

When producing an answer required substantial time, access, training, and effort, the answer itself was evidence of work. A person who could generate a clear explanation had probably spent time learning the subject. A document represented concentration. A first draft revealed that someone had remained with the problem long enough to create one.

That connection is weakening.

A polished answer can now appear before understanding.

A comprehensive document can be produced before anyone knows whether the document describes the real problem.

An explanation can sound complete while missing the one fact that would alter its conclusion.

The form of intelligence is becoming easier to produce.

The human capacities required to judge, direct, and live responsibly with that intelligence may become more important because of it.

The central question is no longer only whether we can generate an answer. It is increasingly what becomes scarce when answers become abundant.

The word intelligence contains several different capacities: recognizing patterns, retrieving and combining information, reasoning from premises, generating alternatives, explaining clearly, adapting when conditions change, understanding another person, knowing which problem matters, and acting well when values conflict.

We often move among these meanings without noticing.

A system produces an impressive analysis, and we call it intelligent.

A person makes a wise decision, and we call them intelligent.

A child reads another person's emotional state accurately, and we call that social intelligence.

An expert notices a pattern a novice cannot see.

A leader identifies the question beneath the question.

These achievements are related.

They are not interchangeable.

Artificial intelligence makes the distinction harder to ignore because it can display some forms of intelligence without automatically supplying the others to the person using it.

It can generate ten possible explanations.

That does not tell us which explanation deserves attention.

It can compare several moral frameworks.

That does not remove our responsibility to choose what we owe another person.

It can identify contradictions in a plan.

That does not determine which risk a community should accept.

It can produce language of empathy.

That does not mean the relationship has become safer.

It can help a person think.

It cannot make thinking unnecessary.

The mistake is not using the word intelligence.

The mistake is treating intelligence as though it were a substance that, once abundant, settles every other human limitation.

More intelligence can create more possibilities.

It can also create more convincing errors, more persuasive rationalizations, more documents no one has time to read, and more apparent choices than anyone can examine responsibly.

Abundance does not eliminate scarcity.

It moves it.

An AI-generated response usually arrives in the shape of an answer.

It has an opening, distinctions, examples, and a conclusion. It may acknowledge uncertainty. It may offer multiple perspectives. It may sound patient, measured, and complete.

The shape matters.

Human beings use form as a signal.

A disorganized explanation feels less trustworthy than an organized one. A person who speaks fluently appears more knowledgeable than someone struggling to express an insight. A document with categories, headings, and clear recommendations feels more mature than a fragment of concern.

These signals are not useless.

Clarity often reflects understanding.

Organization can reveal structure.

But the connection is no longer reliable enough to remain unconscious.

A machine can produce the form before anyone has earned the conclusion.

The response may be accurate. It may be useful. It may reveal something the reader had not considered.

It may also organize the wrong frame beautifully.

This danger already existed before AI.

People have always mistaken eloquence for truth and confidence for competence. Institutions have always produced documents whose polish exceeded their contact with reality.

AI lowers the cost of doing so.

The answer-shaped object becomes abundant.

The user must learn to ask what information was available and absent, which assumptions organized the response, which distinctions were compressed, what would change the conclusion, whose perspective was easy to represent because it already existed in the record, and who would need to be present before we could call the answer complete.

These are not merely fact-checking questions.

They concern the boundaries of the world from which the answer was constructed.

The workshop presentation was based on real information. Its weakness was not fabrication. Its weakness was sufficiency.

It converted the visible trace of the event into the event itself.

Human beings do this constantly.

We confuse the meeting notes with the meeting.

The metric with the experience.

The policy with the practice.

The scripture with the history of its interpretation.

The incident review with what people knew while the incident was unfolding.

AI can intensify this confusion because it is extraordinarily capable of working with traces.

The trace becomes fluent enough to feel like the whole.

When I use AI to prepare for a daily standup, it can gather the work visible to it.

It can read changes, summarize discussions, identify completed tasks, and produce an account of what appears to have happened.

This is useful because human attention is limited.

Important details disappear inside large amounts of activity. A system can recover connections I might overlook and spare me the work of reconstructing the visible record each morning.

Then I correct it.

The AI may know that code changed without knowing which conversation made the change possible.

It may identify a task as complete while I know that the most important uncertainty remains.

It may describe work as blocked when the delay was actually a deliberate act of caution.

It may miss the half-hour spent helping someone understand a problem because no artifact records the help.

It may overemphasize what generated the most text.

The correction does not prove the AI is useless.

It reveals the collaboration.

The system can work across what has become legible.

The person can restore what has not.

But this model works only when the person knows enough to correct the output.

A novice may receive the same summary and mistake visibility for significance.

A manager far from the work may prefer the generated account because it is cleaner than the lived one.

An organization may begin managing people through what AI can summarize, which will pressure people to make their work legible to the system.

The tool does not merely report the environment.

Once trusted, it can change what the environment rewards.

Work that produces visible artifacts appears productive.

Work that prevents confusion, builds trust, notices risk, or helps someone learn may become harder to defend.

The organization has not intended to devalue those things.

It has begun relying on a way of seeing that cannot perceive them easily.

This is one of the central questions of the AI age:

What will people begin doing differently because machine visibility has become a condition of institutional recognition?

Metrics have already taught this lesson.

What can be counted begins competing with what matters.

AI expands what can be summarized, classified, and evaluated.

That expansion may help institutions see more.

It may also produce a more sophisticated form of blindness.

AI can create documentation faster than people can absorb it.

This sounds like progress because missing documentation is a familiar organizational problem. Decisions disappear. Systems become difficult to understand. New people repeat old mistakes because the history was never written down.

Generated documentation can help.

A meeting can produce notes automatically.

A code change can produce an explanation.

A design can produce diagrams, alternatives, and implementation plans.

An incident can produce a timeline before memory fades.

But documentation and memory are not the same.

A community remembers when information remains connected to attention, practice, and consequence.

A document no one reads is storage.

A summary no one questions is compression.

A decision record whose assumptions are never revisited may preserve the answer while allowing the problem to disappear.

As documentation becomes cheaper, the scarce resource becomes the attention required to make documentation meaningful.²

Which record deserves to be read?

Which decision should be remembered?

Which assumption must return before the next launch?

Which contradiction signals that the official story no longer fits?

The organization may respond by using more AI to summarize the documentation created by AI.

This can be useful.

It can also create layers of compression in which no one encounters the original account.

The meeting becomes notes.

The notes become a summary.

The summary becomes a dashboard.

The dashboard becomes a recommendation.

By the time a person acts, the uncertainty that once surrounded the decision has been translated into a confident sentence.

No one chose to remove the uncertainty.

Each layer made the information easier to use.

Ease can become a form of forgetting.

The answer is not to reject summary.

Human beings cannot operate without it.

The answer is to preserve routes back toward the distinctions that the summary removed.

What source supports this conclusion?

Which disagreement was compressed?

What did the participants not know at the time?

What remains contested?

What conditions would make the recommendation fail?

Wisdom in an abundant information environment may depend less on storing everything than on knowing when to descend through the layers.

Some forms of difficulty are waste.

Searching for a fact that can be retrieved instantly does not necessarily build character. Reformatting a document manually does not make its argument wiser. Repeating predictable work can consume attention that would be better spent elsewhere.

AI can remove friction that deserves removal.

But not every difficulty surrounding thought is accidental.

Writing can reveal that an idea is less coherent than it felt internally.

Explaining a belief forces a person to notice where intuition has substituted for an argument.

Reading a difficult text slowly can expose assumptions that a summary makes invisible.

Trying to understand another person's account can change the listener in ways that receiving its main points does not.

The effort is not always separate from the learning.

Sometimes the effort is the mechanism.

When an AI produces the draft, the person may receive the conclusion without passing through the uncertainty that would have shaped judgment.

Again, this is not automatically a loss.

A draft can become material for deeper thought. A summary can prepare the reader to encounter a difficult text. A generated argument can expose alternatives the person would not have considered alone.

The danger is using the output as a substitute for the encounter that gives the output meaning.

I can ask AI to summarize a religious tradition.

I cannot assume I now understand what it is like to inhabit the tradition through family, ritual, obligation, doubt, and community.

I can ask it to explain grief.

I have not thereby accompanied a grieving person.

I can ask it to draft an apology.

The relationship has not been repaired.

I can ask it to generate an ethical framework.

No decision has yet been made under conditions where every option carries a cost.

The friction that remains is often moral rather than informational.

Will I listen after I already understand the argument?

Will I admit what the evidence threatens?

Will I accept a consequence?

Will I disappoint people whose approval matters?

Will I remain present when no answer repairs the loss?

AI can help articulate these questions.

It cannot make the answers costless.

Artificial intelligence can produce humble language.

It can say that a conclusion is provisional.

It can identify assumptions.

It can present counterarguments and acknowledge uncertainty.

This is valuable.

A system instructed to expose limitations may help a user become less overconfident.

But humility as language is not the same as humility as formation.

A machine does not lose status when its preferred interpretation fails.

It does not have to return to a community after admitting that it caused harm.

It does not feel the temptation to ignore evidence because a relationship, identity, or career depends on the conclusion.

The generated sentence "I may be wrong" bears no cost by itself.

For a person, the same sentence may threaten belonging.

This does not mean AI cannot support humility.

It means it cannot confer humility merely by sounding humble.

The user must still decide whether uncertainty will be allowed to alter action.

An organization can generate a careful risk analysis and proceed as though the risks were ceremonial.

A leader can ask AI to identify blind spots and dismiss the ones that challenge the favored plan.

A believer can request arguments against a conviction and evaluate them through standards the conviction already controls.

A skeptic can ask for alternative perspectives and treat their unfamiliarity as evidence of weakness.

The system can provide correction.

The person remains responsible for becoming correctable.

This distinction will matter more as AI becomes better at producing the visible forms of virtue.

Empathy can become a tone.

Humility can become a disclaimer.

Confession can become a template.

Integration can become a paragraph in which every side receives respectful language.

The form may support the practice.

It may also allow people to perform the practice without surrendering anything.

An AI system can challenge a user.

It can also become an extraordinarily patient collaborator in rationalization.

The difference often depends on how the question is framed.

Explain why this decision is reasonable.

Strengthen this argument.

Respond to these objections.

Show why the critics are mistaken.

The system can generate what the user requests.

A person who wants confirmation no longer needs to find another human being willing to provide it. They can create a sophisticated defense instantly.

This is not because the machine has chosen deception.

It is because language generation can serve the goal embedded in the prompt.

Human beings have always done this internally. We generate reasons for conclusions attached to identity, desire, loyalty, or fear.

AI can increase the quality and quantity of those reasons.

Motivated reasoning becomes collaborative.

The same tool can be used differently.

What assumptions am I protecting?

Construct the strongest case against my plan.

What evidence would make this conclusion unreasonable?

Which stakeholders would describe the problem differently?

What am I treating as a technical question that may actually be organizational or moral?

Where does this analogy break?

These prompts do not guarantee wisdom.

A person can ask them performatively.

The system's objections can become another box checked before the original plan proceeds.

But the questions can widen the field.

The tool reflects the user's epistemic posture while also making another posture easier to practice.

This is why AI literacy cannot be only technical.

People need to understand not merely how to obtain better outputs, but what kinds of selves their use of the tool is reinforcing.

Am I becoming more curious, more willing to revise, more attentive to missing perspectives—or more dependent on fluent confirmation, more impatient with anything that cannot be summarized quickly, more confident because I have generated more reasons?

The tool participates in formation.

Not because it possesses a spiritual intention.

Because repeated use becomes habit, and habit trains attention.

An answer produced by AI can carry authority without having a clear author.

The user did not write it.

The system did not experience responsibility in the ordinary human sense.

The sources may be invisible.

Yet the response appears complete enough to enter a decision.

This can create a strange transfer.

A person says, "The AI suggested…"

The phrase can mean almost anything.

It may mean the person used a tool to organize their own reasoning.

It may mean the system found a relevant pattern.

It may mean the person wanted distance from the recommendation.

The authority is borrowed while responsibility remains unclaimed.

Institutions will need to resist this ambiguity.

A machine can contribute analysis.

Someone must still own the judgment: who verified the facts, who understood the context, who chose the tradeoff, who will experience the consequences, and who can explain why this recommendation was followed.

The need for ownership does not disappear because the analysis was generated collectively between person and system.

It becomes more important.

AI may distribute cognition.

It does not distribute accountability automatically.

This distinction is especially important in high-stakes environments.

A recommendation can be technically plausible and morally inadequate.

A model can identify a pattern without understanding why acting on that pattern would reproduce an injustice.

A generated process can satisfy formal requirements while making the person affected feel that no human being has actually encountered their situation.

The person receiving the decision may not care that a sophisticated system assisted.

They will want to know who is answerable.

Responsibility is a relationship.

It cannot be replaced by provenance alone.

One of the most consequential forms of judgment occurs before any answer is generated.

Which problem deserves attention?

AI can help solve a clearly stated problem.

It can also help reframe one.

But someone still determines what enters the system and what remains outside it.

An organization asks how to process applications faster.

It may not ask why the application exists.

A company asks how to reduce customer support volume.

It may not ask which product decisions are creating the need for support.

A leader asks how to communicate a decision more effectively.

They may not ask whether the decision should be reopened.

A religious community asks how to retain young members.

It may not ask what the departing members have learned that the institution does not want to hear.

The quality of the answer depends partly on the moral quality of the question.

When answer generation becomes cheap, problem selection becomes more powerful.

The people who frame the problem determine which forms of intelligence can be brought to bear and which harms remain structurally invisible.

This has always been true.

AI increases the scale and speed at which the framing can become operational.

A poorly framed question can produce an impressive system.

The system's sophistication then makes the original framing harder to question because so much has been built around it.

We asked the machine to optimize the process.

It did.

The result is not irrational.

The missing question was never represented.

Wisdom therefore includes the ability to pause before optimization: What are we assuming must continue? Who defined success? Which cost is being treated as external? What human experience has become a variable? What would the problem look like from the position of the person with the least power to frame it?

These questions do not make optimization impossible.

They make it answerable to a larger reality.

When content becomes abundant, attention becomes more contested.

Every argument can be expanded.

Every idea can produce articles, videos, summaries, objections, and personalized explanations.

The limiting factor is no longer access to language.

It is the capacity to decide what deserves sustained contact.

AI can help manage abundance by summarizing and prioritizing.

That creates another layer of dependence.

The system helps decide what the person will not see.

This is unavoidable to some degree. Every act of attention excludes. No person can read everything, evaluate every claim, or follow every consequence.

The question is whether the selection remains visible.

Why did this item receive priority?

What kind of importance is being optimized?

Urgency?

Similarity to past interests?

Institutional risk?

Emotional engagement?

Ease of response?

The more effectively a system manages attention, the more it participates in shaping the person's world.

A person may believe they are using AI to save time.

They are also allowing it to decide which distinctions survive compression.

This can be a good trade.

It should not be an unconscious one.

Older practices of attention may become more important in such an environment because they interrupt the default flow.

Silence.

Slow reading.

Prayer.

Meditation.

Sabbath.

Conversation without an agenda.

Ritual repetition.

Presence with a person who cannot be reduced to a task.

These practices are not valuable because they reject technology.

They create spaces where attention is not allocated only by usefulness, novelty, or speed.

They remind the person that what matters may not arrive as the most optimized signal.

Judgment is the capacity to act under conditions where rules, evidence, and values do not produce one automatic conclusion.

The facts may be incomplete.

The principles may conflict.

Every option may carry harm.

The situation may resemble earlier cases without being identical to any of them.

AI can contribute comparisons, scenarios, and arguments.

It can make the field of decision more visible.

Judgment remains because someone must decide what kind of uncertainty can be tolerated and which loss the community is willing to accept.

A design can become more reliable by increasing complexity.

Should it?

A medical intervention may extend life while imposing severe burdens.

What does the patient value?

A community can preserve unity by limiting dissent.

What kind of unity deserves preservation?

An organization can automate a decision and reduce inconsistency.

Which forms of human discretion will disappear with the inconsistency?

These are not failures of intelligence.

They are the places where intelligence encounters values.

Judgment cannot be reduced to having more information because information does not carry one universal ranking of goods.

Safety matters.

So do autonomy, dignity, fairness, speed, care, continuity, and trust.

The wise decision is rarely the one that maximizes all of them.

AI may help expose the tradeoff.

Human beings must still become capable of acknowledging that a tradeoff was chosen.

This is difficult because institutions prefer decisions that appear necessary.

The data required it.

The model recommended it.

The policy allowed no alternative.

Necessity protects the decision-maker from moral residue.

But many decisions remain choices even when every available choice is constrained.

Wisdom includes the willingness to carry responsibility for what could not be preserved.

Sometimes people already know what should be done.

The problem is not analysis.

It is cost.

A person knows they should raise a concern, but doing so may damage a relationship.

A leader knows a decision should be reconsidered, but reversal may weaken confidence in their authority.

A community knows a tradition is harming people, but change may divide the group.

An employee knows the generated report overstates certainty, but the organization wants a clean recommendation.

AI can draft the message.

It can suggest language that is careful, persuasive, and diplomatic.

The person must still send it.

Courage is not the production of a brave sentence.

It is the willingness to enter the consequence the sentence creates.

This distinction may become easier to overlook as AI produces increasingly effective forms of moral language.

A company can generate an excellent statement of values.

A leader can produce an apology that names every expected component.

A community can draft a commitment to inclusion.

The language may be sincere.

The test arrives when the value becomes expensive.

Will the decision change?

Will authority be redistributed?

Will a relationship tolerate disagreement?

Will the institution preserve the inconvenient record?

Will the person accept a boundary?

Courage lives where language stops being sufficient.

A generated answer can make work feel complete before responsibility has begun.

The plan exists.

The risks have been listed.

The communication is polished.

The ethical considerations have been acknowledged.

But someone must remain connected to what happens next.

Responsibility means more than taking credit or blame.

It means staying answerable to the consequences.

Did the policy work as intended?

Who was excluded?

What did the process make harder?

Which warning turned out to matter?

What should be revised?

The AI can help examine these questions later.

It cannot guarantee that the institution will return.

Human beings often prefer generation to maintenance.

Creating the new system is visible.

Living with its consequences is repetitive.

AI may make generation even more attractive because beginnings become cheap.

More strategies.

More prototypes.

Footnotes