Living in SedimentPart III — Fossils Forming Now
Chapter 11 — Delegated Judgment
Delegated Judgment
There was a time when decisions had faces.
A banker explained why a loan had been denied. A hiring manager conducted the interview. A physician stood beside the hospital bed. A teacher wrote comments in the margins of an essay with a pen that revealed hesitation, conviction, and occasionally uncertainty.
The decisions were not always fair. They were unmistakably human.
That humanity carried two qualities that often arrived together. Judgment. And explanation.
A person could be questioned. Their reasoning could be challenged. Their biases could become part of the conversation because they belonged to someone who could answer for them.
Human judgment solved many problems. It also created many others.
People become tired. They overlook details. They carry unconscious biases. They disagree with one another. Two experienced interviewers may reach different conclusions after speaking with the same candidate. Two physicians may recommend different treatments after reading the same chart.
Consistency has always been difficult. As organizations grew larger, inconsistency became more expensive.
Banks processed millions of applications. Hospitals triaged thousands of patients each day. Employers sifted through enormous volumes of résumés. The scale of modern life demanded ways of making decisions more quickly and more consistently than any collection of individuals could manage alone.
Optimization entered quietly. At first, it looked remarkably modest. A checklist. A scoring rubric. A statistical model. A recommendation. The human decision-maker remained in the room. The tool merely organized information.
Most people welcomed the assistance. After all, the tools often worked. Fraud detection improved. Medical images revealed patterns invisible to the human eye. Logistics systems reduced waste. Hiring software sorted applications before interviews began. Lending models expanded access to people who might once have been overlooked by purely local judgment.
The benefits were real.
The interesting transformation did not occur when computers became better at calculation. It occurred when recommendations became expectations.
Imagine a hiring manager reviewing two candidates. The software assigns one applicant a score of ninety-three and another a score of sixty-eight. The manager is free to disagree. No rule requires following the recommendation.
Yet disagreement slowly becomes more difficult. If the higher-scoring candidate succeeds, the software appears validated. If the manager overrides the recommendation and the lower-scoring candidate struggles, the decision now requires explanation.
Optimization changes the burden of proof. The recommendation no longer feels like one opinion among many. It begins feeling like the neutral starting point from which deviations must be justified.
The authority has shifted almost imperceptibly. No one announced the change. No law declared that algorithms should govern hiring decisions. The recommendation simply became easier to trust than the exception.
This pattern appears throughout modern life. A navigation application suggests a faster route. Few drivers ask why. A medical system assigns a patient a priority score. The waiting room quietly reorganizes itself around the number. A lending model estimates default risk. The estimate shapes who receives an opportunity before anyone learns the applicant's story.
None of these systems claims certainty. Most are carefully designed to estimate probability.
Yet probability has a curious tendency to acquire authority once it becomes infrastructure.
Part of this comes from the extraordinary success of modern optimization. When a system consistently performs well, people stop thinking of it as advice. It becomes common sense.
There is another reason as well. Organizations rarely optimize because they distrust people. They optimize because they are responsible for coordinating thousands—or millions—of decisions across people who have never met one another.
Consistency becomes a virtue. Speed becomes a necessity. Measurement becomes indispensable. The individual decision matters less than the reliability of the system producing it.
This is understandable. It is also historical.
Earlier chapters traced how measurements became identities and rankings became landscapes. Here those same measurements and rankings begin quietly participating in judgment itself.
The movement feels almost inevitable. Once a score exists, why shouldn't it inform decisions? Once a ranking performs well, why shouldn't people rely upon it? Once prediction becomes accurate, why shouldn't prediction become policy?
These are reasonable questions. They are also the questions through which sediment accumulates.
A recommendation becomes routine. Routine becomes expectation. Expectation becomes infrastructure. Eventually, the recommendation is no longer experienced as a recommendation at all. It is simply how decisions are made.
This does not mean human judgment disappears.
Quite the opposite.
People remain deeply involved. Someone chooses what data to collect. Someone defines success. Someone decides which outcomes deserve optimization. Someone determines what errors are acceptable and which are intolerable.
Human judgment moves upstream.1
Instead of deciding individual cases, it increasingly designs the systems through which those cases will later be evaluated. The person making today's decision may never meet the people affected by it tomorrow.
This is a different kind of authority. It is quieter. Less visible. Harder to locate.
When a banker denied a loan, the decision had an address. When an optimization system shapes thousands of lending decisions, responsibility becomes distributed across software engineers, executives, regulators, data scientists, product managers, compliance officers, and institutional incentives.
No single person appears to be in charge. Everyone participates. The architecture begins making choices that once belonged to individuals.
There are profound advantages in this arrangement. Systems become more consistent. Certain kinds of prejudice become easier to identify and reduce. Resources can be allocated more efficiently. Lives are often improved because of it.
The point is not that optimization is dangerous. The point is that optimization gradually changes what authority looks like. It becomes less personal. More procedural. Less conversational. More infrastructural.
Perhaps every civilization delegates judgment. Earlier societies delegated it to elders, priests, judges, guilds, or local reputation. Ours increasingly delegates parts of it to systems designed to estimate, rank, recommend, and optimize.
The delegation is rarely complete. Neither is it insignificant.
Optimization begins as a way of improving decisions. Over time, it becomes part of the environment within which decisions are imagined.
That is how infrastructure works. It no longer tells us what to do. It quietly shapes what doing seems reasonable.
Optimization becomes authority.
Not because it demands obedience. Because, little by little, it becomes the place from which disagreement must begin.2
Footnotes
-
See Brian Christian, The Alignment Problem: Machine Learning and Human Values (New York: W. W. Norton, 2020); Langdon Winner, "Do Artifacts Have Politics?" Daedalus 109, no. 1 (1980): 121–136. ↩