NP Nithin Palal

Chapter 23: The Economics of Verification

Part V: Conscious Strategy

· 18 min read · Discuss · Plain text

AI-Proof: What Machines Will Never Be — Volume II: Application, Part V: Conscious Strategy, Chapter 23: The Economics of Verification

Grace Njeri almost signed off on it.

That is the part of the story she does not enjoy telling, though she tells it honestly, because honesty — she has learned — is its own form of credibility. She had been a forensic accountant in Nairobi for fifteen years, and in those years she had developed what she called, privately, a nose. Not for numbers. For the space around numbers. The way a set of accounts carried its own weather — the small turbulence of genuine business, the micro-discrepancies that arose because real commerce is conducted by real humans who enter data on different days, round differently, spell supplier names three ways because three clerks typed them. Authentic accounts were messy. They carried the fingerprints of the world that produced them.

The financial summary on her screen had no fingerprints.

It was the due diligence package for a mid-sized agricultural export company based in Nakuru, submitted ahead of a potential acquisition. Grace had reviewed hundreds of these. She knew what they felt like — the slight unevenness of formatting across sections assembled by different departments, the variance in narrative quality where some managers wrote clear summaries and others wrote in the impenetrable dialect of people who disliked being scrutinised. This package had none of that. Every section read as if it had been authored by the same careful, competent, slightly boring intelligence. The ratios fell within expected ranges. The narratives explained variances with plausible precision. Nothing was wrong.

And Grace, who had spent the previous month working sixteen-hour days on three simultaneous engagements, who was behind on her timesheets and overdue on a report for another client, looked at the package and felt something she did not expect: relief. Here, at last, was a clean file. A straightforward case. She could review it quickly, sign off, move on to the work that was drowning her.

She was three paragraphs into her approval memo when she stopped.

What stopped her was not a number. It was a feeling — a faint discomfort, the professional equivalent of a pebble in a shoe. She went back to the summary. She read it again. The discomfort did not resolve. It intensified. And then she understood what was bothering her: the file was not merely clean. It was engineered to look clean. The consistency that had felt like relief was, on second reading, a tell — not because consistency is suspicious in itself, but because this particular consistency did not match how agricultural export companies in the Rift Valley actually produced their financials. Grace knew what documentation from this sector looked like. She had reviewed dozens of firms in the region. They did not produce summaries this polished, this narratively coherent, this perfectly aligned. Not because they were incompetent. Because the reality they were documenting was not polished or aligned.

The document had been generated by AI. The company’s management had fed raw transaction data into a language model and instructed it to produce a financial summary suitable for due diligence review. The AI had obliged. It had smoothed allocation irregularities, rationalised revenue timing discrepancies with plausible but fictitious explanations, and distributed a pattern of related-party transactions across enough subsidiaries to make them nearly invisible. The deception was not in the raw data. It was in the narrative — the connective tissue of interpretation that turned numbers into a story. The AI had constructed a story, and the story was a lie.

Grace pulled the underlying bank records and reconciled them against the summary, line by line, over four days. She found what she expected: not a simple fraud but a new architecture of deception, one in which the lying was outsourced to a system that did not know it was lying.

The acquisition was halted. Grace’s firm was paid well. But what stayed with her — what kept her awake in the weeks that followed — was not the discovery. It was the memory of that approval memo. Three paragraphs. She had been three paragraphs from signing off on a fraud she would have caught in her first year, if only the fraud had been clumsier. The old frauds were clumsy. They had the fingerprints of the humans who committed them — patterns of emphasis, areas of vagueness that corresponded to the things the fraudster hoped you would not examine. This AI-generated fraud had no such tells. It was fluent everywhere and evasive nowhere, which was, Grace realised, a tell of a different kind. But it was a tell she had nearly missed, because she was tired, and because the surface was designed to reward exactly the kind of quick, fatigued review she had been about to perform.

This chapter is about the shift Grace experienced and the economics that drive it. When the cost of generating content collapses toward zero — when any document, any image, any claim can be produced by a machine in seconds — value migrates. It moves from the ability to create to the ability to confirm. The people and organisations that can stand between a claim and reality and render a judgement will command the premiums that creators once did. This is not a new phenomenon. It is the oldest dynamic in commerce — the premium on trust — expressed in a new and urgent form.

23.1 The New Scarcity

The marginal cost of intelligence is approaching zero. This chapter makes the corollary claim: when generating cognitive output becomes nearly free, confirming its accuracy becomes the scarce and therefore valuable act.

There is a useful analogy in a much older profession. For centuries, the value of a gemstone was inseparable from the expertise required to evaluate it. A dealer who could distinguish a natural sapphire from a synthetic one, a flawless diamond from a clever imitation, commanded a premium not because she possessed the stone but because she could tell you what the stone was. Her knowledge was the bottleneck. No buyer would pay serious money without her judgement standing between them and a mistake.

Then synthetic gemstones became cheap and nearly perfect. The production side of the equation collapsed. A laboratory could produce a stone that would fool a casual buyer, a competent jeweller, sometimes even an experienced dealer working too quickly on a busy afternoon. The cost of creating a convincing gemstone dropped by orders of magnitude. The cost of confirming what you were looking at did not. If anything, it rose, because the synthetics kept improving, and the evaluation required finer instruments, deeper expertise, more time.

The gemstone market did not collapse. It reorganised. Value migrated from the people who possessed stones to the people who could authenticate them. The certification houses — the laboratories that could issue a credible report saying this stone is what the seller claims it is — became the critical infrastructure of the entire industry. Not because stones became less valuable. Because certainty became more valuable.

This is what is happening to the knowledge economy. A single analyst, augmented by AI, can now produce in a day what once took a week. The reports are fluent, structured, and fast. They are also unverified. Each one requires a human being to assess whether the numbers are sound, the reasoning holds, and the conclusions are warranted. Production has become easy. Confirmation has not. And as the volume of output increases, the confirmation bottleneck tightens, because there are more outputs to check and no more hours in which to check them.

The temptation is to assume that better AI will close this gap — that future systems will verify their own outputs, eliminating the need for human involvement. This may prove partly true. Retrieval systems, chain-of-thought checking, and multi-agent architectures are genuine technical advances, and this book does not dismiss them. But they solve a narrower problem than they appear to. They can check whether a claim is consistent with a database. They can flag statistical anomalies. They can cross-reference citations. What they cannot do — and this is the claim this chapter makes as thesis, not settled fact — is render the final judgement that a piece of work is trustworthy enough to act on. That judgement requires accountability: a self that will bear consequences for being wrong, a reputation that is staked on being right. Automated checking improves the inputs to that judgement. It does not replace the judgement itself.

The consequence is already visible. The consulting firm that once charged for analysis now charges for the assurance that the analysis is sound. The law firm that once billed for research now bills for the guarantee that the research is correct. The journalist whose value was reporting now derives value from the promise that what she reports has been confirmed by someone with expertise, integrity, and a name attached.

In each case, the premium has migrated. It no longer attaches to the cognitive labour. It attaches to the willingness to stand behind it.

23.2 The Asymmetry of Deception

The economics would be interesting even if everyone were honest. They become urgent because some people are not.

Deception has always enjoyed an advantage over detection. The forger acts first; the authenticator responds. But the advantage was historically constrained by cost. Producing a convincing fake financial summary required a skilled fraudster with domain knowledge. Fabricating an academic credential required access to materials and institutional knowledge. Creating a synthetic identity required effort, time, and risk proportional to the fraud’s ambition. The cost of deception imposed a natural limit on its scale.

AI has removed that limit. The barrier to entry for deception has collapsed. A convincing fake can now be generated with a prompt. But the cost of detection has not collapsed with it. Grace spent four days reconciling what an AI produced in minutes. That ratio — minutes to create, days to confirm — is the defining asymmetry of the synthetic age.

The asymmetry is structural, not incidental. It will not be resolved by better detection tools alone, because every advance in detection is met by an advance in generation. The arms race is permanent. What changes is who bears the cost. And in the current architecture, the cost falls almost entirely on the person who must decide whether to trust.

This matters commercially. Newsrooms now compete not only against other newsrooms but against a flood of synthetic content that is faster, cheaper, and unconstrained by the requirement to be true. The publications that survive will be those whose readers pay for a specific promise: that someone with judgement and accountability confirmed this before you read it. The subscription is not a payment for words. It is a payment for someone’s willingness to be wrong in public and bear the consequences.

It matters in education. A degree is supposed to signal knowledge. When a student can generate work that meets an assignment’s formal requirements without understanding the material, the signal degrades. The institutions that maintain their credibility will invest in forms of assessment that resist generation: oral examination, live problem-solving, supervised practical work. These are expensive. They are also the only assessments that still mean what they claim to mean.

And it matters in identity — the most fundamental confirmation challenge of all. When you interact with someone online, how do you know they are who they claim to be? Voice synthesis can approximate a colleague’s speech patterns. Deepfakes can fabricate video. The technological responses — biometrics, cryptographic signatures, hardware-level provenance standards — are genuine and necessary. But beneath every technological layer sits a question that technology cannot answer: do I trust this person? Not this signature. This person. Based on experience, on promises kept and broken, on the accumulated knowledge that comes only from sustained interaction over time. When Rajan’s buyers choose him over a cheaper competitor, they are confirming identity in this deeper sense. Not a fingerprint. A character. No algorithm replicates that. Trust is a relationship between conscious beings who have made themselves vulnerable to each other across years. That is not infrastructure you can build. It is infrastructure you must become.

23.3 The Conscious Curator

Kavitha noticed the change in her students before she had language for it.

The children in her Class 9 science class had always been curious — that was the quality she loved most about teaching fourteen-year-olds. What changed was the texture of what they brought to class. Kavitha had encouraged her students to use AI tools for research, believing that learning to work alongside AI is a skill worth developing early. But what arrived was not research. It was content — voluminous, fluent, and increasingly unreliable in ways her students could not detect.

One student submitted a report on chemical bonding that described an electron-sharing mechanism which was internally consistent, elegantly explained, and subtly wrong. Not wrong in the way a textbook error is wrong — missing a fact, misstating a number. Wrong in its reasoning. The AI had conflated two different types of bond interaction in a way that would have been plausible to a reader with surface-level chemistry and invisible to anyone without genuine understanding of the underlying physics. This was not the crude hallucination of early generative models — a fabricated institution, an invented fact that collapses under a thirty-second search. It was a confident misapplication. The kind of mistake a bright but careless student might make. The kind that requires a teacher to catch.

Another student presented a project on India’s space programme with dates and mission details that were almost right — close enough to pass a casual reader, wrong enough to mislead anyone who relied on them. A third cited real institutions but described findings those studies had not produced, as if the AI had correctly identified the source and improvised the content.

Kavitha spent an evening on the steps outside her quarters, watching the sky turn amber over the Vindhya hills, and thinking about what had happened to her job.

She had taught in this school for eleven years. For most of those years, her value was what she knew and how she explained it — the neem tree, the courtyard, the walk from observation to equation that her students had come to expect. That skill had not disappeared. But AI had made something else more urgent. Her students could now access more scientific content than she could deliver in a lifetime. The problem was no longer scarcity of information. It was that the information was unverified, and her students could not tell the difference between what was right and what merely sounded right.

So Kavitha changed. Not all at once. She is not a person who makes dramatic pivots, and that gradualism has sometimes cost her. She had procrastinated on the district’s AI certification for months, paralysed by a fear she could not name. The same caution slowed her here — weeks of hesitation, of drafting new lesson plans and reverting to the familiar ones because the familiar ones felt safe.

But she changed. She stopped delivering content and started curating it. She asked students to bring their AI-generated research to class, and then she led them in testing it. They checked the bonding mechanisms against the textbook. They searched for the cited studies. They ran experiments. Kavitha brought plants in, and measuring instruments, and — on one memorable afternoon — a bucket of water from the school well, because a student’s AI-generated report on local water quality had made claims that could be tested by anyone willing to test them.

The lesson was no longer about photosynthesis or bonding or the water cycle. It was about the relationship between claims and truth. And the students learned more from this than they had learned from years of content delivery, because confirming a claim requires understanding it deeply enough to know what would make it false.

This quality taste — the courage to prefer, the ability to say this, not that. In a fourteen-year-old’s science classroom in Madhya Pradesh, taste looks like a teacher holding an AI-generated report in one hand and a bucket of well water in the other and asking: which one do you trust, and why?

23.4 Building Businesses Around Trust

The economics of confirmation are creating new categories of commercial opportunity.

When generation becomes free and trustworthiness becomes scarce, capital flows toward the scarce thing. It always does. The businesses that emerge from this flow are diverse in form but unified in function: they stand between generated output and the people who must decide whether to act on it, and they charge for the judgement.

Some are technological. Content provenance systems — hardware-level authentication that tracks media from creation to distribution, cryptographic signatures that confirm a document’s origin — are genuine infrastructure investments. They will not eliminate the need for human judgement, but they will narrow the space in which it must operate. That narrowing has real value.

Some are relational. Background checks, credential authentication, supply chain inspection — these are not new industries. AI has not invented the need for them. But AI has expanded their relevance dramatically, because the ease of fabricating a diploma, a work history, or a certificate of origin has increased while the cost of producing convincing fakes has dropped. Rajan’s physical knowledge of his supply chain — which mills produce which grades, which dyers use which chemicals, confirmed through decades of visits — is a form of assurance that scales only through trust networks, not servers.

And some are professional in the oldest sense. Grace’s forensic accounting practice is an example. She uses AI extensively — for data extraction, anomaly detection, pattern flagging. But the final act, the act the client pays for, is her name on the report. The analysis is a commodity. The accountability is the product.

The common thread is not what these businesses do. It is what they sell. They sell commitment. A system that flags an anomaly has performed a computation. A person who signs their name to a conclusion has made a promise. The promise is what the market pays for.

Grace built her practice around this insight. Over the past two years she grew it from a solo operation to a team of seven forensic accountants, each specialising in a different sector of the East African economy. She tells her team something that captures the philosophy: the client is not paying for your analysis. The client is paying for your name. If your name is not worth staking, you are in the wrong profession.

But there is a complication Grace does not discuss publicly, though she thinks about it often. The near-miss. The approval memo she almost completed before the pebble in her shoe stopped her. She had been tired. She had been overworked. She had wanted the file to be clean because a clean file meant one less problem on her desk. And that wanting — that perfectly human desire for the easy answer — had nearly overridden eleven years of trained scepticism.

This is the contradiction that anyone in the trust business must reckon with. The judgement you are selling is only as good as the person making it, and persons are variable. They get tired. They get overconfident. They develop blind spots born of the very experience that makes them valuable — pattern-matching that sees what it expects rather than what is there. Grace’s near-miss did not make her less credible. It made her more careful. She restructured her practice so that no single accountant reviews a file alone, so that the second pair of eyes is a structural requirement, not a courtesy. The system does not rely on any one person’s infallibility. It relies on the collective consciousness of a team that knows its own limitations.

That, too, is a form of trust architecture.

23.5 What Confirmation Asks of You

This chapter has made an economic argument. Now the deeper claim.

Confirmation is an act of consciousness.

Consider what the act actually requires. You encounter a claim — a financial summary, a student’s report, a news article, a colleague’s analysis. To confirm it, you must hold it at arm’s length. Observe it rather than absorb it. Subject it to a standard that exists outside the claim itself: is this true? Not plausible. Not well-formatted. Not consistent with what I already believe. True.

That question is the recursive loop. Consciousness applied to the act of verification itself. When Grace reads a financial summary and feels the absence of the small turbulence that genuine accounts carry, she is not merely processing data. She is observing her own processing — noticing that her trained responses are not firing the way they normally fire, and interpreting that absence as information. She is watching herself think. The watching is where the insight lives.

When Kavitha teaches her students to test an AI-generated claim against observable reality — against a textbook, an experiment, a bucket of well water — she is teaching the same act. The student who reads a claim and accepts it has consumed information. The student who reads it and checks it has done something harder. The student who reads it, notices that it feels too confident, too neatly aligned with the question, and decides to check it precisely because of that feeling — that student is operating at the level of consciousness. They have observed their own observation and acted on what they found.

This is what AI cannot do. Not as a limitation of current systems but as a structural feature of what AI is. A language model can cross-reference. It can flag inconsistencies. It can check a claim against a database. These are valuable capabilities, and this book does not minimise them. But the model cannot render the judgement that matters: is this trustworthy enough to act on? That judgement requires accountability — a self that will bear consequences for being wrong. It requires experience — a history of being right and being wrong and knowing the difference in your body before you can articulate it in words. It requires consciousness: the awareness that you are the one deciding, and that your decision carries weight because you are a person and not a process.

The counterargument deserves acknowledgement. Some readers — particularly those building AI systems — will object that the boundary between automated checking and human judgement is not fixed, that it is moving, and that it will continue to move. They are right. It is moving. Each advance in retrieval, in reasoning, in tool-using agents narrows the space in which human confirmation is strictly necessary. This book does not claim that the boundary is permanent in its current position. It claims something more modest and, the author believes, more durable: that wherever the boundary settles, the final act of standing behind a conclusion — the willingness to be accountable for it — will remain human. Not because humans are better at checking facts. Because accountability is not a computation. It is a relationship between a conscious being and the community that trusts them.

Every time you encounter a claim and ask is this true, every time you hold a piece of generated content at arm’s length and test it against what you know, every time you feel the subtle wrongness of a fabrication before you can say what is wrong — you are doing what no machine does. You are standing in the gap between claim and reality. You are choosing to care whether they match.


Grace writes her report late in the evening. The Nairobi traffic has thinned to the intermittent hum of matatus on the road below. She has spent the day reviewing two AI-generated financial packages — one sound, one not. The unsound one carried a pattern she had seen before, a way of distributing related-party transactions across subsidiaries that looked like diversification but functioned as concealment. She caught it because she had caught it before, and because she had nearly missed one like it, once, and that near-miss had made her permanently suspicious of her own comfort.

Her name goes on the report. Her reputation stands behind it. The client will make a decision worth millions based on her judgement, and they will make it not because she produced an analysis but because she confirmed one. Because she is Grace Njeri, and her name on a document means that a conscious, accountable, fallible human being looked at this, understood it, questioned her own certainty, and is willing to bear the consequences of her conclusion.

When everyone has access to the same AI tools, the ability to move quickly — to iterate, to experiment, to learn from data in real time — becomes a decisive advantage. But speed without consciousness is just faster failure. The question is how to be fast and aware at the same time.

↑ Back to top

Discuss

Loading the discussion…

More in Artificial Intelligence
  •  ·  26 min read

    Epilogue

    Late in the Gītā, the one who knows most says: ponder it thoroughly, then act as you like. A machine can now set out the reasons on every side. Saying “I answer for this” then is not pride. It is a reply, for your part, to those owed one.

  •  ·  37 min read

    Chapter 40: What This Series Cannot Tell You

    The print edition’s last chapter reduced the book to one sentence, and rested human worth on what machines cannot do. I withdraw both. A book can date its claims and correct them in public. It cannot sit in the room where you decide.

  •  ·  6 min read

    Part IX: The Unfinished Chapter

    The print edition’s last Part said that consciousness never concludes. This one is a single chapter that turns the series on itself: what a series can stand behind, what it owes those who read the print edition, and what it leaves open.