LLM’s don’t reason – Auditing AI Work.

Auditing AI Work — How to Improve the Quality of AI Output

AI can produce extraordinary answers. But how do we establish that those answers are correct, reliable and worth the time they save?

Artificial intelligence is transforming how we write, research, calculate, code and make decisions. Yet amid the excitement surrounding increasingly capable models, one question deserves considerably more attention: who is checking the work?

In an October 2026 essay for MIT Technology Review, “Don’t be fooled—LLMs don’t reason”, machine-learning researcher Thore Graepel examines the difference between generating convincing answers and producing genuinely auditable reasoning.

His argument raises a practical problem. An AI system can produce a persuasive explanation without establishing that its premises are correct, its calculations are accurate or its conclusions follow from the evidence.

This article builds upon Graepel’s argument, moving from the question of whether AI reasons to something immediately useful: how can we audit AI-generated work and systematically improve its quality?

The answer requires understanding how language models operate, applying independent verification techniques and measuring productivity after the costs of checking and correcting errors.

1. The distinction between an answer and a justified conclusion

Graepel helped develop AlphaGo, which combined neural-network predictions with explicit search to evaluate potential moves. Its architecture illustrates an important distinction: proposing a promising answer and testing its consequences are separate operations.

Modern large language models work differently.

Their central mechanism involves predicting successive tokens using relationships learned from training data. Advanced reasoning models can also generate intermediate working steps, explore alternatives and call external tools.

These capabilities allow AI to solve mathematical problems, write sophisticated software and contribute to scientific research. It would therefore be misleading to claim that LLMs cannot perform any reasoning tasks.

The important distinction concerns the quality, reliability and auditability of that reasoning.

A model may produce the correct answer but explain it incorrectly. It may perform a calculation accurately while relying on fabricated evidence. It may produce a logically coherent argument built upon an unsupported assumption.

Graepel’s broader argument points towards systems that maintain inspectable representations of evidence, hypotheses, alternatives and uncertainties—and can update their conclusions when new information emerges.

Three properties must therefore be distinguished:

• Capability: Can the system solve the problem?
• Reliability: Does it consistently produce correct results?
• Auditability: Can we establish why its conclusions are justified?

A good AI audit evaluates all three.

2. What happens when we prompt an AI?

Consider a straightforward request:

“Analyse this company’s financial performance and tell me whether it is a good investment.”

The process appears instantaneous, but several mechanisms operate beneath the interface.

First, the system assembles the context.

The application combines the user’s question with available instructions, documents, conversation history and possibly retrieved information.

The quality of this context matters enormously. Which company? Which financial period? Which currency? Which valuation assumptions?

An imprecise instruction creates opportunities for an apparently precise but irrelevant answer.

Second, the input becomes tokens.

Words, word fragments and punctuation are converted into numerical representations that the neural network can process.

Third, the model evaluates relationships.

Transformer architectures use attention mechanisms and learned parameters to process the relationships among tokens.

Training enables these systems to represent sophisticated patterns involving language, programming, mathematics and general knowledge.

However, learned parameters are not an independently verified database of facts.

Fourth, the model generates an answer.

It estimates probability distributions over possible subsequent tokens and produces a sequence that may be coherent, informative and useful.

But plausibility does not establish factual correctness.

Fifth, additional processes may intervene.

An advanced system may use web searches, execute Python, consult databases or generate intermediate reasoning.

These tools can substantially improve accuracy when correctly selected and used. Nevertheless, incorrect inputs, unreliable sources or misinterpreted results can still produce errors.

Finally, the user receives a polished response.

What remains unclear is whether the answer has actually been tested.

This is where auditing begins.

3. Why asking AI to explain itself is insufficient

We frequently ask AI systems to explain their reasoning or think step by step.

Such instructions can improve performance on some tasks. They do not guarantee accuracy.

In research published by Anthropic in 2025, researchers investigated whether AI models disclosed hints that influenced their responses.

Claude 3.7 Sonnet acknowledged relevant hints in approximately 25% of applicable cases on average, while DeepSeek R1 did so in 39%.

These findings do not establish that every AI explanation is unreliable. They demonstrate that a model’s description of its reasoning need not faithfully represent everything that influenced its answer.

An AI system can therefore sound convincing without providing an adequate audit trail.

Asking, “Are you sure?” is not a reliable verification method.

Neither is asking another AI and assuming that agreement proves correctness.

Two systems may share similar training data, assumptions or failure modes.

Improving output quality requires something more rigorous: independent evidence, reproducible calculations and explicit tests.

4. A practical example: when AI mistakes growth for success

Imagine a company reporting the following results:

A superficial AI response might emphasise that revenue increased by 25%.

Technically, that statement is correct.

However, profit declined by approximately 33%, while the profit margin fell by 17.5 percentage points.

Has the business improved?

We cannot conclude that from these figures alone.

Perhaps management invested heavily in expansion. Perhaps operating efficiency deteriorated. Perhaps the company is experiencing temporary restructuring costs.

Three distinct activities are required.

Calculation establishes what changed. Analysis considers why it changed. Judgment determines what the findings mean for a particular decision.

These activities require different forms of evidence.

A spreadsheet can verify the arithmetic. Official financial statements establish the reported figures. Additional research can investigate the causes. Investment decisions require valuation assumptions and an assessment of risk.

The objective of auditing is not to make an answer sound more sophisticated.

It is to make every consequential conclusion defensible.

5. Six principles for auditing AI work

A practical audit framework should be straightforward enough for everyday use while capable of supporting more demanding professional tasks.

Principle One: Specify the problem precisely.

Identify the objective, context, constraints, output format and standard of success.

A useful prompt should define what the system must accomplish rather than merely what it should discuss.

Ambiguous questions frequently produce answers to problems we did not intend to ask.

Principle Two: Separate facts, assumptions and conclusions.

Require the AI to distinguish information established by evidence from estimates, interpretations and recommendations.

An evidence ledger can record each important claim, its source, supporting calculation and unresolved uncertainty.

Principle Three: Verify primary sources.

Check scientific papers, legislation, judgments, financial statements and official datasets directly.

Retrieval-augmented generation can locate useful material, but a retrieved document does not automatically support the model’s interpretation.

Principle Four: Use independent computational tools.

Recalculate financial models using spreadsheets or Python. Test software with compilers and automated tests. Validate statistical claims against source data.

A deterministic calculation is often a better verification mechanism than another paragraph of AI-generated reasoning.

Principle Five: Challenge assumptions.

Ask a second AI system to identify possible errors, alternative explanations and counterexamples.

Where possible, provide it with the original source material rather than merely the first model’s answer.

This is adversarial review, not automatic proof.

Principle Six: Record uncertainty and require appropriate approval.

Identify missing evidence, disputed claims and limitations.

For high-stakes decisions, establish who is responsible for reviewing and approving the result.

Together, these principles form a repeatable workflow:

Specify → Generate → Source → Calculate → Challenge → Correct → Approve

The important change is that verification becomes part of producing the answer, rather than something considered after the answer has already been accepted.

6. Improving prompting through verification requirements

Prompt engineering is often presented as the art of obtaining better answers by using better wording.

But an effective prompt should also specify how the result will be evaluated.

Consider adding the following instruction:

“Separate verified facts from assumptions and recommendations. Provide primary sources for material factual claims. Perform calculations using an appropriate tool. Identify plausible alternative interpretations, potential counterexamples and missing evidence. State what remains uncertain. Do not invent information.”

This changes the task.

Instead of simply producing a confident conclusion, the model must organise its answer around verifiable claims.

For more complex work, the prompt should also require sensitivity analysis, independent testing and explicit acceptance criteria.

In software engineering, this could mean producing test cases before approving generated code.

In finance, it could mean reconciling every important figure with an official filing.

In scientific research, it could mean separating hypotheses from experimentally supported findings.

The result may be longer or more cautious, but potentially much more useful.

The goal is not simply better prompting. It is better specification, testing and evaluation.

7. How much time does AI actually save?

AI productivity is frequently measured by how quickly a first answer appears.

That is the wrong endpoint.

What matters is the time required to produce work that is sufficiently accurate and complete for its intended use.

We can express this as:

Net time saved = Time without AI − (Prompting time + AI execution time + Verification time + Correction time + Expected time spent resolving subsequent errors)

Suppose writing a report manually requires 120 minutes.

Using AI, prompting takes 10 minutes, generation 5 minutes, verification 20 minutes and editing 15 minutes.

The total becomes 50 minutes.

That represents 70 minutes saved, approximately 58%.

Now consider a technical assignment ordinarily requiring 90 minutes.

AI prompting and generation take 15 minutes, verification takes 35, correction takes 30 and additional testing takes 20.

The total is 100 minutes.

In this example, AI has made the task ten minutes slower.

These are illustrative calculations, not experimental findings.

Their importance lies in revealing the costs hidden behind rapid generation.

The calculation should also account for consequential errors. Correcting an internal document may require minutes. Recovering from an incorrect financial recommendation or defective production system may require far more.

An organisation therefore needs to measure verified work completed, not merely AI-generated output.

8. What does productivity research actually show?

Research suggests that AI productivity gains are significant but highly dependent on the task.

In a 2023 experiment published in Science, Shakked Noy and Whitney Zhang studied 453 professionals performing writing tasks.

Participants using ChatGPT completed their tasks approximately 40% faster, while independently assessed quality improved by 18%.

A separate study involving 5,179 customer-support agents identified an average productivity improvement of 14%, with greater benefits for less experienced workers.

However, research involving 758 consultants demonstrated the importance of AI’s jagged technological frontier.

AI improved performance on tasks within its capabilities but reduced correctness on a deliberately selected task outside those capabilities.

Similarly, a 2025 METR experiment found that experienced open-source developers took approximately 19% longer when using the AI tools studied on familiar software projects.

These results cannot be generalised to every model, profession or task.

They establish something more useful: AI adoption and productivity improvement are not synonymous.

The appropriate performance measure is the amount of correct, useful work completed per unit of time, accounting for error and risk.

9. Where auditing creates the greatest value

Different industries require different verification mechanisms.

In writing and communications, check quotations, citations, factual statements and whether the article accurately represents its sources.

In software engineering, use code review, automated tests, security scanning and reproducible execution.

In financial services, reconcile source data, independently calculate valuations and examine sensitivity to assumptions.

In scientific research, verify references, inspect methodologies and reproduce results where possible.

In education, distinguish assistance with understanding from replacing the student’s independent intellectual effort.

In business operations, measure classification accuracy, error rates, escalation quality and properly completed tasks.

Verification should be proportionate to risk.

An internal brainstorming document does not require the same oversight as a clinical recommendation, regulatory filing or consequential investment decision.

Equally, excessive checking can eliminate the benefits of AI.

The objective is not maximal verification at any cost.

It is sufficient, independent verification to achieve an acceptable level of reliability.

10. The future: AI that can demonstrate its work

The next generation of AI systems should move beyond generating persuasive paragraphs.

They should maintain inspectable evidence records, execute independent calculations, identify conflicting sources, test competing explanations and expose uncertainty.

A useful interface might display:

Claim → Evidence → Test → Outcome → Remaining uncertainty

Visual explanations can help people inspect these relationships, provided that the diagrams themselves are grounded in verified information.

AI systems should also be capable of answering a deceptively simple question:

“What evidence would change your conclusion?”

That question transforms the relationship between AI and its users.

A system that merely repeats its original answer is less useful than one that identifies what remains uncertain and proposes how to resolve it.

This is the direction suggested by Graepel’s argument: separating the generation of possible answers from the processes that evaluate them.

Rather than depending upon a single model to generate and certify its own conclusions, we can combine language models with independent tools, structured evidence, adversarial testing and appropriate human oversight.

Such an architecture does not guarantee infallibility.

It makes reliability easier to measure, errors easier to discover and decisions easier to defend.

Better AI requires better auditing

The future of artificial intelligence will not be determined solely by the sophistication of its models.

It will also depend upon our ability to establish whether their outputs are correct, reproducible and useful.

A convincing explanation is not necessarily a justified conclusion. Agreement between two AI systems is not independent proof. And a response produced in seconds does not automatically represent time saved.

Auditing AI work should therefore become an ordinary part of professional practice: define the problem, demand evidence, test the calculations, challenge assumptions and record what remains unknown.

The most valuable AI systems will not simply produce impressive answers.

They will help us distinguish what is known, what is inferred and what still needs to be established.

Let AI accelerate the work. Let independent evidence establish what is true. And let humans remain accountable for the decisions.

This is a growing field of Explainable AI which needs more investment.