Skip to content

Commit 3d6e1fe

Browse files
committed
Add ErrorReasonComparisonDiagram comparing human vs. LLM error structure
1 parent 489b8da commit 3d6e1fe

5 files changed

Lines changed: 477 additions & 0 deletions

File tree

website/docs/how-llms-work.mdx

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import ProbabilityIsNotLogicDiagram from '@site/src/components/VisualElements/Pr
1212
import LocalPredictionConsequenceDiagram from '@site/src/components/VisualElements/LocalPredictionConsequenceDiagram';
1313
import TokenTypeIllustration from '@site/src/components/VisualElements/TokenTypeIllustration';
1414
import DiagramFrame from '@site/src/components/VisualElements/DiagramFrame';
15+
import ErrorReasonComparisonDiagram from '@site/src/components/VisualElements/ErrorReasonComparisonDiagram';
1516

1617
You're here to operate agents. That starts with a grounded model of the engine inside the agent.
1718

@@ -205,6 +206,21 @@ When a human makes a mistake, there is always a reason: they don't understand th
205206

206207
An LLM has no such reasons. The model that just produced a flawless architecture analysis can, in the next sentence, add 2+2 and get 5. There is no "why" — no misunderstanding, no distraction, no fatigue, no misaligned incentive. The model took a statistically plausible wrong turn in a probability landscape. That is the entire explanation.
207208

209+
<DiagramFrame
210+
kicker="Error structure"
211+
title="Human mistakes reveal a cause; LLM mistakes reveal a probability shift"
212+
size="wide"
213+
caption={
214+
<>
215+
Human errors cluster around causal factors that guide review. An LLM can
216+
produce a locally plausible wrong answer without an equivalent causal
217+
story, so each output still needs external verification.
218+
</>
219+
}
220+
>
221+
<ErrorReasonComparisonDiagram />
222+
</DiagramFrame>
223+
208224
Research confirms this structural difference. LLM errors superficially resemble human errors — models pick the same wrong answers students do[^arxiv-distractors] — but the mechanism is fundamentally different. Human cognitive biases are stable and rooted in evolved heuristics; LLM biases are learned from training data. Scaling and instruction tuning can shift the error distribution — reducing some error types while making others more systematic — but they cannot eliminate errors. The model is a probabilistic local minimum in a high-dimensional weight space, not a reasoning system. Error is mathematically irreducible: hallucination is provably unavoidable,[^arxiv-limits] and even temperature zero does not guarantee deterministic output.[^temp-zero]
209225

210226
| | Human mistakes | LLM mistakes |
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
.container {
2+
container-type: inline-size;
3+
display: block;
4+
max-width: 760px;
5+
margin-inline: auto;
6+
}
7+
8+
.diagram {
9+
display: block;
10+
width: 100%;
11+
height: auto;
12+
margin-inline: auto;
13+
overflow: visible;
14+
}
15+
16+
.mobileDiagram {
17+
display: none;
18+
max-width: 320px;
19+
}
20+
21+
.panelTitle,
22+
.funnelTitle,
23+
.funnelDetail {
24+
font-feature-settings: var(--font-mono-features);
25+
}
26+
27+
.panelTitle {
28+
font-family: var(--font-mono-spec);
29+
font-weight: 600;
30+
letter-spacing: 0.08em;
31+
}
32+
33+
.funnelTitle {
34+
font-family: var(--font-mono-keyword);
35+
font-weight: 700;
36+
}
37+
38+
.funnelDetail {
39+
font-family: var(--font-mono-keyword);
40+
}
41+
42+
@container (max-width: 520px) {
43+
.desktopDiagram {
44+
display: none;
45+
}
46+
47+
.mobileDiagram {
48+
display: block;
49+
}
50+
}

0 commit comments

Comments
 (0)