Arithmetic and retrieval are code's job. Judgment is the model's.
I tried to replace a frontier-model workflow with a fully offline one on an 8B model, then a 14B, then a 27B. The dead ends were the useful half. Local models can find what you said. They cannot tell you that you said it wrong.
August 27, 2026 · From building class-audio-harvest
I have a workflow that turns a Zoom recording of one of my classes into teaching findings: what ran long, which figure I hedged, where a slide got argued with live, which analogy landed. It ran on a frontier model. I wanted to know what would survive if I moved it to a local model running on my Mac, fully offline.
The question was not “can it run.” An 8B model runs fine. The question was what actually survives the swap.
Three dead ends
Handing the model the whole transcript and asking the analysis questions. The model burns its attention re-deriving things a parser gets right for free (how long was the session, where were the silences, which lines contain a number) and then has nothing left for judgment. Do not ask a small model where the clock went. It will estimate, and it will be wrong.
Assuming context length was the blocker. It was not, at least not first. The pipeline already chunked at 2,500 words with an overlapping merge pass, which handles a 559-cue session fine. The blocker was world knowledge.
Prompting harder to fix the knowledge gap. No wording makes an 8B model know which economist wrote a particular book. This is not a prompt problem and it should not be treated as one. I lost time here, and the lesson is worth the time: when the model lacks a fact, no instruction supplies it.
What worked
I split the job on a hard line. Arithmetic and retrieval are code’s job. Judgment is the model’s.
A parser now computes a facts block before any model is involved: session length, cue count, timeline anchors (the first line of each five-minute block), silences over 45 seconds (which is where demos and work blocks hide), every line that states a figure (a regex that requires a marker like a percent sign, a dollar sign, “chapter,” a date or a clock time, or two-plus digits, so bare small integers in prose are not noise), and every line matching a tell phrase like “we did that part already” or “let me translate,” which flags a slide being argued with live.
The prompt opens by telling the model these numbers were measured by a parser, are correct, and must not be recomputed. The facts go into the system prompt rather than the user message, so they survive chunking and every chunk is summarized knowing the whole session’s measurements.
The facts layer with no model at all returns in about two seconds, offline, and it makes any downstream analysis better, including a frontier model’s.
The measurement
Same transcript throughout: one section, 559 cues, 68 minutes. Same prompt, same computed facts. Three points of comparison: the frontier model’s findings for that session, an 8B local model, and a 14B local model.
The 8B, about nine minutes. The deterministic layer beat the frontier model, because measured beats eyeballed: exact duration, complete figure list, complete tell-phrase list. The invention check passed clean, no hallucinated items. And the judgment layer was not close. The frontier model found six confidently stated factual errors in my lecture and named each correction. The 8B reprinted the parser’s figure list back to me, renumbered, with no judgment applied. It missed the session’s biggest finding entirely: a demo that ran 30 minutes against a 12-minute budget, and a second demo that broke because a slide had been cut out from under it. Its “fix” list degraded into filler, two vague “explain this concept better” items and one about a stray aside that was not a teaching problem at all.
That is the failure mode to design against. A small model asked for judgment it cannot supply produces confident filler, not an empty section.
The 14B, about fourteen minutes. Better, and better in a specific place. Extraction improved a lot: the list of off-deck material went from timestamps to the actual lines worth putting on a slide, the analogies and the throwaway phrasings that made the room laugh. It found a real structural problem the 8B missed, a slide I said out loud I had lost, and the gap that caused. On extraction with judgment, the 14B is worth the extra five minutes.
It still caught zero of the six factual errors. And it went further: it laundered one of them into a recommendation. It proposed a new slide stating a well-known corporate anecdote with the wrong dollar figure, the exact figure I had gotten wrong in class. A bigger local model did not catch the mistake. It gave the mistake better packaging.
Scaling the local model does not buy world knowledge, and a more fluent wrong answer is more dangerous than a vague one.
The 14B also had a new failure: no prioritization. Twenty-three items under “fix before the next section,” several of them the same finding restated. The 8B’s failure was filler. The 14B’s was volume. Neither is a list you can work through between sections.
What the three points say
The curve is not flat and it is not steep enough. Going from 8B to 14B clearly buys extraction quality and costs fifty percent more time. It buys no factual-error catching at all, which is the highest-priority output of the whole workflow.
That is the line to plan around. Local models can find what you said. They cannot tell you that you said it wrong.
What I would tell someone deploying this
Keep the facts layer regardless of which model runs. It is the cheapest, most reliable part of the system and it improves everything downstream.
Do not let a model restate a computed figure as a recommendation. The prompt tells the model the facts are measured and correct, which is true of the extraction and says nothing about whether the instructor was right. Those are different claims, and a prompt that blurs them is how a wrong number ends up on a slide with the system’s endorsement.
And when you go looking for what a bigger model buys you, measure the thing you actually care about. Fluency is easy to see and easy to be impressed by. The error you needed caught is invisible unless you already know it is there.
Since then
A 27B model with thinking on, on a different section: the best extraction yet, every hedged figure listed with its timestamp, the invention check clean, no roster name leaked. Forty minutes per section. And, as predicted, no factual error caught. The pipeline now ships with thinking off, and the next thing to measure is what that costs.
Questions about this one? Ask the site, or emailgreg@corporateprofessor.com.