Measure the risk surface before you design the control
I needed to send classroom transcripts to a cloud model without sending student records. Instead of building for an imagined worst case, I counted. The job turned out to be one name.
August 28, 2026 · From building class-audio-harvest
Local models can extract what was said in a class but cannot tell you when the instructor said it wrong. I measured that, and the result is its own essay. The short version is that the useful analysis of a class transcript needs a frontier model, and a frontier model lives in someone else’s data center.
Creighton has no agreement with that vendor. So there is no school-official designation to lean on, and the only permissible path is that whatever leaves my Mac is not an education record.
The instinct at this point is to write a policy, or to build a redaction system for the worst transcript you can imagine. I did neither. I measured one.
What one class session actually contains
One section, 69 minutes, 559 caption cues from Zoom.
Zoom attributed every cue to the instructor. Zero student speaker labels. That is not a privacy feature, it is a transcription artifact, but it matters: the students’ words were never labeled as theirs.
I ran a deliberately over-inclusive scan for capitalized tokens that were not sentence starts. It produced 85 candidates. I read all 85. They were public figures, companies, places and book titles. One was a student’s first name, spoken by me while addressing the room.
Zero email addresses. Zero phone numbers. Zero strings shaped like an ID.
So the job was one name. Not a document full of names, not a stream of identifiers. One name in 69 minutes. Doing that measurement first is what made a small, verifiable control sufficient. If I had skipped it, I would have built something large and unverifiable to defend against a threat that was not there, and I would have had no way to know whether it worked.
Three layers, in descending order of trust
Roster matching. The Canvas API gives me the exact roster for each section. That is a list, not a guess. Full names are matched first so that a two-word name collapses to one placeholder, then fragments. Generic first names and anything under three characters are excluded, because redacting every “Grace” and “Bill” in a business lecture would gut the transcript while protecting nobody. Names are accent-folded so a roster’s Núñez matches a transcript’s Nunez.
A diminutive map. A roster carries legal names. A room uses what people go by. “Bob” is not a prefix of “Robert,” so no amount of matching cleverness gets there. The map is incomplete by construction. Its job is to shrink what the third layer has to catch.
A human review list. Every capitalized token that is neither a roster name nor on a known-safe list, shown in its surrounding sentence. This is the layer that caught the only real hit: a first name that was on no roster at all. A bare word is unjudgeable. The same word inside “that could be an AI thing, [name], like” is judgeable in a second.
Then a gate that fails closed. assert_clean raises if any roster name still matches the exact bytes about to leave. I verified it four ways on real data, and the second check is the one people forget: the gate must refuse the unredacted original. If it does not, it is inert and you have no way of knowing.
What is reviewed must be what is sent
Everything else follows from one rule: the bytes a human reviewed must be byte-identical to the bytes that leave. A review of roughly-what-will-go is theater, because the thing that leaves is not the thing anyone read.
So the payload is written once, as one file, not assembled at send time from a template. Approval records a sha256 of the file, never a filename, because approving “the transcript” means nothing once the transcript can change. Re-staging deletes any existing approval, so new bytes cannot inherit old clearance. The gate runs again at send time, on the same bytes. Every refusal raises. There is no return value that means “probably fine.”
What I got wrong on the way
The review page compares the original against the redacted payload, so it has to contain the original names. That means it cannot be part of what gets sent. It is named review.local.html, it is gitignored, and the send path returns the payload alone rather than the directory it sits in. A test that asserts “no file in the bundle contains a name” fails on that page, correctly. The fix was the naming and the return value, not weakening the test.
The first version of that review page diffed the raw transcript against the whole payload, which has a computed-facts block in front of the transcript. Every line was offset, and it reported 560 lines redacted when two had changed. It looked plausible. The tell was in the manifest the whole time: placeholders: 5 sitting next to lines_redacted: 560, two numbers describing one event that could not both be true. Make the review and the redactor unable to disagree, and assert it.
A public figure’s name got redacted because a student shared his first name. Harmless, but it means the known-safe list, which only controls what gets reported, cannot solve it. Exposing a full public name needs a separate never-redact phrase list, checked before roster matching. Not built yet, and I would rather say so than imply it is.
And the ordering mistake that is easy to make silently: redact before computing derived artifacts. The facts block, the timeline, the figure list are all derived text and all leave the machine. Redacting the transcript and then deriving from the original is a one-line bug that leaks everything.
Why this is worth an essay
Every compliance officer I have ever worked with is afraid of the same thing: someone pastes a document into a chatbot, and the document contained something it should not have. The usual response is a policy that says not to. A policy about shadow AI does not change what people paste.
What changes it is a control that is small enough to read, sized to a threat someone actually measured, verified against real data including the case where it must refuse, and wrapped in an approval that cannot be satisfied by anything other than the exact bytes leaving. That is a different kind of answer, and it is the one I would want to hear from anyone asking me to trust their system with my data.
Questions about this one? Ask the site, or emailgreg@corporateprofessor.com.