The Gate That Never Fired: Auditing Cereby's Intent Classifier
The number behind every routing decision was mostly two constants. How we found that, and what replaced it.
Cereby has a rule for when it is not sure what a student means: it asks. "Did you want flashcards or a quiz?" The rule lives in the intent classifier, the step that reads each message and decides which tool it is for: a quiz, flashcards, a study schedule, or a plain answer. When the classifier's confidence falls below 0.5, Cereby stops and asks instead of guessing.
In the 30 days before this audit, it never asked once. Not because every message was clear. Every routed action, whatever the message, came out between 0.60 and 0.77 confident, with a median of 0.71. A number that never leaves a narrow band is not measuring much. So we went looking for what it was made of.
This post is one half of an audit of Cereby AI. The other half, about what the answering model reads once the classifier has decided, is Hello Instead of an Answer.
What the number was made of
In May we rewrote the classifier so that one model call decides the intent. The number every routing decision used was not that model's, though. It was a blend: 60% a multi-factor score, 40% the model's own confidence. We described the factors in Inside Cereby's Intent Classifier. By October, two of them were constants:
The "model confidence" factor was meant to come from log-probabilities (how sure the model was of each word it wrote). The classifier asked the provider for them and never passed them on, so the factor fell back to 0.6 on every message. The "past accuracy" factor read earlier classifications with the database client built for the browser. Our row-level security (rules that let a session read only its own user's rows) blocks that client on the server. So the factor was 0.5, after a wasted round trip to the database, on every turn.
The worst part was what the blend threw away. When the model was genuinely unsure, it said so twice: a low confidence of its own, and a clarifying question it had already written. The blend diluted the first, and a keyword heuristic wrote a different question in place of the second.
Three quarters of the prompt was a phrasebook
The classifier's instructions ran to 43,645 characters, about 10,900 tokens, sent with every message a student typed:
Most of it was a catalog of example phrasings for every tool. Worse, 22 of our 48 eval messages appeared in it word for word, so the eval was partly grading the prompt's memory. We rebuilt the catalog from the tool definitions themselves, with two examples per tool and one block of rules, which brought the prompt to about 6,200 tokens.
No deadline at all
Nothing bounded how long one message could take to classify. The client library's defaults gave each attempt ten minutes and two retries. Any error triggered a second request without JSON mode. Then the pipeline retried the whole thing:
Production classifies in about a second (1.5 seconds at the 99th percentile), so ten seconds is generous, and a stalled provider now costs a student ten seconds instead of minutes.
"Yes please"
The classifier reads a little of the conversation along with the message, so that "yes" can mean something. It used to read the first 200 characters of each of the last four turns. An offer sits at the end of a reply. Here is that case on a live call; the reply and the "yes" are synthetic, and the result is what the classifier returns today:
In fairness to the old code, production had not hit this one yet: 17 of 411 replies ended in an offer, and nobody answered one with a bare "yes". It was a bug waiting for its first student. The envelope now carries six turns, and the last reply keeps its opening and its end.
What we changed
The fix was mostly subtraction. The scorer is gone. The model's own confidence decides, on the same scale as the alternatives it lists, and when it is unsure, its own clarifying question is the one the student sees. Each message gets one ten-second deadline, at most two attempts, and a second attempt only for a failure a retry can fix. Along the way we deleted five branches nothing could reach, and fixed an attachment request that had been answered as a plain question since May.
Then we rebuilt the eval it all had to pass: 102 cases, now with conversation history, pinned content and attachments, run live on Gemini 2.5 Flash-Lite before and after. Because the old prompt had quoted so much of the old eval, we also report the 80 cases it never contained:
That is 97 of 102 before and 101 after; 75 of 80 and 79 of 80 on the unquoted cases; 9 of 12 follow-ups and then all 12; 7 of 8 questions about pinned content and then all 8. The median classification in the eval also got faster, from 870 to 712 ms, with a smaller prompt and one database round trip fewer.
The boring failures
The eval that ran on an empty catalog. The tools register themselves when the chat controller starts, and the eval never started one, so its classifier ran with no catalog at all. Production was fine; the measurement was not. The catalog is now built straight from the tool definitions, so it cannot be empty.
The substring that cancelled a card. When Cereby offers a quiz, the student can reply to change it, confirm it or cancel it, and a few phrases were caught before the model saw them. They matched anywhere in the message:
The confirm side had the mirror bug: "let's go over chapter 3 instead" confirmed the card, because it began with "let's go". Both now act only on short messages that are plainly a yes or a no, and leave everything else to the classifier, which already tells a change from a new request.
A regex that filled in missing subjects wrote "from this" as the subject of a quiz about a pinned note. It no longer runs when the student pinned or attached something, which is exactly when the model has the better answer.
Before and after
| Before | After | |
|---|---|---|
| Confidence | A blend, two of its five factors constant | The model's own number and question |
| The clarification gate | Never fired in 30 days | Fires on the model's own doubt |
| Prompt per message | 10,900 tokens | About 6,200 tokens |
| Requests per message | Up to 12, no deadline | At most 2, within 10 seconds |
| Eval, all 102 cases | 97 correct | 101 correct |
What this taught us
A blended score with constant inputs still produces plausible numbers. The tell was the band: every action between 0.60 and 0.77, for a month, and a gate that never fired. When a number never moves, find out what it is made of.
The model already told us when it was unsure. It wrote the question and lowered its own confidence, and we averaged both away. Listen to the model before building a heuristic around it.
An eval that shares text with the prompt it grades measures recall, not judgment. We now report the uncontaminated slice next to the total.
And our February post described a log-probability signal that, by October, nothing read. Reading the code against what we said it does was one of the cheapest audits we ran.
What's next
The classifier still cannot see the card a student is answering, so it judges "make it 15 instead" without knowing what "it" is. And a message that asks for two things at once still loses the second on some paths.