Much of the work in a practice is checking records against rules. Which screenings is this patient due for, and were any already done somewhere else? Where does this fax go? Is this referral complete? Which billing code fits the visit? A small AI model can learn to make these calls the way the guidelines and your staff make them, and to pass the ones it isn't sure about to a bigger model or a person. The live page shows the method on a puzzle game, where every move can be checked against an expert.
Watch it live →"AI isn't reliable enough for real work." You've probably heard it, and it's a fair concern. A model can be right nine times and wrong the tenth, and sound equally sure both times. The CCE makes a model reliable enough to use: it's trained on your rules, it says how sure it is, and when it's unsure it hands the decision to a bigger model or a person instead of guessing.
The engine suits decisions that come up thousands of times a week, have a fixed set of answers, and have a history of past decisions to learn from.
| Task | The question | The answers | Handed off |
|---|---|---|---|
Preventive screening | Is this screening due, or was it already done somewhere else? | Due, done elsewhere, or doesn't apply to this patient | Missing or conflicting records go to the care team |
Incoming faxes and outside records | What is this document, and whose queue does it belong in? | Your document types and queues, plus "none of these" | Unusual or hard-to-read pages go to front-desk staff |
Referrals and prior authorization | Is this request complete, and what is it missing? | Complete, or the specific item that's missing | Requests that don't match the payer's checklist go to the referral coordinator |
Billing codes | Which code fits this visit? | The codes that apply to this kind of visit | It suggests; a coder confirms, and notes that could support more than one code go straight to them |
The model is small enough to run on one graphics card inside your own network, so records stay in the building. The same pattern works outside healthcare, for support tickets or a shared email inbox.
Your EHR already knows which screenings are due. Its reminder rules work from what's entered in the chart as structured data, so they can't see the colonoscopy that exists only as a scanned outside report, or the hysterectomy mentioned in a note.
That's where staff time goes: chasing reminders for tests that were already done, and missing the patients who really are overdue in the noise. The engine reads the documents the rules can't. For each patient and screening it answers from a fixed list (due, done elsewhere, doesn't apply, or can't tell) and points to the page it used.
The rules themselves come from published guidelines, such as the U.S. Preventive Services Task Force recommendations, and from the national quality measures that define what counts as done. In a practice it would run overnight on the next day's schedule through the standard data connection Epic and other EHRs provide, and leave staff a short worklist in the morning.
All four live boards use the same model, with 0.8 billion parameters, and play side by side on the same pieces.
A general-purpose model. It can read and answer, but it has never seen your rules.
We show it tens of thousands of decisions made by the expert. In a clinic, that means decisions that follow the written rules: screening guidelines, coding manuals, payer checklists, your routing policy.
The trained model with a confidence threshold. When it's less sure than the threshold, it hands the decision to the expert instead of guessing.
This is what "calibrated" means. For every decision, the model gives each option a probability, and those probabilities hold up: across our test positions its average confidence in its pick was 64%, and it matched the expert 65% of the time. So you can set a rule. When the model is less sure than a threshold you choose, the decision goes to a bigger model or a person instead.
A higher threshold sends more decisions to people and lets fewer mistakes through. A lower one lets the model handle more on its own. The live page runs the trained model at two settings so you can watch the trade-off.
More training means fewer hand-offs at the same setting. We let the model play its own games, had the expert correct its mistakes, and trained it on those corrections. After one round it lasted twice as long on its own and needed help on 23% of moves at 50%, down from 28%, still without losing. With real work, the corrections come from the people reviewing its hand-offs.
| Live board | Games lost | Moves handed off | Pieces in its best game |
|---|---|---|---|
| Loading the live numbers… | |||
Take tomorrow's clinic schedule and one question per patient: is colorectal screening due? The puzzle has an exact right answer and no patient data, which makes it a safe place to show the training method.
| In the game | For a screening check |
|---|---|
| The board | The patient's chart, including scanned outside records |
| A new piece arrives | The next patient on the schedule needs an answer |
| Every place the piece could go | Due, done elsewhere, doesn't apply, can't tell, listed ahead of time by regular software, not by the AI |
| The bars beside each board | The model's ranked answers, with a probability for each |
| ★ the expert's choice | What the guideline says, as an experienced nurse would read the chart |
| Handed to the expert | Sent to the care team to decide |
| A lost game | Wrong reminders piling up: patients called for tests they already had, and real gaps nobody catches |
The engine sorts and suggests. Anything it isn't sure about arrives at a person with its ranked options attached, so the reviewer starts from a short list. For clinical questions, a clinician makes the decision with time to look at the evidence, which is why the engine fits planned work like screening and paperwork and stays out of emergency care. The corrections people make become new training examples, so the model hands off less over time.
Measured on 2,000 positions the models never trained on, drawn from 9 games. Accuracy means choosing exactly the expert's move; about one in five of the trained model's misses are near-ties that cost almost nothing.
Trained model, as on the live boards Off-the-shelf model
| Model | Accuracy | Average confidence | Calibration error (ECE) | Brier score | At 50%: handled alone · right |
|---|---|---|---|---|---|
| Off-the-shelf | 3.9% | 52.6% | 0.488 | 0.304 | 50% · 4% |
| Trained (live boards) | 64.8% | 64.3% | 0.018 | 0.197 | 73% · 72% |
| After one round of corrections (not charted) | 66.0% | 66.7% | 0.026 | 0.189 | 77% · 73% |
ECE is the expected calibration error over ten equal-width confidence bins (0 is perfect). Resampling whole games 1,000 times puts it at 0.013–0.041 for the trained model and 0.48–0.49 for the off-the-shelf model. The Brier score is taken on the confidence of the chosen move (0 is perfect). Bins with fewer than 20 decisions are not drawn.