Probe Monitors

A probe monitor is a small linear readout trained in miWorkbench: a vector, a threshold, and a record of how well it actually worked. miServer imports one, checks it was built for exactly the model you have loaded, proves it scores the way miWorkbench measured, and then reads it on live traffic.

Not the Feature Monitor page

Feature Monitor watches SAE feature activations. This page runs trained detectors. The older page was called "Probe" until this feature arrived; two pages with that name was a coin flip for anyone reading them.

What it does, and what it deliberately does not

It records, shows and annotates. It does not stop a generation, re-route it, or escalate to a judge. That is a later phase, and the reason it is later is that thresholds have to be proven on real traffic before anything acts on them.

So a verdict is evidence for a human or a downstream process, not a gate.

Importing

Upload a .probe.json exported from miWorkbench, or import from HuggingFace (repos tagged mistudio-probe-definition). The whole document is stored verbatim, so re-export is lossless.

on_conflict is rename or fail. There is deliberately no replace: overwriting a definition in place while its probe is armed would change the detector underneath a running monitor, while every event before and after kept the same probe id — the history would describe two different detectors as one. Re-importing a rebuilt probe goes disarm → delete → import.

MOVING A BAR IS NOT REPLACING A DETECTOR.

A probe definition carries two kinds of fact. The DETECTOR is everything that determines what number the probe produces: head.weights, bias, norm_mean, norm_std, attention_query, read.layer, read.hook_point, scope, basis, the sae block and its feature_indices, aggregation.rule and its params, model, and the evidence that says what the number is evidence of. The BAR is everything that determines only where that number is cut: decision.threshold, target_fpr, realised_fpr, threshold_source, calibration, windows and length_bands.

replace was refused because it replaces the first kind, and the objection above stands exactly as written. It turns on a specific property of probe_events: the row records a score whose MEANING comes from the detector, and nothing on the row records which detector produced it. Change the weights and event #1's score = 2.9 and event #900's score = 2.9 are measurements of different quantities under one id, with nothing to tell them apart.

A moved bar is not that, for one concrete reason: the event row ALREADY records the bar it was judged against, per verdict, at judgement time — including the length-band override — and nothing joins an event back to probes.threshold. After a re-cut, event #1 still says it was judged at 2.8786 and event #900 says 2.4011; both are true and both remain comparable, because the score beneath each was produced by the same weights at the same layer under the same scope with the same rule. The score is the measurement; the bar is the line drawn across it.

THE RULE: a probe's identity is everything that determines its SCORE; its bar is everything that only determines the CUT. The first may never change in place under a probe id. The second may, through POST /api/probes/{probe_id}/recalibrate, which is extra="forbid" and therefore structurally incapable of carrying a detector, refuses any cut it cannot match to provenance.probe_id, never stores the incoming object as the definition, and refuses a threshold with no budget and no named source. on_conflict remains rename|fail.

Arming: four gates

They run in the order they are cheapest to fail.

#	Gate	Refuses when
1	Limit	PROBE_MAX_ARMED (4, measured; the cost budget is the real limit) are already armed
2	Identity	the loaded model is not the one the probe was fitted on
3	Evidence	the probe is below rung 2 and you have not acknowledged it
4	Parity	this build does not reproduce miWorkbench's recorded scores

Identity

Four fields must match and they refuse on mismatch: hf_id, d_model, n_layers, and the chat-template hash. Every mismatch is named, not just the first — otherwise you fix one field, retry, and meet the next, never learning whether you loaded the wrong model or imported the wrong probe.

The template hash is the least obvious and the most valuable. Same weights plus a different chat template is a different token stream, so the probe reads positions that do not mean what it learned — and every other field would match.

The revision warns rather than refusing when miServer cannot establish which commit is on disk. But two known revisions that differ do refuse: not knowing and knowing-they-differ are different situations.

Why a probe refuses where a circuit binds

A circuit can be bound across an identity mismatch, because it names features a human can reason about. A probe cannot. Its weights are a direction in one specific model's residual space; read in another model's space they produce numbers that are plausible, stable, well-behaved, and about nothing. There is no symptom.

Arming, disarming and checking parity all happen on this page. A refusal names the gate it came from, because "wrong model" and "needs your acknowledgement" call for entirely different actions.

Parity

Before arming, the definition's test vectors are re-scored here and compared with the scores miWorkbench recorded, within 1e-3. If they differ, the probe is refused — the alternative is a detector reporting under a measurement taken of something slightly different.

The gate is the SCORE, not the per-token trace. Parity passes or fails on the combined score — what the probe actually decides with — and reports per-token divergence beside it without gating on it.

That is deliberate, and measured. miWorkbench scores probes in float16; miServer serves bfloat16, because float16 overflows on models trained in bfloat16. Over sixteen real vectors that costs a worst case of 0.098 on the score against a threshold of 2.879 — 3.4% — while per-token traces diverge by up to 6.9, because the recorded tolerance is absolute against values reaching 55. A gate at that tightness is reachable only by computing bit-identically, and an independent implementation is exactly what parity exists to verify.

It remains a real gate: the wrong dictionary, the wrong layer, the wrong hook or the wrong feature selection all move the score by far more than precision does. A basis error found during acceptance moved it to 1.68, seventeen times the tolerance.

Parity scores from the recorded token_ids, never by re-rendering messages. miWorkbench measured the ids reproducing exactly while re-rendered messages missed by over 1.1, because messages is a reconstruction whose re-render adds template tokens. Whether messages re-renders to the same ids is reported separately, as tokenization drift, and never affects pass or fail.

Parity can also be checked without arming — Check parity on a probe's row. That is the useful thing to do after reloading a model: the same definition against a differently-loaded model is exactly the case parity exists to catch, and the report is stored either way so you can read it later. Checking never arms, and never marks the probe armed.

Scope and reproducibility

Only scope: all is exactly reproducible. For prompt and response, miWorkbench recorded scores under a narrower internal mask and the contract does not carry which positions those were — so parity reports that it cannot check, and the probe is refused rather than armed unverified.

The evidence rung

rung	what it means
0	trained
1	detects on held-out data
2	detects on unseen tasks
3	detects on unseen tasks, compared with a judge

Arming below rung 2 opens a dialog asking why, and records the answer. The acknowledgement is stored separately from the one in the definition — the person who exported a weak probe and the person arming it against live traffic are not necessarily the same, and only the second is choosing to monitor with it.

"Compared with", not "beats"

Rung 3 means a judge scored the same data. It may have won. On miWorkbench's reference run it did: the judge averaged 0.8744 AUROC against the 1.2B probe's 0.7938, on all five sets. A probe's case on a small model is cost and latency, not accuracy.

Getting a probe here

Two paths, and the page offers both:

• Import definition — a .probe.json file exported from miWorkbench.
• Browse Hub — search Hugging Face for repos tagged mistudio-probe-definition, open one, and import a single definition. Anonymous and read-only.

A listing shows each definition's rung where the repo's manifest carries one, and says "rung not stated" where it does not. That distinction matters: rung 0 means trained, and nothing further measured, which is a claim about the probe's evidence — not the same as an unknown.

Reading a verdict

Non-streaming responses carry a header:

X-miLLM-Probe-Verdicts: "high-stakes";score=2.31;threshold=1.07;verdict=?1;rung=3;window=all;behaviour="high-stakes";probe="pr_0a1b2c3d4e5f"

The header is an RFC 8941 list: every string is quoted and escaped, every number a decimal with at most three fractional digits (the body field and the stream chunk carry the full-precision score). Each member names its behaviour and probe, and one behaviour's members are always next to each other. The header is bounded (PROBE_VERDICT_HEADER_MAX_BYTES, 8,192 bytes): past it, whole behaviours are left out and a last member "...";truncated;total=<n> says how many results there were. Nothing is lost — the body field, the stream chunk and the events list always carry every result. Behind a proxy, the proxy must accept a response-header block that large: ingress-nginx's default 4 KB proxy_buffer_size turns a header over about 3.9 KB into a 502, so miServer's own Ingress sets nginx.ingress.kubernetes.io/proxy-buffer-size: "16k". If you raise the bound past about 14 KB, raise that buffer too.

Streaming responses carry a final chunk before [DONE], shaped like OpenAI's usage chunk:

{"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[],
 "millm_probe_verdicts":[{"name":"high-stakes","scored":true,"score":2.31,
   "threshold":1.07,"verdict":true,"rung":3,"rung_language":"detects on unseen tasks, compared with a judge"}]}

Both clients we tested handle this: the OpenAI SDK exposes the extension through model_extra, and Open WebUI skips chunks with empty choices. Open WebUI will not display the verdict — its filter surface has no outlet hook — so a verdict reaches a human through this page, not the chat.

Three things a verdict does not say

• verdict absent means no threshold was placed. The probe ranks without deciding; it is not a "no".
• scored: false means the request was not scored, and always carries a reason code (the list below). It is not a "no" either.
• A high score is not a cause. A probe detects; it does not explain.

The template prefix

Every request begins with the same tokens in the same context: the start token, the chat template's preamble (Llama 3.1 injects a system block with a date line) and the first user header. A model gives those positions the same activations every time, so their scores are a constant, and for a max probe that constant can be a floor the content never rises above. A probe whose definition sets aggregation.skip_template_prefix (every probe miWorkbench trains from 2026-11-08 on) leaves those positions out of its all, prompt and last_user windows (on a single-turn request the newest user message's header is part of the prefix); response is unchanged. miServer measures the prefix for each request from its own tokens, so a request with its own system prompt leaves out only the part it shares with the template's default. A probe without the field is served exactly as before. If the prefix cannot be measured, those windows say template_prefix_unknown instead of scoring positions the probe was never calibrated over.

Where the bar is

A verdict fires when score >= threshold — a score exactly on the bar fires. miWorkbench cuts the bar at a negative's score and counts that negative as admitted, so this is the rule that makes the definition's stated false-positive rate true here. It is the same everywhere a verdict is reported: the header, the streaming chunk, the event, and stateless scoring below.

The result contract

Since Feature 38, every place a probe result appears carries the same object, built once per probe and window when the request finishes: mi.probe.result/v1, published as docs/schemas/probe-result-v1.json. The places are the non-streaming response body, the streaming terminal chunk, the stored event, the probe:results socket message, the events list and detail routes, session timelines, POST /api/probes/score and Batch output lines (each keeps its older keys beside the contract fields), and, when telemetry export is on, the exported mi.probe.result log record.

Non-streaming responses now carry the results in the body too, beside choices: millm_probe_verdicts is a list of contract objects. It is absent when nothing is armed, so an unarmed response is byte-for-byte what it was.

The fields worth knowing:

Field	Means
result_id	One id (a time-ordered UUID) for this result on every surface and in every export, retry included. Filter events by it: GET /api/probes/events?result_id=
request_id, trace_id, span_id	The request; the W3C trace it ran in (from a traceparent header, or minted); the probe-evaluation span. GET /api/probes/events?trace_id= returns that request's results
outcome, reason	completed, or skipped, failed or interrupted with a reason code (below). A reason is a code, never free text
status	final for a finished request. An interrupted one is partial (a score over the tokens seen, with no verdict) or unavailable (no score)
persistence	Whether miServer stored it: persisted, not_persisted (stateless scoring, or a failed write) or unknown
detector_digest, detector_digest_source, identity_algorithm	What produced the score: a digest of the detector (weights, layer, scope, rule, model). producer when miWorkbench supplied it and miServer verified it, computed when miServer computed it, unknown otherwise
decision_digest, decision_revision, threshold_revision	Where the bar was, and which cut: a recalibration moves both revisions
engine, execution_mode, precision, served_model, model_revision, model_load_id, template_sha256	How the request was actually served
score, threshold, verdict, fires, n_scored_tokens, rung, rung_language	The reading, as before

The header X-miLLM-Probe-Verdicts keeps its keys and carries no result_id (it has to stay short); its reason is now the code.

Interrupted requests

A stream that ends early (the client disconnects, the request is cancelled, generation raises, or the generation thread hangs) still records a result for every armed probe and window, with outcome: interrupted, the reason, and status: partial (a score over the tokens it saw, never a verdict) or unavailable. It cannot reach the client, whose stream is gone; it is stored and exported like any other.

Outcomes and reasons

The published list. Codes may be added within v1 (a consumer reads an unknown code as "other"); none is removed or renamed.

Reason	Outcome	Means
batched_request	skipped	more than one row in the forward pass (n > 1 or a batch)
continuous_batching	skipped	served by the continuous-batching engine, which has no hook
engine_unsupported	skipped	served by an engine with no forward hooks (llama.cpp)
judge_traffic	skipped	a judge's own call (X-miLLM-Origin: judge); never user traffic
last_user_span_unknown	skipped	the last_user window needs a span and none was stated
last_user_span_unresolved	skipped	the newest user message could not be located
no_chat_template	skipped	the tokenizer has no chat template to locate a turn with
no_scored_tokens	skipped	the window selected no position the probe observed
no_user_header	skipped	the template's user header could not be found in the ids
no_user_turn	skipped	the conversation has no user message
prompt_boundary_unknown	skipped	the window needs the prompt boundary and none was stated
rule_set_not_attached	skipped	the verdict definition's rule set is not attached to the behaviour, so the rules are unknown and the members decide
rule_worker_unavailable	skipped	operator code has no runner result: scored after the response (D41), or the runner absent or not answering (D57)
speculative_decoding	skipped	a draft model proposed tokens; positions are not served
template_prefix_unknown	skipped	the probe skips the template prefix (G38) and the request's prefix length was not stated
text_completion_has_no_user_turn	skipped	a text completion has no user message
encoder_failed	failed	the k-sparse encoder raised (detail in the log)
head_mismatch	failed	the head's width does not match the activations
non_finite_score	failed	the score was NaN or infinite, so no score is reported
prompt_boundary_conflict	failed	two different prompt lengths were stated (detail in the log)
prompt_boundary_invalid	failed	a negative prompt length was stated (detail in the log)
rule_error	failed	a validator or operator code failed while the signal ran (detail in the log)
rule_input_too_large	failed	the window's text is over the runner's input limit, so operator code never saw it
rule_load_refused	failed	the runner refused to load the operator code (its document or prelude failed)
rule_output_invalid	failed	operator code returned something other than a count
rule_resource_limit	failed	operator code ran out of fuel, memory or stack
rule_set_vectors_failed	failed	the rule set disagrees with its own cases under this build, so it is not evaluated
rule_signals_disabled	failed	rule signals are switched off (RULE_SIGNALS_ENABLED)
rule_timeout	failed	the rule time budget ran out before this signal ran, or operator code hit its wall time
cancelled	interrupted	the request was cancelled before it finished
client_disconnected	interrupted	the client went away before the stream finished
generation_error	interrupted	generation raised before it finished
generation_thread_hung	interrupted	the generation thread did not stop in time

Digests on import

A definition exported by a recent miWorkbench carries an identity block with its own digests. Import (file or Hub) recomputes both and refuses a mismatch with 409 PROBE_IDENTITY_MISMATCH, naming the field and both values, and an algorithm it does not know with 409 PROBE_IDENTITY_ALGORITHM_UNKNOWN. Either way nothing is stored: re-export the definition from miWorkbench. A recalibration that sends decision_digest is checked the same way against the decision it would store.

When a probe is not scoring

On the Probe Monitors page, each probe's results open and close under that probe (N results · M fired), with filters for fired verdicts and for one window, and each result shows the date and time it was recorded (the exact instant in its tooltip). Rule-signal results, and results of a probe that has since been deleted, are listed under Rule signals and other results.

An armed probe that is quiet always says why, on the page and in GET /api/probes/status. Silence would read as "nothing detected", which is a claim it never made.

Requests are not scored when they are batched (n > 1 or extra_messages, reason batched_request), under speculative decoding, served by continuous batching (chat, streaming chat and text completions all record continuous_batching), or served by llama.cpp (engine_unsupported). Every one of these still writes an event and still sends the header — on /v1/completions too, which never sent X-miLLM-Probe-Verdicts before Feature 27.

Scoring stored text

POST /api/probes/score asks imported probes about inputs you already have — rows in a dataset, a transcript, a definition's test vectors — without arming anything and without recording anything:

{"probe_ids": ["prb_…"],
 "inputs": [{"token_ids": [1, 2, 3], "prompt_tokens": 2},
            {"messages": [{"role": "user", "content": "…"}, {"role": "assistant", "content": "…"}]}],
 "windows": ["all", "prompt"]}

• Nothing is persisted. No event, no change to what is armed, no change to a stored parity report. An armed probe on the same layer sees nothing either.
• Offline equals live. The probe is built, run and decided by the same code live serving uses, on the same layer hook, unsteered (every attached SAE is suppressed for the scoring forward).
• Same ids as a live chat. A messages input is tokenized exactly as live serving tokenizes a chat — one BOS, never two. Before 2026-10-08 both carried a duplicate BOS on Llama 3, gemma and LFM2.5, so scores from that period differ slightly from today's (see How a chat becomes token ids in the OpenAI-compatible API page).
• One input at a time. Each input takes its own place in the request queue, so a chat request waits at most one input's forward behind a scoring batch. Inputs are never packed together: bfloat16 results change with batch shape.
• Window boundaries are never guessed. token_ids may carry prompt_tokens; a messages input ending in an assistant turn derives it; a messages input ending in a user turn is all prompt. Otherwise a prompt or response window says prompt_boundary_unknown, and last_user on bare token_ids says token_ids_have_no_turns.
• verdict is three-valued, rung_language is verbatim, and provisional is carried as recorded — a provisional verdict is a ranking, not a rate.
• Parity is reported, not required. Each probe says whether its stored parity passed, failed or never ran, and which model load it was checked against ("unknown" for older reports).
• Wrong model: with probe_ids given, the request is refused naming every mismatched field, as arming refuses. With probe_ids omitted, every matching probe is scored and the others are listed under skipped with their reason.
• If the model changes mid-request, the remaining inputs fail with MODEL_CHANGED; earlier results are kept. No input is scored on a different model from the first.

text inputs

text is scored as one user turn, the way miWorkbench builds its training corpus from prose: the result is identical to sending messages with that one user turn. This render was checked on the GPU node against a miWorkbench probe trained on prose, and reproduced miWorkbench's AUROC on both of its evaluation sets.

Limits: PROBE_SCORE_MAX_INPUTS (64) inputs and PROBE_MAX_ARMED (the arm limit, 4 by default) probes per request; token ids must lie inside the model's vocabulary; each input must fit the context window.

Arming costs throughput

While any probe is armed, continuous batching is disabled and requests run serially (PROBE_FORCE_SERIAL, default true). On a busy deployment that is a capacity decision, not a detail. Turning it off does not make probes score under continuous batching — those requests are marked not_scored with a reason instead.

Sessions

While at least one probe is armed, miServer groups the requests of one conversation into a session and records which turn of it each verdict belongs to. The Sessions page shows each session as a timeline: one row per turn, one column per behaviour (a probe whose definition names no behaviour is its own column).

How a conversation is recognised.

• A client can name it with the X-miLLM-Session request header: 1 to 128 printable characters, stored as given. Do not put personal data in it. An invalid value is refused with 400 SESSION_ID_INVALID. The response says which session it joined in X-miLLM-Session-Id.
• Open WebUI: in Admin → Settings → Connections, edit the miServer connection and add the custom header X-miLLM-Session with the value {{CHAT_ID}}. Each Open WebUI chat then becomes one session. No filter change is needed.
• Without the header, a request that resends the conversation so far (as every chat client does) joins the session it continues. A client that rewrites the earlier reply still joins when exactly one conversation matches; when two match, the request opens its own session, marked as ambiguous.
• A session idle for 30 minutes (PROBE_SESSION_IDLE_MINUTES) ends; a later continuation opens a new session linked to it.

Text recording is off by default (PROBE_TRANSCRIPTS, Settings → Environment). When it is on, each monitored chat turn keeps the newest user message and the visible reply (never the reasoning) for 7 days (PROBE_TRANSCRIPT_MAX_AGE_DAYS, at most PROBE_TRANSCRIPT_MAX_TURNS turns in all). Every response of a monitored request says whether its text was kept: X-miLLM-Transcript: recorded or off. Text is served only by GET /api/probes/sessions/{id}/transcript, never on the live socket, and DELETE /api/probes/sessions/{id} removes a session's text at once (events=keep keeps its verdicts, detached; events=delete removes them too). A background pass removes expired text, sessions and verdicts every PROBE_RETENTION_INTERVAL_S seconds.

Behaviour sets

A behaviour is usually watched by several probes — different layers, rules and windows, each looking at a different aspect — and an application blends their results however it wants. miServer treats every imported probe whose definition names one behaviour key as that behaviour's set. Nothing stores the set; it is whatever is imported under the key (one version at a time: when probes of two versions are imported, name one with version, or the probes with probe_ids).

• Arm a behaviour in one call — the Arm button on the behaviour's group on this page, or POST /api/probes/behaviours/{key}/arm. Every probe runs its own gates (identity, rung, scope, parity, cost), then the set is checked once against the behaviour limit, the probe backstop and the cost budget. All or nothing: if any probe fails, none is armed, and the refusal lists every probe with the gate that stopped it.
• Dry run first. The Dry run button (dry_run: true) runs every gate — parity and cost included — arms nothing, and shows each probe's result and the projected cost against both halves of the budget.
• Partial sets say so. A probe imported later into an armed behaviour stays disarmed, and the group reads "partial: 3 of 4 armed", naming the missing probe. Arming the behaviour again arms only the missing ones.
• After an unload or a restart every probe is disarmed (as before); the page names the behaviours that were armed, so each comes back with one click.
• Disarm a behaviour with Disarm (POST /api/probes/behaviours/{key}/disarm). A layer hook another behaviour still uses stays.

Every result a request produces — header, body field, stream chunk, stored event and the live probe:results feed (one message per request) — is listed in one order, behaviour by behaviour, and each result of a behaviour carries a set block (armed_members, total_members, complete, missing_probe_ids, members_digest), so an application knows whether its blend's inputs were all there.

POST /api/probes/score accepts behaviours: [key] to score a behaviour's probes on stored text — armed or not — before arming them.

Rule sets and rule signals

A rule set is a miDataworks rule-set document: patterns (a linear-time dialect — no back-references or look-around, so no pattern can stall serving), proximity to a cue phrase, validators (check digits, length and prefix tables), weighted word lists, and compositions of those. Import one with POST /api/rule-sets/import (the body is the document as the producer published it); POST /api/rule-sets/validate runs every check and writes nothing. Each set must pass its own cases to be accepted, and re-proves them at every start: a set that a new miServer build disagrees with is marked vectors_failed and stops being evaluated.

Attach a set to a behaviour with PUT /api/probes/behaviours/{key}/rule-signals. While at least one probe of that behaviour is armed, every named expression of the set runs beside the probes on every request, on the windows those probes report, within RULE_TIME_BUDGET_MS (20 ms). Each reports as its own result — kind: "rule", with fires and a count, never the matched text — next to the probe results in the header, the body, the stream, the events list, the session timeline and Batch lines. A rule result never changes a probe result: a set's vetoes and output columns are for building datasets and are not applied here.

• A pending result reports count: 0. A rule result that has not been decided (status: "unavailable", fires: null) still carries count: 0, so check status (or fires: null) before you trust count.
• Judge traffic and requests the probes do not score report each signal skipped with the same reason; nothing runs.
• RULE_SIGNALS_ENABLED=false (Settings → Environment) reports rule_signals_disabled for every attached signal and keeps the attachments.
• Operator code never delays a chat. A set may name a code_rule signal; it is accepted, its live results report skipped / rule_worker_unavailable, and the rule-runner sidecar scores them after the response (the stored result is revised in place, same result_id).
• What needs a code rule's positions is refused. Operator code reports a count, never where it matched. So validate and import refuse a set that asks for its positions (suite decision D65): a within_n with a code rule as an operand (code_rule_in_within_n), or a proximity anchor or validator span_of that names one (code_rule_positions_needed). Since 2026-10-28 (decision D74) a veto is refused too when its when expression or its removes list names a code rule (a veto removes matched spans, and a code rule has none), and so is a field output that reads a code rule (code_rule_positions_needed, with use veto_when, veto_removes or field). The checks run in miDataworks' order, and the refusal names the expression, signal, veto or output column. Nothing is stored. Counting a code rule ({signal}, all, any or not over it, or a fired or count output) is still accepted and pending until it runs. A set stored before these changes that reads those positions fails its next startup self-check (rule_set_vectors_failed). Re-import it without that read.
• The stateless routes wait for operator code. POST /api/probes/score (and a Batch probe line) and POST /api/rule-sets/{id}/evaluate have no live budget, so they ask the runner and return its result. Each call is bounded by twice RULE_CODE_CALL_WALL_MS plus a margin, and one request makes at most RULE_CODE_CALLS_PER_REQUEST calls and spends at most 30 seconds on operator code in all. They report rule_worker_unavailable only when the runner is absent or does not answer in time, or the request's call count or 30 seconds are spent. A result decided by operator code carries a count, never spans.
• When operator code fails, the result says why with the runner's own code: rule_resource_limit, rule_output_invalid, rule_load_refused, rule_input_too_large, rule_timeout or rule_error. rule_timeout means the runner stopped the code at its wall time. A runner that does not answer at all decides nothing: the result stays rule_worker_unavailable, on the stateless routes and after a chat alike.
• POST /api/rule-sets/{id}/evaluate runs a set on text you send; return_rule_positions: true returns the matched spans of that text, and only of that text.

Probe limits per model

Probes may slow each token by at most 5% of the model's own time (decision D83, 2026-10-29). As many probes arm as fit within that; there is no limit in behaviours. The model's own time is its measured time per generated token and per 1,000 prompt tokens, so a large model, whose tokens take longer, carries more probes than a small one. PROBE_SLOWDOWN_BUDGET_PCT (5) sets the share, and arming fills at most PROBE_BUDGET_HEADROOM (0.9) of it to leave room for measurement noise. PROBE_MAX_ARMED (64) stays as a backstop on probes.

A model whose step time has not been measured uses the fallback budgets PROBE_MAX_OVERHEAD_MS_PER_PASS (0.25 ms per generated token) and PROBE_MAX_PREFILL_MS_PER_1K (0.5 ms per 1,000 prompt tokens) until you measure it; the panel and every refusal say so. Measure it once, in a window when the card is free: Measure on its card on the model's Probe capacity panel.

How many probes fit depends on the model and the card. Each probe costs a little on every forward pass: a fixed part per hooked layer plus its own arithmetic. From the 2026-10-21 measurements on the RTX 3090, dense probes on one layer:

Model	Time per generated token	5% of it	Dense probes that fit, per generated token
LFM2.5-1.2B	6.69 ms	0.33 ms	9
Llama-3.1-8B	22.68 ms	1.13 ms	40

Those measurements did not time the prompt step, so their prompt half is still the 0.5 ms fallback (7 and 3 dense probes); measuring the model again replaces it with 5% of its own prompt time. A k-sparse probe costs about five times a dense one per token.

Each model gets an explained recommendation in its Probe capacity panel: the 5% budget in milliseconds on this measurement, and how many reference sets (nine probes over three layers, mixed rules) and how many dense probes fit under it, costed exactly as arming costs them. Approve writes the budget as a percentage of the model's own time, so it follows the model's next measurement (the server checks the recommendation has not changed since you read it); Override sets your own, as a percentage or in milliseconds. The panel shows where each limit came from and who set it, and marks it stale when the model, its precision or its card has changed since the measurement. A budget approved before 2026-10-29 was sized from the old count of behaviours; it is marked no longer in force and the 5% budget applies until you approve the new recommendation.

The Probe capacity panel (open the model's details on the Models page) shows the measured lines, the limits they imply, the limits in force and where each comes from, and how close the armed set is to each. Measure on its card times the loaded model's own step time and the cost of mean, max and attention probes on the card(s) it runs on; it needs the model loaded and nothing armed, and it uses the graphics card. Over the API: GET /api/models/{id}/probe-capacity, PUT /api/models/{id}/probe-limits, POST /api/models/{id}/probe-capacity/measure.

Privacy

The socket broadcast (probe:results) carries no prompt or context text. The decoded window around the top firing position is stored on the event and served only by the single-event detail route, so a reviewer asks for one conversation rather than a dashboard receiving a feed of them.

Overhead

Under 5 ms per request at 4k tokens with two probes armed, with exactly one device-to-host copy per forward pass however many probes are armed. GET /api/probes/status reports the last request's measured overhead and the warning threshold.

Limits worth knowing

• Probes cannot run on GGUF models — llama.cpp exposes no module tree to hook.
• A probe can be suppressed. Published work shows adversarial inputs pushing a probe down. It is the cheap first stage of a cascade, not a perimeter.
• Recall at a tight false-positive budget is modest. On miWorkbench's reference run the 8B probe caught 47% of positives at a 1% false-positive rate. It is a triage signal.
• Probes read the pre-steering residual, so steering cannot change what a probe sees.