Skip to main content

Error Codes

Machine-readable error codes returned by the management API in the error envelope (error.code), with their HTTP status. The /v1 OpenAI-compatible endpoints translate the same conditions into OpenAI-format errors.

Model errors​

CodeHTTPMeaning
MODEL_NOT_FOUND404No model with that ID
MODEL_ALREADY_EXISTS409Same repo + quantization already downloaded
MODEL_LOAD_FAILED500Load crashed — see server logs
MODEL_NOT_LOADED400Operation needs a loaded model
MODEL_ALREADY_LOADED400Load called on the loaded model
MODEL_BUSY409Operation conflicts with one in progress: a load is running, or the model is being unloaded (a second unload, or a load that would unload it again), or a weights operation (a checkpoint load or a serving pause) holds the resident weights (details.reason: weights_operation, details.operation). On /v1 it is 503 model_busy, typed server_error: the request is fine and succeeds once the other operation finishes, including a request for a model that is being unloaded
MODEL_LOCKED409Another model is locked for steering: a lock request, or a /v1 request that would auto-load over it (model_locked). The management load and unload do not read it
MODEL_LEASED409A load, unload or swap while another holder leases the resident model. details: holder, reason, expires_at, leased_model_id, leased_model_name, operation, target_model_id. On /v1 it is 409 model_leased, typed invalid_request_error — never model_locked, even when the model is also locked. The holder proceeds by sending X-miLLM-Lease
MODEL_NOT_RESIDENT409A lease was requested on a model that is not the resident, loaded one; details name the resident model. On /v1, model_not_resident answers X-miLLM-Load-Policy: refuse for a model that is not resident: nothing loads
LEASE_NOT_FOUND404Renew or release with an unknown lease ID, or one for another model. The message says that a restart ends every lease
LEASE_EXPIRED409Renew or release with a lease ID that has ended; details.end_reason is released, expired, model_unloaded or restart
INVALID_LEASE_REQUEST400holder (1–128 chars), reason (1–512), ttl_seconds (1–7200, integer, never clamped) or the X-miLLM-Lease header is missing or out of bounds; details.param names it and the limit
MODEL_LOADING (/v1)503X-miLLM-Load-Policy: refuse and the model named is being loaded now; retry after Retry-After

Retry-After on every 503​

Every response miServer sends with status 503, on any route, carries a Retry-After header in whole seconds (at least 1). The error envelope, type and code are unchanged — the header is the only addition.

CodeRetry-After
QUEUE_FULLThe estimated wait rounded up, at most RETRY_AFTER_MAX_S (60); RETRY_AFTER_QUEUE_DEFAULT_S (5) with no estimate
MODEL_BUSY, MODEL_LOADINGRETRY_AFTER_LOAD_S (15) for a load in progress; RETRY_AFTER_UNLOAD_S (5) for an unload
MODEL_NOT_LOADEDRETRY_AFTER_NOT_LOADED_S (30) — a retry succeeds only after a model is loaded
INSUFFICIENT_MEMORY (/v1)RETRY_AFTER_MEMORY_S (30)
HUB_UNAVAILABLEThe hub circuit breaker's remaining recovery time (1 when it is closed)
GET /api/health/ready not readyRETRY_AFTER_READINESS_S (5)
WEIGHTS_TORNRETRY_AFTER_WEIGHTS_TORN_S (30) — serving returns after a late commit-end or a weights load
any other 503RETRY_AFTER_FALLBACK_S (10), added by a safety net that also logs retry_after_defaulted

A refusal raised inside a stream whose 200 is already sent cannot carry a header; its in-stream error event carries "retry_after": <seconds> instead.

Resource errors​

CodeHTTPMeaning
INSUFFICIENT_MEMORY507A card the model would use cannot hold its weights, its KV cache at TRANSFORMERS_MIN_CONTEXT tokens, a request's working memory at that length and its CUDA context. details.per_card gives each card's free, weights, KV, working memory (working_mb), bitsandbytes staging (staging_mb) and context MiB; details.working_memory says how the working memory was sized (method: traced or estimated) and at how many tokens; details.short_devices names the short cards; details.min_context_tokens is the context the cache was sized at and details.model_max_context_tokens the model's own limit; details.rebalance_passes says how many times a split was re-planned before it was refused. An SAE attachment refused for memory carries device, projected_mb, kv_reserve_mb, kv_context_tokens, working_reserve_mb (the request working memory the load kept on that card) and available_mb. For a model whose KV cache miServer cannot size, the estimate exceeds what is free. A request on /v1 that loads the model gets 503 insufficient_memory. A request that runs a card out of memory while generating gets this code too, typed invalid_request_error on /v1 (503; a stream ends with that error event and [DONE]), with details.device, device_name, prompt_tokens, max_new_tokens and batch_rows: send a shorter prompt or fewer max_tokens
INSUFFICIENT_DISK507Not enough disk for the download
UNSUPPORTED_QUANTIZATION400The load asks for a quantization its engine cannot apply (Q2 on a transformers checkpoint that is not already quantized)
GPU_NOT_FOUND404The gpu named in a load is not visible to miServer
SPLIT_NOT_HONOURED409"gpu": "all" was requested, and at least one card would get none of the model: it is too full to take a share, or the model's whole layers leave it empty. details.unused_devices names the cards. details.mapped_mb_by_device shows the layout when it was worked out before loading; otherwise details.landed_on_devices shows where the model landed
INVALID_GGUF_TENSOR_SPLIT500GGUF_TENSOR_SPLIT does not name one proportion per card the GGUF split uses; only the setting fixes it

Download errors​

CodeHTTPMeaning
DOWNLOAD_FAILED502Upstream (HuggingFace) failure
DOWNLOAD_CANCELLED499Cancelled by user
REPO_NOT_FOUND404Repository doesn't exist
GATED_MODEL_NO_TOKEN401Gated repo, no token — accept the license and supply hf_token
INVALID_HF_TOKEN401Token rejected by HuggingFace
SETTINGS_KEY_MISSING409Saving the Settings page's Hugging Face token needs SETTINGS_ENCRYPTION_KEY (unset or under 32 characters); the message carries the command that creates it
HF_TOKEN_INVALID_VALUE400The value given to save is not a token (empty, none, whitespace inside, non-ASCII, over 256 characters); never echoed
HF_TOKEN_NOT_CONFIGURED404Token test asked for with no token at any tier
HF_HUB_UNREACHABLE503Token test could not reach Hugging Face — the token was not judged
INVALID_LOCAL_PATH400Local import path missing, malformed, or in a protected system directory

SAE & steering errors​

CodeHTTPMeaning
SAE_NOT_FOUND404No SAE with that ID
SAE_NOT_ATTACHED400Steering/monitoring requires an attached SAE
SAE_ALREADY_ATTACHED409That exact (sae_id, layer) is already attached (re-attach is rejected; multi-SAE attach_set on other layers is fine); also raised deleting an attached SAE
SAE_INCOMPATIBLE400d_in doesn't match the loaded model's hidden size
SAE_LOAD_FAILED500Weight loading crashed
INVALID_FEATURE_INDEX400Feature index outside [0, d_sae) — details names the offender
SAE_SET_INCOMPLETE422Serving a cross-layer circuit but a member's layer has no (unique) attached SAE — details.offenders names each {feature_idx, layer, sae_id?, reason?} (Feature 12 multi-SAE)
Steering value range: reject on set, clamp on dial

The two paths differ. Setting a steering value directly — POST /api/saes/steering and /steering/batch — validates against [-200, 200], so an out-of-range value is rejected with 422 VALIDATION_ERROR (the schema bound), not silently clamped. The ±200 clamp applies only on the dial/intensity path — profile activation and per-request/cluster/circuit λ — where a profile's stored strengths are scaled by λ and the scaled result is clamped to ±200 at apply time rather than failing the request. So a value you type is checked; a value the dial produces is clamped.

Profile errors​

CodeHTTPMeaning
PROFILE_NOT_FOUND404Unknown profile ID/name (including the per-request profile parameter)
PROFILE_ALREADY_EXISTS409Duplicate name
PROFILE_INCOMPATIBLE400Profile can't apply to the current configuration
VALIDATION_ERROR200†Malformed import payload — steering entries that can't convert to int→float (returned in the envelope, details.invalid_keys names them)
IS_CLUSTER_DOCUMENT200†A cluster definition/bundle was posted to /api/profiles/import; import it via /api/clusters/import instead (the flat profile format has no member/budget semantics)

† Handler-level refusal returned inside the {success:false, error} envelope with HTTP 200 — see the 200-envelope note below.

Probe identity errors​

CodeHTTPMeaning
PROBE_IDENTITY_MISMATCH409A definition's identity digest (or a recalibration's decision_digest) differs from the digest miServer recomputes. details: field, carried, recomputed, algorithm, next_step. Nothing is stored: re-export the definition from miWorkbench (export_probe_definition)
PROBE_IDENTITY_ALGORITHM_UNKNOWN409The definition's identity.algorithm is not one this server can verify. details: algorithm, supported, next_step. Refused rather than stored unverified

Behaviour set errors​

Every one of these is a management-route refusal and carries error.next, the MCP tool that resolves it (Feature 39).

CodeHTTPMeaningnext
BEHAVIOUR_LIMIT409No longer raised (2026-10-29, decision D83): there is no limit in behaviours; probes may slow each token by at most 5% of the model's own time, and PROBE_BUDGET refuses a set that does not fit. Kept here for older clientsmiserver_get_probe_limit_recommendation
BEHAVIOUR_VERSION_MIXED409The behaviour's probes span several versions; arm one (version) or name the probes (probe_ids)miserver_arm_behaviour
INVALID_PROBE_SELECTION400A probe_ids entry is not a probe of that behaviourmiserver_list_probes
MODEL_CHANGED409The loaded model changed while the set's gates ran; nothing was armedmiserver_arm_behaviour
CATALOGUE_LIMIT409A deployment limit (PROBE_MAX_IMPORTED, PROBE_MAX_PER_BEHAVIOUR, RULE_SETS_MAX_IMPORTED, RULE_SET_MAX_SIGNALS) would be passedmiserver_delete_probe (a rule-set limit names its own tool)
RECOMMENDATION_CHANGED409An approval named a recommendation that has since changed; nothing was written; details carry the new valuesmiserver_get_probe_limit_recommendation
INVALID_CURSOR400A page cursor this list did not issuemiserver_list_probes
RULE_SET_INVALID422The rule-set document is too large, is not strict JSON, does not match the vendored rule-set schema or names something it does not declare. details.rule says which; details.pointer or the signal / case says where — never the document's textmiserver_validate_rule_set
RULE_PATTERN_UNSUPPORTED422A pattern uses a construct the linear-time dialect refuses (a back-reference, look-around, …); details name the signal, the construct and its code-point positionmiserver_validate_rule_set
RULE_FORM_UNAVAILABLE422A signal kind this miServer does not evaluate, or a validator id/version it does not registermiserver_list_rule_validators
RULE_VECTORS_FAILED422The rule set disagrees with one of its own cases; details name the case index and id, the column and the signalmiserver_validate_rule_set
RULE_SET_DIGEST_MISMATCH422The declared digest is not the document's identity (RFC 8785 + SHA-256)miserver_validate_rule_set
RULE_SET_NOT_FOUND404No imported rule set has that idmiserver_list_rule_sets
RULE_SET_IN_USE409The rule set is attached to a behaviour that has an armed probe; clear the attachment or disarm firstmiserver_clear_behaviour_rule_signals
RULE_SET_BEHAVIOUR_MISMATCH409A rule set keyed to another behaviour cannot be attached heremiserver_list_rule_sets

PROBE_LIMIT (the armed-probe backstop, next: miserver_disarm_behaviour), PROBE_BUDGET (the cost of the whole set, every probe's own cost in details.probes; next: miserver_get_probe_limit_recommendation) and INVALID_PROBE_SCORE_REQUEST (next: miserver_score_probes) also carry next.

Approval queue errors​

An AI agent's request to a gated route waits on the Approvals page. These codes are lower case, the same spelling as the other suite apps' approval queues.

CodeHTTPMeaningnext
approval_operator_only403An agent tried to approve or reject. Only the operator decides, on the Approvals page; nothing changed—
approval_not_found404No approval request has that idmiserver_list_approvals
approval_not_pending409The request was already decided, or is running; details.status says whichmiserver_get_approval_status
approval_expired409Nobody decided it in time; it will never run. The agent must ask againmiserver_get_approval_status
approval_tampered409The stored request no longer matches the digest taken when it was made; it ended failed and did not runmiserver_get_approval_status
approval_executor_missing409The request names a route this miServer no longer serves; reject it—
approval_batch_refused409"Approve all settings changes" or "Approve all code rules" named a request that is not waiting or not of that kind (the two kinds are never mixed); details.refused lists each one and why. Nothing in the batch was approvedmiserver_list_approvals

Circuit & sensing errors​

Circuit serving spans several SAEs and carries an evidence rung; several of these are handler-level refusals returned in the envelope with HTTP 200 (see the note below) rather than HTTP error statuses, so the client can surface the rung/contention and re-send with an acknowledgement.

CodeHTTPMeaning
CIRCUIT_NOT_FOUND404No circuit with that ID
UNVALIDATED_CIRCUIT200†Activating a circuit below rung 2 (CAUSALLY_VALIDATED) without acknowledgement — re-send with acknowledge_unvalidated=true. The payload carries the evidence rung so the override is deliberate
CIRCUIT_LAYER_CONTENTION200†The circuit's layers are already served by another active circuit. details names the incumbent(s) and the measured hazard; overridable with allow_layer_overlap=true unless a same-key collision (colliding_keys present) makes it non-overridable
NO_ACTIVE_CIRCUIT200†An operation needing an active circuit (e.g. PUT /api/circuits/active/intensity) was called with none serving
AMBIGUOUS_ACTIVE_CIRCUIT200†Several circuits serve, so there is no single "active circuit" to dial — details.active_circuits lists them; deactivate all but one, or dial through the owning cluster
SAE_SET_INCOMPLETE422A circuit member's layer has no (unique) attached SAE (also listed under SAE & steering above)
CIRCUIT_SENSING_EVENT_NOT_FOUND404Circuit edge sensing event id that doesn't exist (pruned, cleared, or never existed)
SENSING_EVENT_NOT_FOUND404Cluster co-activation sensing event id that doesn't exist

† Handler-level refusal in the envelope with HTTP 200 — see below.

The 200-envelope house style​

Most errors map to an HTTP error status. A few circuit refusals deliberately do not: UNVALIDATED_CIRCUIT, CIRCUIT_LAYER_CONTENTION, NO_ACTIVE_CIRCUIT, AMBIGUOUS_ACTIVE_CIRCUIT (and the profile import guards VALIDATION_ERROR / IS_CLUSTER_DOCUMENT) return HTTP 200 with the standard {success:false, data:null, error:{code, message, details}} envelope. These are decisions the handler makes about a well-formed request — an evidence-rung gate, a layer-contention gate, "no/ambiguous active circuit" — rather than malformed input or a missing resource. The code is still stable and machine-readable; clients must branch on success/error.code, not on the HTTP status, for these. The rich details (the rung, the incumbent, the measured hazard) is what lets the caller re-send with acknowledge_unvalidated or allow_layer_overlap.

Attachable weights errors​

Raised by the /api/weights routes (weight attachment and the step barrier, checkpoints, adapter serving). No refusal ever carries a token, an attachment ID or a path outside the checkpoint roots.

Attachment, step barrier and tokens​

CodeHTTPMeaning
WEIGHTS_UNAUTHORIZED401A gated weights call without a valid bearer token. The response carries WWW-Authenticate: Bearer. details.reason: missing (no Authorization: Bearer header), invalid_token, no_tokens_configured (WEIGHTS_TOKENS is empty, so the gate fails closed) or tokens_malformed (the setting does not parse; the server log names the faulty entry)
WEIGHTS_FORBIDDEN403The token's role may not perform this call; details.required_roles names the roles that may
NOT_ATTACHMENT_HOLDER403A holder-only call whose X-miLLM-Attachment is missing or names another attachment, or whose token label is not the one that attached
ALREADY_ATTACHED409A trainer holds the resident weights (active or stale); details name its role, label, state, attached_at and the refused operation. On /v1 an auto-load refused for this reason is 409 already_attached, typed invalid_request_error
WEIGHTS_UNSAVED409A /v1 request named a model other than the resident while the resident weights carry commits no checkpoint records (commits since the weights were last loaded or last saved). Answering it would auto-load the other model and discard those commits, so nothing is loaded or unloaded. details: operation (auto_load), resident (its served name), resident_model_id, unsaved_commits, requested_model, requested_model_id, recovery. Save the commits (POST /api/weights/save, as the attachment holder) or replace them deliberately (POST /api/weights/load, or the management load POST /api/models/{id}/load); either clears it. On /v1, 409 weights_unsaved typed invalid_request_error; the message carries the same names and count (decision D-45)
NOT_ATTACHABLE400The resident model cannot be attached. details.reason: gguf, quantized, dtype, not_on_gpu, continuous_batching_running, allocator (not loaded as an attachable load), concurrency (MAX_CONCURRENT_REQUESTS above 1) or model_transition (a load, unload or internal write in progress); details.parameter names the first offending tensor where there is one
NOT_ATTACHED409A holder-only call while nothing is attached
NOT_QUIESCED409commit-begin while serving, or resume while a quiesce is still draining (details.reason: quiesce_in_progress); a resume while serving is a no-op success
ALREADY_QUIESCED409quiesce while already quiesced
NO_OPEN_COMMIT409commit-end with no open commit, or from someone other than its owner
ATTACHMENT_NOT_STALE409clear-stale against an attachment that is still sending heartbeats
WEIGHTS_NOT_CLEAN409The operation needs clean weights and a commit is open (writing) or the weights are torn; details.weights_state says which. A holder that re-attached after a watchdog tear gets this from commit-begin and resume (it is checked before the holder check)
WEIGHTS_TORN503The resident weights were left mid-write and are not served. A late commit-end from the writer or a load through POST /api/weights/load restores serving. On /v1, 503 weights_torn typed server_error, with Retry-After
PAUSE_IN_PROGRESS409A serving pause (a save snapshot, an adapter load or unload) is in progress, so a commit-begin, a second pause or a concurrent save is refused; details.operation names the pause. No Retry-After (a 409)
BARRIER_UNAVAILABLE409Serving cannot be paused under this deployment's settings. details.reason: cbm (continuous batching is enabled and generates without the request slot) or concurrency (MAX_CONCURRENT_REQUESTS above 1). A retry does not help; change the setting

Checkpoints​

CodeHTTPMeaning
INVALID_RUN_ID400A run_id outside the save rule or the load rule, or too long for a model name; details: run_id, pattern
MANIFEST_FIELD_NOT_CALLERS400The save request sets manifest fields that miServer writes itself; details.fields names them
INVALID_MANIFEST400A manifest fails the vendored schema; details.errors[] gives each json_path and message
CHECKPOINT_EXISTS409A checkpoint already exists at that step; details.path
CHECKPOINT_AMBIGUOUS409The run exists under more than one checkpoint root (WEIGHTS_CHECKPOINT_DIR and WEIGHTS_CHECKPOINT_READ_ROOTS); details.roots
INSUFFICIENT_STORAGE507Free disk below the checkpoint's estimate times WEIGHTS_SAVE_FREE_SPACE_FACTOR, refused before serving pauses; details: need_bytes, free_bytes, factor
CHECKPOINT_WRITE_FAILED500Writing the checkpoint failed after the snapshot; nothing was published. details: stage, error_id
INVALID_CHECKPOINT_PATH400The path resolves outside every checkpoint root; details.input is what was sent, never the resolved path
CHECKPOINT_NOT_FOUND404No checkpoint by that name or path
CHECKPOINT_HASH_MISMATCH422The weights file's SHA-256 differs from the manifest (expected, actual), or it changed after hashing (changed_since_hash: true). Nothing was written
CHECKPOINT_NOT_LOADABLE400A verified checkpoint miServer will not load as it stands. details.reason: dtype (weights not 16-bit; never cast silently) or remote_code (its config names custom code and the request did not set trust_remote_code: true with an operator token)
WEIGHTS_LOAD_FAILED500A weights load failed after its first write; the weights are torn. details: stage, weights_state, error_id
LOCAL_PATH_REGISTERED409The checkpoint directory is already registered as a plain local model; delete or re-register that model first (details.model_id)
LOAD_THROUGH_WEIGHTS_API409A checkpoint model named by the management load or by /v1; it loads only through POST /api/weights/load, which verifies its hash. details: model_id, weights_origin
MODEL_NAME_TAKEN409Another model already uses the name a checkpoint or a new registration needs (a taken custom_name); details.name, details.model_id
NOT_INSTANTIABLE400Reserved for instantiate, which is deferred

Adapters​

CodeHTTPMeaning
ADAPTER_NOT_SERVABLE400The resident model cannot serve an adapter. details.reason: engine (GGUF), quantization or cbm (continuous batching enabled)
ADAPTER_INVALID400The adapter directory is outside the checkpoint roots, a file is missing or not a regular file, or the manifest names a scaling miServer does not implement. The path is never echoed beyond the root
ADAPTER_MANIFEST_INVALID400adapter_manifest.json fails the vendored schema; each failing field is named
ADAPTER_HASH_MISMATCH400The adapter weights' SHA-256 differs from the manifest; details: expected, actual. Nothing reached the GPU
ADAPTER_BASE_MISMATCH400The manifest names another base model (name or manifest hash); both identities in details
ADAPTER_SHAPE_MISMATCH400A target module is missing, not linear, the wrong shape or rank, fused by transformers, or the adapter carries tensors other than LoRA's A and B. details.reason and the first offender with both shapes
ADAPTER_SCHEMA_NOT_VENDORED409The adapter manifest schema is not installed, so every adapter load is refused
ADAPTER_ALREADY_LOADED409Another adapter is applied (named in details); unload it first
NO_ADAPTER_LOADED409Adapter unload with nothing applied
ADAPTER_BASE_CHANGED409The base model was replaced between verification and the pause; nothing was applied
ADAPTER_INSTALL_FAILED500Installing the adapter failed partway and every installed hook was removed; details.reason, details.error_id
ADAPTER_APPLIED409Probe parity (a comparison against the base) refused while an adapter is applied; details.adapter names it

General​

CodeHTTPMeaning
VALIDATION_ERROR422Request body failed schema validation (also FastAPI's native 422s). A few handlers also return this code in a 200 envelope for semantic input problems — see the 200-envelope note
CONTEXT_LENGTH_EXCEEDED400A /v1 request whose prompt plus max_tokens (or an embeddings input) is longer than the model's context window. details carries max_context_tokens, requested_tokens, prompt_tokens and max_tokens. On /v1 it is 400 context_length_exceeded, typed invalid_request_error; a streamed request is refused before its stream starts. Retrying unchanged cannot succeed: shorten the prompt or ask for fewer max_tokens

Server faults and the error id​

A server fault is any 5xx other than 503 (busy, retry later) and 507 (not enough memory or disk). Its text can wrap an internal exception (a file path, a library's own message), so miServer never sends that text to the client, whatever LOG_LEVEL is set to. The response carries the code, a fixed sentence and an error_id: in details.error_id and details.technical_message on the management API, and in error.message on /v1. The server log line for the fault carries the same error_id with the full text and traceback, so search the log for it:

kubectl -n miserver logs deploy/miserver-backend | grep <error_id>

An unexpected exception (one that is not a miServer error) is answered as INTERNAL_ERROR (500) the same way. The inference and weights blocks of GET /api/health/detailed report a read failure as unavailable (error id …) in their error field, for the same reason.

Handling errors in code​

r = requests.post(f"{MISERVER}/api/saes/steering",
json={"feature_idx": 999999, "value": 40})
body = r.json()
if not body["success"]:
code = body["error"]["code"] # "INVALID_FEATURE_INDEX"
details = body["error"]["details"] # {"feature_idx": 999999, "d_sae": 16384}

For /v1 endpoints, OpenAI SDKs raise their native exceptions (NotFoundError, RateLimitError, …) based on the HTTP status.