> ## Documentation Index > Fetch the complete documentation index at: https://docs.getmillwork.dev/llms.txt > Use this file to discover all available pages before exploring further. # Run a DSPy program with a Jev check > Serve your DSPy program as an HTTPS endpoint, check each answer with a Jev probability, and get a receipt for every approved run. **Goal:** run your DSPy program and Jev check through Millwork when you need an approved run and a receipt. ## See it in a warranty answer In the included offline example, someone asks whether a refurbished laptop has a warranty. DSPy first proposes a lifetime warranty, but the trusted policy promises 12 months. A scripted Jev decision scores the first answer low, so DSPy tries again. Your output check asks whether the cited policy passage supports the final answer. This diagram shows the checks you try offline and the receipt produced when you run the same endpoints through Millwork. ```mermaid theme={null} flowchart TB A["Your app: DSPy proposes a lifetime warranty"] --> J["Jev: low support from the policy"] J -->|Try again| B["Your app: DSPy answers 12 months"] B --> V["Your output check: quote and separate Jev decision"] V -->|Supported| R["Millwork receipt: saved agent, check and verdict"] classDef customer fill:#EEF4F6,stroke:#7895A0,color:#211F1B,stroke-width:1px classDef external fill:#F3F0F8,stroke:#9A86B8,color:#211F1B,stroke-width:1px classDef decision fill:#F8F6F3,stroke:#B92B2B,color:#211F1B,stroke-width:1px classDef receipt fill:#EDF6EF,stroke:#4E8A60,color:#211F1B,stroke-width:1px class A,B customer class J external class V decision class R receipt linkStyle default stroke:#4B5563,stroke-width:1px ``` For an approved live run, Millwork calls the same Python app and records which saved agent and output check handled it, plus the verdict. Your DSPy program and acceptance rules run in your Python app. The model provider and TypeSafe receive the inputs your app sends them; Millwork's receipt stores neither the question, answer, nor passage. The saved agent and check can handle later approved runs; your app chooses which questions to route. ## Where else this fits The same pattern works for short answers grounded in source passages your team controls. The warranty example is included; each idea below needs its own source and offline cases: Bring the current return policy. Test a correct answer, a wrong return window, and a missing quote. Bring approved source excerpts. Check one cited claim at a time; test a misquote and a contradiction. Bring the reference for the right API version. Test a supported claim and a quote from an older release. For each adaptation, replace the sample answerer and passage store with your DSPy module and trusted sources. Your app controls source freshness and access. The check can judge whether a cited passage supports an answer; it cannot prove the answer completes the user's whole task. For other kinds of Jev decisions, explore [TypeSafe's use-case map](https://docs.typesafe.ai/concepts/use-case-map); those need their own question, evidence, and tests.
No account or key. Run the included cases before adapting the program or sending live traffic.
**You are done with the offline example when:** `offline_cases.py` reports `4 passed, 0 failed` and `check_locally.py` reports `14 passed, 0 failed`. After an approved live run, its receipt names your saved agent, output check, and verdict; the usage report separates your answers and Jev decisions from Millwork's accepted runs. TypeSafe's **Jev** returns a probability that a passage supports an answer. DSPy's **BestOfN** tries up to your set limit and stops when an answer reaches your threshold. Your output check applies exact quote rules and a separate Jev decision to the final answer. ## Before you begin **For the offline example:** Python 3.10 or newer. It needs no account, key or credit. **For the endpoint test and the live run:** * Node.js and the Millwork CLI release that [Set up with the CLI](/get-started/tenant-start#run-your-next-task) installs, plus `curl` and `jq`. Run `millwork --version --json` first and confirm it reports that release; use it for every CLI command on this page. * An organization API key in your secret store, available as `SOLVERAPI_API_KEY`, as described in [Get your first model answer](/get-started/tenant-solver#before-you-begin). * A TypeSafe account for Jev, and an account with the model provider your DSPy program already uses. * Millwork credit and quota for one run, and an administrator who can register your agent endpoint's key and give you a credential handle, a reference to that key. Without that handle you can complete only the offline steps. Give this page to your coding agent. Ask it to run the offline example, return both test results and name the files it would adapt, then stop at each **Approval** line. The named owner approves that action and enters keys privately. The example is a small policy-answering app. `program.py` holds a DSPy module standing in for yours, wrapped in a bounded BestOfN. `check.py` holds the output check, served through a Python output-check adapter. Copy the downloader into `dspy-example.py`, review it, then run it. It creates a `dspy-jev-governed-run` directory and checks every file against the manifest before writing anything. ```python theme={null} # Generated from the DSPy + Jev example. Review before running. import hashlib import json import pathlib import re import urllib.request target = pathlib.Path("dspy-jev-governed-run").resolve() url = "https://docs.getmillwork.dev/downloads/dspy-jev-governed-run.json" expected = "86e8ba2fd0c548c255b99b8a8c7b114e6b3e2d0378caa2263fe9f4cd29c1b949" if target.exists(): raise SystemExit("Target directory already exists") with urllib.request.urlopen(url, timeout=30) as response: data = response.read(300001) if len(data) > 300000: raise SystemExit("Example bundle is too large") bundle = json.loads(data.decode("utf-8")) files = bundle.get("files") if not isinstance(files, list) or len(files) != 25: raise SystemExit("Unexpected file inventory") seen, checked = set(), [] for entry in files: name, digest, content = entry.get("path"), entry.get("sha256"), entry.get("content") parts = name.split("/") if isinstance(name, str) else [] if not parts or name.startswith("/") or "\\" in name or any(part in ("", ".", "..") for part in parts) or name in seen: raise SystemExit("Unsafe file path") if not isinstance(digest, str) or not re.fullmatch(r"[0-9a-f]{64}", digest) or not isinstance(content, str): raise SystemExit("Invalid file entry") path = (target / name).resolve() if target not in path.parents: raise SystemExit("Unsafe target path") raw = content.encode("utf-8") if hashlib.sha256(raw).hexdigest() != digest: raise SystemExit(f"Checksum mismatch: {name}") seen.add(name) checked.append((name, digest, path, raw)) manifest = "".join(f"{name}\0{digest}\n" for name, digest, _, _ in checked) if hashlib.sha256(manifest.encode("utf-8")).hexdigest() != expected or bundle.get("aggregate_sha256") != expected: raise SystemExit("Bundle manifest mismatch") for _name, _digest, path, raw in checked: path.parent.mkdir(parents=True, exist_ok=True) with open(path, "xb") as handle: handle.write(raw) print(f"Created {target} with {len(checked)} checked files.") ``` ```bash theme={null} python3 dspy-example.py cd dspy-jev-governed-run python3 -m venv .venv . .venv/bin/activate pip install -r requirements.txt python offline_cases.py python check_locally.py --check offline_check.py ``` **Expected:** `4 passed, 0 failed` and `14 passed, 0 failed`. `offline_cases.py` walks four questions through both endpoints: an answer supported on the first attempt, one supported on the second, an abstention after three unsupported attempts, and a Jev outage. `check_locally.py` checks supported, contradicted, missing-source, low-confidence, misquoted and abstained answers, and an unavailable check. It sends each answer again as model text, which must get exactly the same result, and confirms that the quality score never changes a verdict. Scripted answers stand in for your model and for Jev, so neither command needs a key or makes a paid call. Both still run DSPy's `Predict`, BestOfN and the code that turns Jev's answer into a decision, as production does. Run every later command from this `dspy-jev-governed-run` directory unless a step says otherwise. The same app serves both endpoints on your machine with the scripted answers. The two keys below are local demo values that only this offline server accepts; never use them for a deployed endpoint. ```bash theme={null} export MILLWORK_AGENT_KEYS="offline-demo-agent" MILLWORK_VERIFIER_KEYS="offline-demo-check" python app.py --offline & APP_PID=$! READY="" for attempt in $(seq 60); do if curl -sf http://127.0.0.1:8080/healthz > /dev/null; then READY=yes; break; fi kill -0 "$APP_PID" 2> /dev/null || break sleep 0.5 done if [ -n "$READY" ]; then curl -s --header "Authorization: Bearer offline-demo-agent" \ --header "Content-Type: application/json" \ --data '{"task":{"objective":"Can I return an unused item three weeks after delivery?"}}' \ http://127.0.0.1:8080/millwork/agent | tee answer.json jq '{candidate}' answer.json | curl -s --header "Authorization: Bearer offline-demo-check" \ --header "Content-Type: application/json" --data @- \ http://127.0.0.1:8080/millwork/check else echo "app.py did not answer /healthz within 30 seconds; see its output above." fi kill "$APP_PID" 2> /dev/null ``` **Expected:** the agent returns an answer with its `source_id` and `quote`, and the check returns `"is_correct": true` with four named results. The app also prints one count record per endpoint; those lines hold numbers and outcome codes only. Change these files and keep the rest: | What you already have | Where it goes | | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Your DSPy module | Replace `PolicyAnswerer` in `program.py`, or pass a module you already built or loaded to `build_live_components(..., answerer=your_module)`. It takes `question` and `passages`, returns `answer`, `source_id` and `quote`, and makes its model calls through DSPy predictors. If it makes more than one call per answer, also pass `model_calls_per_attempt`. | | The passages your answers rely on | Replace the file-backed store in `trusted_store.py` with the source you already trust. The check reads each cited passage from here, never from the answer. | | Your Jev question | Edit `SourceSupport` in `jev.py`. It returns a probability, using TypeSafe's `Noul` [question type](https://docs.typesafe.ai/primitives). | | Your acceptance rule | `support.py`: the exact checks, then `SUPPORT_THRESHOLD` (0.8). | | How many answers to try | `attempts` in `GovernedPolicyProgram`: 3 by default, 5 at most. Each attempt can produce one answer and at most one Jev decision. The run's model calls are capped at `attempts` times `model_calls_per_attempt` (1 by default), and all attempts must finish inside the program's 25-second deadline. | If the quote is not in the cited passage, the check rejects the answer without asking Jev. It passes an answer only when the exact checks hold and Jev's probability reaches your threshold. Add your own offline cases before going live. The scripted model answers live in `offline.py`: give each question one scripted answer per attempt, and include every output field your module declares, such as `reasoning` for `dspy.ChainOfThought`. Add a `JourneyCase` in `offline_cases.py` for each new question, and your check's cases to `check_cases.json` and `heldout_cases.json`. Include a pass, a contradiction, a missing source, a low-confidence answer and an unavailable check. Then rerun both offline commands. Offline runs use scripted answers, so they test your check and your wiring, not your prompt; judge the prompt on the live path. **Cost:** from this step on, some commands cost money. TypeSafe bills each Jev decision, and your model provider bills each answer your program generates, including every BestOfN attempt. Millwork charges its platform fee only when it accepts a live run, which happens in the last steps. [Model access and billing](/concepts/access) gives the current amount and the full rule. Local cases, endpoint tests, BestOfN attempts and Jev calls on their own are not Millwork runs and carry no platform fee. The threshold is a decision rule you choose from labelled answers, not a calibrated chance of being right. `evaluate_quality.py` compares the check with the exact-quote check alone. It shows false acceptance and false rejection at every threshold from 0.50 to 0.95, with the latency and TypeSafe requests for the whole run. First see the report's shape with scripted answers: ```bash theme={null} python evaluate_quality.py --offline ``` Its numbers are not a measurement: the scripted answers give every labelled answer the same probability, 0.3, so every row rejects all supported answers. For the real measurement, run `python evaluate_quality.py --live`. It prints how many TypeSafe requests it needs and stops. **Approval:** the TypeSafe account owner approves that many requests. Then the same person loads the key in their own terminal, so it never appears in a command, a file or a chat: ```bash theme={null} read -rs TYPESAFE_API_KEY && export TYPESAFE_API_KEY APPROVED_COUNT="" python evaluate_quality.py --live --authorize-typesafe-requests "$APPROVED_COUNT" ``` **Expected:** a report marked `"evidence": "live"`. Set `SUPPORT_THRESHOLD` in `support.py` to the `jev_check.by_threshold` row whose false acceptance and false rejection your organization accepts. If Jev times out, refuses or returns a malformed answer, that case counts as a technical failure with no probability. The report holds case IDs and numbers, never answer or passage text. Deployment can incur hosting charges. The endpoint test at the end of this step sends paid Jev requests. Deploy through the application or host your team already owns, following the deployment step of [Build and connect an output check](/cookbook/output-checks/build-the-dock#deploy-and-test-https) for HTTPS, access and test approvals. Copy the `dspy-jev-governed-run` directory to the host and install `requirements.txt` there. Configure these values in the host's secret settings: | Setting | Holds | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `MILLWORK_AGENT_KEYS` | The key Millwork presents to `/millwork/agent`. | | `MILLWORK_VERIFIER_KEYS` | A different key, for `/millwork/check`. | | `GENERATOR_MODEL` | The language model (LM) your DSPy program uses, such as `openai/gpt-4o-mini`, plus that provider's own key setting. | | `TYPESAFE_API_KEY` | Read by the TypeSafe library; the example never reads or logs it. | | `MILLWORK_COUNTER_LOG` | A file for the count records on a volume that survives restarts, such as `/var/log/app/counters.jsonl`. When it is unset, the records go to the app's error output; export them from your log platform instead. | Neither endpoint key is a Millwork organization API key. Both are read on each request, so you can rotate them by changing the settings. Run the app with a production WSGI server behind your HTTPS ingress, reachable only from that ingress. `wsgi.py` builds the app once per worker: ```bash theme={null} gunicorn --worker-class gthread --threads 8 --timeout 60 --bind 0.0.0.0:8080 wsgi:application ``` Use a threaded worker: each agent call holds a thread for up to the 25-second program deadline. `python app.py` starts a single-process server for the offline walkthrough only. If your web app is ASGI, such as FastAPI, run this as its own service and route `/millwork/agent` and `/millwork/check` to it from your ingress. `GET /healthz` reports the example, adapter, DSPy and Python versions only. Then test the check endpoint with the endpoint test kit: Node scripts, created by the CLI, that send labelled requests to your check, including cases meant to fail. `check-cases.mjs` gives the kit each case in `check_cases.json` once, as model text, and checks its verdict and named results. The exact match between the JSON and model-text forms of a case was already checked offline by `check_locally.py`; against a live endpoint each form would be a separate Jev request, and two requests need not return the same probability. Only cases whose quote is in their passage reach Jev. Count the requests one kit run sends with your current cases: ```bash theme={null} python kit_request_bound.py ``` **Approval:** the endpoint owner and the TypeSafe account owner approve the printed count for each kit run you plan. Ask again before any run beyond that. Then create the kit inside `dspy-jev-governed-run`, and have the endpoint owner load the check key with `read -rs MILLWORK_KIT_ENDPOINT_KEY && export MILLWORK_KIT_ENDPOINT_KEY`: ```bash theme={null} millwork verifier init output-check --json cd output-check node check-endpoint.mjs \ --deployed https://app.example.com/millwork/check \ --authorize-endpoint-test \ --check ../check-cases.mjs \ --access authenticated \ --json cd .. ``` **Expected:** `passed` is `true` and `summary.failed` is `0`; the local-only `evaluator-unavailable` case shows as not applicable. Passing means the deployed endpoint answers Millwork's contract with your access settings, and live Jev gives each labelled case the verdict you expect. If a case gets a different verdict, your labelled case and live Jev disagree: fix the case or your threshold offline, then get approval before testing again. The kit is a test tool on your workstation; the deployed app does not need Node. The API calls a saved agent an `arm`. Ask your administrator to register the agent endpoint's key (`MILLWORK_AGENT_KEYS`) and give you its credential handle. The key itself never appears in your request. Save this body as `dspy-agent.json`: ```json theme={null} { "kind": "agent", "display_name": "DSPy policy answerer", "endpoint": { "url": "https://app.example.com/millwork/agent", "auth_ref": "" }, "capability_tags": ["policy_answers"], "data_class_grants": ["sandbox"], "lifecycle": { "long_running": false, "cancellable": false, "max_runtime_s": 30 } } ``` **Approval:** saving the agent sends one authenticated `HEAD` request, a probe, to the agent endpoint, which answers it without running your program. The endpoint owner approves that probe. Then save the agent with your organization API key: ```bash theme={null} set -o pipefail curl --fail-with-body --request POST \ --header "Authorization: Bearer $SOLVERAPI_API_KEY" \ --header "Content-Type: application/json" \ --data @dspy-agent.json \ --output dspy-agent-response.json \ "https://api.getmillwork.dev/v1/arms" && jq -e '{arm_id, status, status_reason}' dspy-agent-response.json ``` **Expected:** `status: "ready"` and an `arm_id`, which the run request reads from `dspy-agent-response.json`. A degraded status with `authentication_failed` means the handle's key and `MILLWORK_AGENT_KEYS` differ. The API calls an output check a verifier (`verifier_id`). Connecting and testing the output check have no Millwork platform fee, and neither calls Jev. Connecting sends two test requests to your endpoint: a probe without the key, which the adapter refuses, then a test that checks the key. `millwork verifier test` sends one that checks the endpoint contract. `CUSTOMER_APP_ORIGIN` is the Millwork app you sign in to, not your endpoint. **Approval:** the endpoint owner approves these three test requests. Then: ```bash theme={null} export VERIFIER_URL="https://app.example.com/millwork/check" export CUSTOMER_APP_ORIGIN="https://app.getmillwork.dev" millwork verifier connect \ --endpoint "$VERIFIER_URL" \ --access managed \ --name "Policy answer check" \ --version "1.0.0" \ --stop-days 90 \ --connect-only ``` The endpoint owner enters the `MILLWORK_VERIFIER_KEYS` key in the hidden terminal prompt; the command then prints `Verifier connected.` Without a private terminal, it returns `state: "action_required"` with a `verifier_id`, a private `continue_url` and an `intent_id`. Open `continue_url` in your browser, signed in to the same organization, enter the key there, then finish the connection: ```bash theme={null} millwork verifier continue \ --verifier-id "" \ --intent-id "" \ --json ``` Keep the `verifier_id` once the command printed `Verifier connected.` or the `continue` output shows `connection.status: "active"`. Then test it: ```bash theme={null} export VERIFIER_ID="" millwork verifier test \ --verifier-id "$VERIFIER_ID" \ --idempotency-key "verifier-test-$(node -e 'process.stdout.write(crypto.randomUUID())')" \ --json ``` **Expected:** `headline: "ready"`, `probe.contract.validated: true` and the same `verifier_id`. The reserved probe may return `is_correct: false` on purpose; that does not make the endpoint unusable. To repair a connection, see [protected-connection recovery](/guides/connect-an-output-check#a-protected-connection-needs-recovery). **Cost:** Millwork's platform fee, charged when it accepts the run; see [Model access and billing](/concepts/access) for the current amount. With the default three attempts, one run makes at most three model calls and sends at most four Jev decisions: one per attempt and one in the check. The request's `max_cost_usd` limits only the model spend Millwork records, and this example reports none, so it does not limit your provider or TypeSafe bills; the program's attempts and deadline limit those. Build the request. `run_ref` is a new random ID the usage report uses to match the run to its counts, so it never holds customer data. The request key lets Millwork recognize a retry, so save it for any retry: ```bash theme={null} RUN_REF="run-$(node -e 'process.stdout.write(crypto.randomUUID())')" RUN_KEY="run-key-$(node -e 'process.stdout.write(crypto.randomUUID())')" printf '%s\n' "$RUN_KEY" > run-key.txt jq -n \ --arg run_ref "$RUN_REF" \ --arg arm "$(jq -er '.arm_id' dspy-agent-response.json)" \ --arg check "$VERIFIER_ID" \ '{mode: "live", task: {objective: "Can I return an unused item three weeks after delivery?", inputs_ref: {kind: "inline_json", json: {run_ref: $run_ref}}}, policy: {data_classes: ["sandbox"], budget: {max_cost_usd: 0.05, max_runtime_s: 60}, on_eval: []}, routing: {required_arm_id: $arm}, verifier_id: $check}' > run-request.json cat run-request.json ``` `routing.required_arm_id` runs the task on your saved agent, and `verifier_id` selects your check. Without `verifier_id`, Millwork would apply only a basic check that output is present. **Approval:** a person who can authorize paid runs reviews `run-request.json` and approves this one run. Then submit it: ```bash theme={null} set -o pipefail curl --fail-with-body --request POST \ --header "Authorization: Bearer $SOLVERAPI_API_KEY" \ --header "Idempotency-Key: $(cat run-key.txt)" \ --header "Content-Type: application/json" \ --data @run-request.json \ --output run-execution.json \ "https://api.getmillwork.dev/v1/executions" && jq -e '{execution_id, status}' run-execution.json ``` **Expected:** a run ID and a queued status, not a finished run. If the response is lost, send the same command again: the saved request key returns the same run instead of starting a second one. Check the status until it is `completed`, `failed`, `cancelled` or `expired`, as [Wait for a final status](/guides/execution-lifecycle#wait-for-a-final-status) describes: ```bash theme={null} EXECUTION_ID="$(jq -er '.execution_id' run-execution.json)" curl --fail-with-body \ --header "Authorization: Bearer $SOLVERAPI_API_KEY" \ "https://api.getmillwork.dev/v1/executions/$EXECUTION_ID" | jq -er '.status' ``` Then read the receipt: ```bash theme={null} curl --fail-with-body \ --header "Authorization: Bearer $SOLVERAPI_API_KEY" \ "https://api.getmillwork.dev/v1/receipts/$EXECUTION_ID" | jq -e '.slices[0] | {agent: .route.selected_arm, check: .verifier.verifier_id, verdict: .verifier.is_correct, results: .verifier.anchor_results, recorded_check: .recorded_check}' ``` **Expected:** `agent` is your `arm_id`, `check` is your `verifier_id`, and `verdict` is the check's decision with its four named results. The receipt shows the agent attempt at \$0 because the example reports no cost to Millwork; your provider and TypeSafe bills carry that spend. What other outcomes look like: | What you see | What it means | | -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `verdict: false`, status `completed` | The check rejected the answer, or your program abstained. With `on_eval: []` Millwork does not retry. It records `needs_review` on the run's lifecycle event and sends the `execution.needs_review` webhook if you subscribe to it. | | No verdict, `recorded_check.state` is `customer_unavailable` | Your check could not decide, for example Jev did not answer in time. Its `failure_class` says why. | | No verdict, `recorded_check.state` is `terminal_no_evaluation` | The run failed before the check ran. A `dispatch_timeout` reason means the agent call ran past its time limit. | [Results and receipts](/concepts/results-and-receipts) explains the verdict, quality score and named results. Copy the count file that `MILLWORK_COUNTER_LOG` names from your host into this directory, or export the records from your log platform. Save Millwork's usage report for the month the run was accepted, read from the run's own record: ```bash theme={null} RUN_MONTH="$(jq -er '.created_at[0:7]' run-execution.json)" curl --fail-with-body \ --header "Authorization: Bearer $SOLVERAPI_API_KEY" \ --output usage.json \ "https://api.getmillwork.dev/v1/usage?period=$RUN_MONTH" python usage_report.py \ --counters counters.jsonl \ --run run-request.json run-execution.json \ --usage usage.json ``` **Expected:** an `executions` entry for your run ID with the counts your app recorded, and a `millwork` section with the runs Millwork accepted in the whole period. The report skips log lines that are not count records. | Count | Where it appears | Billed by | | ------------------------------------------------ | --------------------------------------------------- | ----------------------- | | Model calls your program made | `generator_calls` in the run's entry | Your model provider | | Answers your program generated | `best_of_n_candidates_generated` in the run's entry | Included in model calls | | Jev decisions sent, in the program and the check | `typesafe_requests_sent` in the run's entry | TypeSafe | | Runs Millwork accepted in the period | `millwork.accepted_executions` | Millwork | `generator_calls` counts every model call, including ones whose output DSPy could not parse (`generation_failures`); a call beyond the run's cap or after the deadline is refused, not sent. `typesafe_requests_sent` includes requests that failed, which `typesafe_failures` counts again on their own. Decisions answered from DSPy's cache are counted in `typesafe_cache_hits` and are not sent. Count records hold numbers, outcome codes and your `run_ref` only: never a question, answer, passage or key. Records from local cases and endpoint tests have no matching run and appear under `unattributed`. To use this path for another eligible question, keep the saved agent and output check. Build a new request with a fresh `run_ref` and request key, and get approval for that paid run. Your app decides which questions to route; finishing this example does not route future questions automatically. ## Time limits Millwork waits at most **30 seconds** for the agent call, or less when the saved agent's `max_runtime_s` or the run's remaining time is smaller. The example's program deadline, 25 seconds, usually ends first and answers `504 program_timeout`, a technical failure with no verdict. A run that waited long in the queue can reach Millwork's limit first instead. A program that needs longer is not suited to this recipe. Each model answer has an 8-second limit and each Jev decision a 6-second limit, with no automatic retries. After a `504`, no new model answer or Jev decision starts. A call already in flight finishes, and your app then writes a late-work record with its counts, which the usage report adds to the run. The check's Jev decision runs inside Millwork's 10-second wait for a check. ## What the receipt does not show The receipt records the check's identity, verdict, quality score and named results. It does not keep the question, the answer, the passage or Jev's response. The check sees only the answer. It checks whether the cited passage supports the answer, not whether the answer completed the customer's task. ## Recover without starting work twice | Symptom | What to do | | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | Saving the agent returns `authentication_failed` | Register the handle again with the exact key in `MILLWORK_AGENT_KEYS`. | | The receipt shows `dispatch_timeout`, or your logs show `program_timeout` | Lower the BestOfN attempts or your generator's latency, then send a new run. | | The receipt shows `customer_unavailable` | Your check could not decide. Check TypeSafe's status and your key, then send a new run. | | The agent endpoint answers `503 ranking_unavailable` | Jev did not answer while your program scored its answers. Check TypeSafe's status and your key, then send a new run. | | The check test fails | Follow [endpoint-test recovery](/guides/connect-an-output-check#the-endpoint-test-fails). | | A submission's outcome is unknown | Send the same command again with the request key saved in `run-key.txt`. Read [Errors and retries](/guides/errors-and-retries). | For a Jev check without DSPy, start from [Check citations with Jev](/cookbook/output-checks/jev-citation-check).