Build and connect an output check

Goal: turn one output check into a tested HTTPS endpoint and use it on one separately approved run. You are done when: a final receipt names the selected output check and shows its verdict or technical failure, quality score, named results, and cost. This route tests request shape, access, timing, and your labelled cases. It does not prove that your rules are right for your product. Your organization owns those rules, thresholds, deployment, and availability.

Create and test the small example

The endpoint contract explains the request and response fields; the timing limits apply when you deploy.

millwork verifier init output-check --recipe 0 --json

Expected result: selected_recipe is 0, selected_check ends in minimal-output-check.mjs, deployed_check ends in selected-check.mjs, and next_action tests selected-check.mjs. The command writes no secret and makes no network call.Run the printed command:

cd output-check
millwork verifier test --local \
  --check selected-check.mjs \
  --access authenticated \
  --json

Expected result: passed is true. The report includes:

See the five local case results

labelled.recipe-0-pass-non-empty as a pass;
labelled.recipe-0-reject-empty as a rejection;
labelled.recipe-0-technical-failure as HTTP 500 with no verdict;
request.malformed_json as a refusal with no verdict; and
failure.check_timeout as HTTP 504 with no verdict.

The local test injects each technical failure outside the candidate. A candidate cannot request one, and deployed testing skips these fixtures.Recovery: if the CLI does not recognize --recipe, stop. Check the install guide for a recipe-capable version; until it lists one, use its non-recipe path. If a labelled case fails, fix the rule or expected case and rerun the same local command. Do not deploy a failed report.

Test an existing check locally

If you already have check functions, export them as runHardCheck and scoreQuality. Add labelled pass and rejection cases. For a technical-failure case, use a local-only fault beside the candidate; never make candidate content switch the check into a failure. Change the export in selected-check.mjs to point to ./your-check.mjs, then test the file your endpoint will use:

millwork verifier test --local \
  --check selected-check.mjs \
  --access authenticated \
  --json

Expected result: passed is true, failed is 0, and every labelled case has outcome passed. This local authenticated test creates temporary loopback keys and needs no account or endpoint key.Recovery: if passed is false, use each failed case ID and reason to repair the rule, expected result, response shape, or timing. Rerun the same command until failed is 0.

Deploy and test HTTPS

Deploy through the application or host your team already owns. The generated existing-node-app.mjs shows how to mount only the output-check route in an existing Node application. It imports selected-check.mjs, which points to the example check chosen by verifier init. Keep both files with that check when you deploy; this is the same selection used by Recipes A-D. Check the endpoint contract and probe and timing limits before you serve the route.Follow the authenticated branch in the generated DEPLOYMENT_RECIPE.md for the default server. Configure an endpoint-only key as MILLWORK_VERIFIER_KEYS at your host. Load that same key privately into MILLWORK_KIT_ENDPOINT_KEY for the deployed test; do not put it in the command. To use a public endpoint instead, configure publicAccess() as the recipe describes and change the test to --access public.The deployed test sends pass and rejection cases plus protocol failure cases. It skips local-only fault cases; no candidate can request one. The endpoint owner must approve that traffic. If the check calls a billable evaluator, its account owner must also approve the exact batch and spending limit.

node check-endpoint.mjs \
  --deployed https://checks.example.com/millwork-check \
  --authorize-endpoint-test \
  --check selected-check.mjs \
  --access authenticated \
  --json

Expected result: passed is true, failed is 0, and every case finishes inside its 3-second or 10-second limit. The report contains no candidate body, endpoint key, or free-form endpoint error.Recovery: a timeout or HTTP 500 is a technical failure. Fix availability or the response shape and test the same deployment again. Do not rewrite it as a negative verdict.

Connect the endpoint

Choose the canonical procedure that matches your access:

Connect a public endpoint when anyone may call it;
Connect a protected endpoint when Millwork must present an endpoint key.

For protected access, stop at Millwork’s private entry page. The person who manages the check enters the endpoint key there. Do not request it in chat or put it in command arguments.Expected result: keep the returned output-check ID, shown as verifier_id. The connection is usable only when its readback is ready and the probe contract is validated.Next, describe the check’s correctness rules, then prepare one run.

Approve one paid run

Follow Use the check on a live run. Inspect the exact preview before you approve it. Connecting the endpoint and testing it do not approve model or evaluator spending. Only you or another authorized person may approve the paid run.Expected result: submission returns a run ID. Submission does not mean the run is complete.Wait for a final status, then open the result and receipt.

Read the result and receipt

The result shows whether the output check recorded a hard decision and quality score. The receipt records the attempted check, its ID and definition hash, stable boolean results, cost, and any failure class.

is_correct: false is a rejection even when quality_score is high.
A missing verdict or NOT CHECKED means the check was unavailable.
The receipt does not retain evaluator traces, raw datasets, prompts, or outputs.

Read Results and receipts for field meaning.

Technical reference

The endpoint contract

Millwork sends an HTTPS POST with a candidate. A successful response has a hard decision and a quality score; named boolean results are optional.

See request and response JSON

The request body has one field:

{
  "candidate": "the model output, or structured agent output"
}

The endpoint returns HTTP 200 and:

{
  "is_correct": true,
  "quality_score": 0.84,
  "anchor_results": {
    "named_hard_rule": true
  }
}

is_correct is the required hard decision. quality_score is required and ranges from 0 to 1. anchor_results, when present, maps stable names to booleans. The API calls the output check a verifier, so its saved ID appears as verifier_id in commands and receipts.The request does not contain a run ID, organization ID, model ID, or output check ID. Keep datasets, traces, prompts, credentials, and provider metrics inside your endpoint.

Reserved probe and timing

Registration sends a reserved probe that the generated server answers without calling your rules.

See the reserved probe JSON

The exact candidate is:

{
  "solverapi_probe": "registration_preflight"
}

The generated server returns is_correct: false and quality_score: 0 without calling your rules. It also refuses that marker when nested inside an ordinary candidate.

Your deployed endpoint must:

use internet-reachable HTTPS and resolve to a public IP;
follow no redirect;
accept chunked POST bodies;
answer the reserved probe within 3 seconds;
answer a run evaluation within 10 seconds;
return less than 1 MiB; and
expect no automatic retry from Millwork.

A timeout, unreachable endpoint, authentication refusal, or malformed result is a technical failure. It has no verdict.

Minimal server examples

The generated project includes minimal-node-dock.mjs and minimal-python-dock.py. Both implement the same non-empty-output example and reserved probe. The Python file is a small public-access wire example. For a protected production endpoint, use the Node server or implement the same authentication-before-body-read, size, timeout, and error limits.

Show local run commands

Run one server:

node minimal-node-dock.mjs

python3 minimal-python-dock.py

Expected result: the process listens on PORT, defaulting to 8080. In a second terminal, send the reserved probe and an ordinary candidate:

curl -sS http://localhost:8080/ \
  -H 'content-type: application/json' \
  --data '{"candidate":{"solverapi_probe":"registration_preflight"}}'

curl -sS http://localhost:8080/ \
  -H 'content-type: application/json' \
  --data '{"candidate":{"summary":"ready"}}'

The probe returns is_correct: false and quality_score: 0. The ordinary candidate returns is_correct: true and anchor_results.non_empty_output: true. Press Ctrl-C in the server terminal before you change the selected check.

Recover without starting work twice

Use endpoint-test recovery for reachability, authentication, timeout, and response-shape failures. Use protected-connection recovery for continue, replace, disconnect, and restore behavior. When a request outcome is unknown, reuse the original request key so a retry does not start the work twice. Read the connection before you claim that a new key or state is active.

If an assistant helps with this recipe

Require the assistant to show passed: true and failed: 0 locally before deployment. Approve deployed test traffic and any evaluator cost before it sends requests. Enter endpoint keys yourself. Inspect the exact run preview and approve each paid run separately. Finish only after the final receipt shows the selected output check and the expected decision or technical failure.

Output checks

Operations recipes

Build and connect an output check

Technical reference

The endpoint contract

Reserved probe and timing

Minimal server examples

Recover without starting work twice

​Technical reference

​The endpoint contract

​Reserved probe and timing

​Minimal server examples

​Recover without starting work twice

Technical reference

The endpoint contract

Reserved probe and timing

Minimal server examples

Recover without starting work twice