Skip to main content
Goal: connect one evaluator without moving your organization’s acceptance rules into a provider response. You are done when: your customized check has a local passed: true report for an allowed case, a blocked case, and an evaluator failure with no verdict. Use this recipe when your team already operates a hosted judge, moderation service, ranking service, or internal scoring API. Do not pass an unchecked provider response directly to Millwork. Allowed low-risk content passes; blocked content rejects. A malformed or unavailable evaluator response returns no verdict.
1

Create the example

Expected result: selected_recipe is c, selected_check ends in recipe-c-evaluator-adapter.mjs, deployed_check ends in selected-check.mjs, and next_action tests selected-check.mjs.
2

Run the local cases

Run the printed command:
Expected result: passed is true. Check the labelled outcomes if a case fails.
  • labelled.c-pass-existing-evaluator as a pass;
  • labelled.c-reject-existing-evaluator as a rejection;
  • labelled.c-technical-invalid-check-result as HTTP 500 with no verdict; and
  • labelled.c-technical-evaluator as HTTP 500 with no verdict.
The labelled technical cases are local listener faults, outside the candidate. Separately, Recipe C rejects an evaluator response that omits a required field as a technical failure.
Recovery: if the CLI does not recognize --recipe, stop. Check the install guide for a recipe-capable version; until it lists one, use its non-recipe path. If a labelled case fails, inspect its named results. Fix the provider translation or your risk limit, then rerun the same local command.
3

Adapt the provider response

Replace callExistingEvaluator with your provider client. Then:
  1. validate every required field and numeric range;
  2. keep your risk limit outside the provider translation;
  3. map stable pass or fail facts to named boolean results;
  4. use the provider score only as quality or an explicit input to your rules;
  5. return a technical failure for timeout, authentication failure, or an unknown response;
  6. keep credentials, raw traces, and free-form provider errors inside your service.
Require a boolean stable_rules.evaluator_policy_passed; an absent or malformed value is a technical failure, not a customer rejection. Rerun the Step 2 command against selected-check.mjs after your edits. Deploy only when your own pass, rejection, and technical-failure cases all pass.The generated example shares one evaluation promise between runHardCheck and scoreQuality. Preserve that sharing so the provider is called once and both outputs use the same response.Before you send a billable evaluator request, the person who owns that account must approve the exact held-out or endpoint-test batch and its spending limit. Local tests use the offline example and incur no evaluator cost.
4

Continue to deployment

Read the shared endpoint contract and reserved-probe limits. Your next executable step is Deploy and test HTTPS. Continue with Recipe 0 for connection, paid-run approval, receipt reading, and recovery. In the final receipt for a checked run, confirm evaluator_allowed, risk_within_organization_limit, and evaluator_policy_passed under named results.
Require the assistant to validate the provider response and keep your hard rules outside it. Supply credentials through your service, never chat. Approve the exact billable test batch before it runs. Approve each paid Millwork run separately after you inspect its preview.