Agent Builder & Evals Exit Desk

What shipped

A focused, standalone Python 3.11+ standard-library CLI now handles one complete offline exit-review flow:

• creates one deterministic fictional workflow workspace;
• strictly validates a closed five-file input contract plus one optional frozen receipt pair;
• inventories source nodes, tools, handoff, approval, guardrail, state, output, and eval obligations;
• preserves a human-supplied agents_sdk, workspace_agent, or retire route without choosing it;
• compares exact IDs, contract references, statuses, thresholds, and receipt evidence;
• emits all eleven deterministic exit artifacts;
• records explicit hash-bound accept, retest, or retire decisions with mandatory supersession;
• invalidates prior decisions when any bound evidence or derived base finding changes;
• independently reconstructs and byte-validates the packet.

The fixture is intentionally narrow and exact: one workflow, four nodes, three tools, one handoff, one approval checkpoint, one guardrail, one state expectation, one output contract, twelve evals, one human-selected agents_sdk route, and one frozen receipt. Its intact queue shows the required missing approval, changed handoff, untested session state, and missing eval evidence.

Architecture

• agent_exit_desk/safeio.py: strict UTF-8/JSON/CSV, duplicate rejection, path/reference safety, content canaries, hashing, file closure, and atomic writes.
• agent_exit_desk/contracts.py: closed constants, severities, schemas, immutable finding/analysis records, and the shared scope notice.
• agent_exit_desk/sample.py: deterministic sanitized fixture and parent-compatible frozen receipt.
• agent_exit_desk/engine.py: validation, normalization, exact route/parity/receipt rules, all eleven finding codes, five bindings, and decision-state derivation.
• agent_exit_desk/render.py: allowlist Markdown/CSV/JSON renderers and the self-digesting canonical index.
• agent_exit_desk/commands.py and cli.py: the four-command interface, atomic planning, explicit decisions, and independent validation.
• tests/test_exit_desk.py: 39 unit and adversarial tests spanning all finding codes, clean exact matches, all five states, staleness sources, malformed data, links, closure, leakage, determinism, and output tampering.

The linear pipeline is deliberate: validate safe files → decode closed schemas → hash exact bytes → derive exact facts/findings → derive decision state → render all bytes → atomically replace → independently recompute during validation.

Trimmed scope

• Exactly one workflow, one selected route, at most 50 components, at most 20 evals, and one optional receipt.
• Manual sanitized capture only; no source screenshots or raw export retention.
• Exact ID/reference/status/threshold comparisons only; no fuzzy, semantic, model-based, or weighted scoring.
• Evidence and decision packet only; no automatic route selection, translation, replacement implementation, execution, deployment, or rollback.
• Local files only; no API, OAuth, account access, SDK runtime, connector, browser, database, server, worker, analytics, or telemetry.
• No claim of semantic equivalence, production safety, compliance, certification, or migration success.

Limitations

• The engine cannot identify a source expectation that a human did not capture.
• A matched exact reference is still supplied evidence, not behavioral proof.
• The narrow receipt adapter accepts only the frozen closed 0.1.0 subset; another receipt shape must be converted and reviewed separately.
• Numeric weakening is defined only for the documented score>=N representation; other threshold forms remain exact labels.
• An accepted decision can acknowledge open facts. It intentionally cannot resolve or hide them.
• The tool does not retain or inspect raw prompts, outputs, customer records, tool payloads, or reviewer-private notes.

Suggested next steps

1. Next day, sell one **$2,500 paid-upfront exit assessment** for one workflow, one selected route, up to 20 evals, one packet, and one decision session.
2. Use the manual capture checklist with a named source owner, cutover owner, and reviewer; do not collect raw prompts or customer payloads.
3. Observe which schema fields or finding rules create real review friction before broadening the format.
4. If the assessment converts, anchor a **$6,000 migration-plus-parity sprint** with one structured retest cycle and separately scoped implementation.
5. Add a new read-only adapter only after a buyer supplies a stable export format; keep it fixture-tested and outside the core engine.