Prove · Structured output

Structured output

Define a JSON schema, run a prompt against it, and see either the validated output or the exact field that failed. Getting a model to return usable JSON is a skill you otherwise rebuild in every project. This is the same schema validation RepoOps runs on itself, turned into a surface you can use.

Where to find it

  • Localhost: /structured-output.html
  • API: POST /api/structured-output/run, POST /api/structured-output/repair, GET and POST /api/structured-output/schemas, POST /api/structured-output/schemas/delete, POST /api/structured-output/to-eval-bar
  • Sidebar: Prove, then Loop Engineering, then Structured output

What it does for you

Names the field that failed, not just that it failed.Every validation error carries a path into the document, so /items/0/name points at the offending field rather than leaving you to diff two blobs. A model that returns malformed or non-conforming JSON reads invalid with those errors attached; it never reads as a quiet pass.
Retries with the errors fed back.The repair route re-prompts on invalid output, bounded, handing the model the exact schema errors from the previous attempt rather than asking it to try again blind.
Keeps the schemas you use.Save a schema under a name, list what you have saved, and delete one you no longer want. Turn any run into a versioned eval bar with one conformance case, so a schema you care about becomes something CI can hold the model to.
Fail-closed with no key.This runs on your own Anthropic key. With no key set, the route returns an explicit blocked state. It does not fabricate output, and it does not report a pass it could not verify.

Built vs. planned

All five routes ship today, backed by lib/routes/structured-output.mjs. The validator is dependency-free by decision: a small JSON-schema subset covering type, required, properties, items, enum, minimum and maximum, minLength and maxLength, and additionalProperties: false. That subset covers the structured-output use and keeps the repo's five-dependency budget intact, so a schema using a keyword outside it is not enforced. The pure pieces (validateAgainstSchema, evaluateStructuredOutput) are exported and tested without the network.

Last updated