Structured Outputs: The Contract Between Your Agent and Everything Else

Somewhere in every agent system there's a line where model output stops being "a message for a human" and becomes "an input to a machine" — a database write, an API call, a workflow trigger, a row in someone's CRM. Everything on the far side of that line was built on an assumption models cheerfully violate: that inputs are well-formed.

I've built LLM pipelines where model output feeds billing systems, scheduling systems, and telephony flows. The lesson, learned the usual way: the boundary between the model and your systems is the most important interface in the architecture, and it must be defended like one. This post is about that defense.

The failure is never where you think

The naive integration — prompt says "respond in JSON," code does JSON.parse(response) — fails in ways that get progressively harder to catch:

Not JSON at all. "Sure! Here's the JSON you asked for:" followed by a code fence. The rookie failure; easiest to catch, first to get fixed.
Valid JSON, wrong shape. A missing required field, a string where you wanted a number, "25%" instead of 0.25. Parses fine. Explodes two services downstream.
Right shape, invalid values. A status of "completed!" when your enum says completed. A date of February 30th. A customer_id that matches the regex but belongs to nobody.
Everything valid, semantically wrong. All checks pass; the refund amount is 10x the order total.

Each level needs a different defense, and the deeper levels are the expensive ones — they don't throw, they corrupt. A parse error pages you at least. A well-formed lie just quietly becomes data.

Level 1–2: enforce the schema, don't request it

Asking nicely in the prompt is not enforcement. Every serious provider now supports actual structured output — constrained decoding against a JSON Schema, or a forced tool call whose input schema is your output schema. Use it. The "please respond in JSON" era is over, and good riddance.

My pattern in TypeScript is one schema, three jobs:

import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

const ExtractionV2 = z.object({
  schema_version: z.literal(2),
  intent: z.enum(["book", "reschedule", "cancel", "question"]),
  customer_phone: z.string().regex(/^\+[1-9]\d{6,14}$/),
  requested_time: z.string().datetime().nullable(),
  confidence: z.number().min(0).max(1),
  notes: z.string().max(500).default(""),
});

// Job 1: sent to the model as the enforced output schema
const jsonSchema = zodToJsonSchema(ExtractionV2);

// Job 2: runtime validation — never skipped, even with
// provider-side enforcement on
const parsed = ExtractionV2.safeParse(JSON.parse(raw));

// Job 3: the static type the rest of the codebase uses
type Extraction = z.infer<typeof ExtractionV2>;

Why validate again when the provider enforced the schema? Because provider enforcement covers structure, not always every constraint (regex patterns, cross-field rules); because you will one day swap providers or hit a fallback path that doesn't enforce; and because the validator is your single choke point — the one door through which model output enters the system. One door is auditable. Five doors are an incident report.

A design note that pays off: keep the schema flat-ish and boring. Deeply nested optionals, unions of unions, clever polymorphism — all of it degrades model accuracy and complicates retry messages. If the schema is hard for you to explain, it's hard for the model to fill.

Level 3: the retry loop that fixes itself

When validation fails, don't fail the request — feed the error back. Models are genuinely good at repairing their own output when told precisely what's wrong, which makes the validate-and-retry loop the workhorse of every structured pipeline I ship:

async function extractWithRetry(input: string, maxAttempts = 3) {
  let feedback = "";
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const raw = await callModel(input, feedback);
    const result = ExtractionV2.safeParse(tryParse(raw));
    if (result.success) return result.data;

    feedback =
      `Your previous response failed validation:\n` +
      result.error.issues
        .map((i) => `- ${i.path.join(".")}: ${i.message}`)
        .join("\n") +
      `\nReturn ONLY corrected JSON matching the schema.`;
  }
  throw new StructuredOutputError(input); // -> fallback path
}

The details that separate a good loop from a flaky one:

Feed back the specific violations, not "invalid, try again." Zod's issue paths are exactly the right granularity — the model fixes the named field and stops.
Cap attempts at two or three. If it fails three times, the problem is your schema or your prompt, not the dice. Retrying five times just converts a design bug into latency and cost.
Have a real fallback. Route to a human queue, return a typed "extraction failed" state, degrade gracefully. The fallback path is part of the design; "throw and 500" is not a fallback.
Log every retry. Retry rate per field is the best schema-health metric you'll get. When requested_time starts failing 8% of the time after a model upgrade, you want a graph that says so, not a support ticket.

In practice, with enforced decoding plus one retry, structural failures become rare. Which is exactly when teams get complacent — and level 4 is still out there.

Level 4: valid is not the same as true

Schema validation cannot tell you the refund is too large, the phone number belongs to a different customer, or the meeting was booked for 3 a.m. That takes semantic checks — plain old business logic that runs after the schema gate:

Cross-field invariants: end time after start time, line items summing to the total, confidence actually consulted before acting.
Referential checks: does this customer_id exist? Does this SKU belong to this catalog?
Plausibility bounds: amounts within policy, dates within the booking horizon, quantities a human would recognize.

And for high-stakes writes, the same two-step shape I use in tool design: the model proposes a typed action, deterministic code (or a human) approves it. The model never holds the pen on anything irreversible. Structured outputs make that possible — you can't route a proposal through an approval gate if the proposal is freeform prose.

Version your schemas like the APIs they are

The part everyone skips until it hurts: that schema is now a contract with every consumer downstream, and it will need to change. Unversioned schema changes in an LLM pipeline are worse than in a normal API, because you have two producers to migrate — your code and the model's learned behavior in the prompt.

What works:

A literal schema_version field in every payload. Persisted outputs become self-describing; the consumer knows how to read a record written four months ago.
Additive changes when possible — new optional fields with defaults don't break old consumers.
Breaking changes get a new version, and both run in parallel during migration. Old traffic validates against V1, new against V2, and your logs tell you when V1 is finally dead.
Re-run your eval suite on every schema change. Renaming a field or tightening an enum shifts model accuracy in ways that are invisible until measured — the schema is part of the prompt, whether you think of it that way or not.
Keep old versions replayable. When you reprocess historical traces — for evals, migrations, or an audit — you need the schema that was live then. Schemas live in version control next to the prompts they ship with.

The principle under all of it

If you take one thing from this post: treat the model as an untrusted client. The same posture you'd take toward a browser sending you form data — never trust, always validate, sanitize at the boundary, version the contract. The model isn't malicious, but it is unreliable in ways you can't enumerate, and from an architecture standpoint that's the same threat model.

Everything above is that principle mechanized: enforced schemas so malformed data can't enter, retries so transient failures self-heal, semantic checks so plausible lies get caught, versioning so the contract can evolve without breaking the past.

The payoff is bigger than safety. Once model output is typed, validated data, the rest of your system gets to be normal software — testable, composable, boring. The probabilistic chaos is contained behind one well-guarded door, and everything downstream of it just works with data it can trust.

Related reading

Designing Tools for Function-Calling Agents
Building Production AI Agents in 2026
The production pipelines where these contracts hold.