LLM Structured Outputs

Pathrule2 Rules • 1 Memory • 1 Skill

LLM structured outputs replace prose parsing with an explicit schema contract, but schema-constrained generation does not make values correct or safe. This bundle versions schemas, distinguishes refusal and incomplete states, validates semantics, and migrates consumers deliberately. Unlike AI SDK, it defines a provider-independent data boundary rather than one library's generation and transport conventions.

Suggested path map

Pathrule places each piece on the matching path, so your assistant only sees it where it belongs. This is the scoping you get on import; you can adjust it in your workspace.

/ workspace root

migrate-llm-schema

src/

ai/

Validate syntax, schema, and semantics before side effects

Constrained generation is not trusted generation

schemas/

Version every structured-output contract

Rules

Validate syntax, schema, and semantics before side effects/src/aihighstrictModel output crosses three validation layers before it can mutate state, call a tool, or reach another system.

1	Model output crosses three validation layers before it can mutate state, call a tool, or reach another system. A payload can be valid JSON and match a schema while containing impossible amounts, unauthorized identifiers, stale versions, or unsafe instructions. Enforce this boundary in /src/ai so invalid work stops before it reaches another subsystem or creates an externally visible side effect.
2
3	- Separate transport completion, refusal, truncation, and provider error from the domain result type.
4	- Parse once and validate against the exact schema version requested from the provider.
5	- Apply domain checks for ranges, references, authorization, invariants, and current state after structural validation.
6	- Perform no write, message, charge, or tool call until every layer succeeds.
7
8	Verification: Feed malformed JSON, schema mismatches, refusal, truncation, valid but unauthorized IDs, and a fully valid payload; confirm only the final case reaches effects.

Version every structured-output contract/src/schemashighstrictSchema identity and version travel with prompts, model requests, stored results, fixtures, and consumers.

1	Schema identity and version travel with prompts, model requests, stored results, fixtures, and consumers. Editing a shared schema in place makes old stored output ambiguous and can turn a harmless field rename into a production parsing failure. Enforce this boundary in /src/schemas so invalid work stops before it reaches another subsystem or creates an externally visible side effect.
2
3	- Give each contract a stable name and explicit version independent from the prompt or provider model name.
4	- Keep old readers while stored or in-flight results can still use the old version.
5	- Treat required-field, enum, type, meaning, and nesting changes as compatibility decisions with migration plans.
6	- Generate provider-specific schema subsets from one canonical domain contract when providers support different JSON Schema features.
7
8	Verification: Load fixtures for every supported version through current readers and reject an unknown version before domain work; test one planned migration end to end.

Memories

Constrained generation is not trusted generation/src/aiA provider can constrain shape while the model still selects incorrect, fabricated, stale, or attacker-influenced values.

1	A provider can constrain shape while the model still selects incorrect, fabricated, stale, or attacker-influenced values. Keep the schema at the generation boundary to reduce parser ambiguity, then treat the parsed object exactly like untrusted API input.
2
3	Authorize referenced resources against the current principal and tenant. Resolve identifiers from trusted stores rather than accepting display text as authority. Keep high-impact decisions behind deterministic policy or human review even when the payload is structurally perfect. Keep the decision explicit at /src/ai; moving it into an incidental caller makes behavior depend on which route happened to execute first.
4
5	See /src/schemas for contract versions and /tests/ai for refusal and adversarial fixtures. That related boundary consumes this decision and carries the evidence that proves it still holds.

Skills

migrate-llm-schema/rootMigrate a structured-output schema, stored results, prompts, validators, and consumers without an ambiguous cutover.

1	---
2	name: migrate-llm-schema
3	description: Migrate a structured-output schema, stored results, prompts, validators, and consumers without an ambiguous cutover.
4	---
5
6	# Migrate Llm Schema
7
8	Run this procedure whenever the governed surface changes or its operational evidence becomes stale.
9
10	1. Classify the proposed change as backward compatible, reader-compatible, or breaking for both generation and every downstream consumer.
11	2. Create the new schema version, provider adapter, fixtures, semantic validator, and dual-version reader before switching production requests.
12	3. Replay representative and adversarial inputs, compare refusal and validation rates, and migrate stored output only when its meaning can be preserved.
13	4. Roll out by cohort with version metrics, keep rollback readers, then retire the old version only after in-flight and stored dependencies are gone.
14
15	Record the decision, failed checks, and follow-up owner with the change. A successful run leaves reproducible evidence that another reviewer can inspect without repeating the investigation from memory.

Why this pattern

Agents parse model prose with regex, trust schema-constrained values without semantic validation, or change a shared schema without migrating stored output and consumers.

Built for Teams using model output to drive application state, tools, workflows, or persisted records.

Keeps your assistant from:

Regex parsing of prose presented as a stable contract
Refusals and truncated output treated as valid business data
Schema changes that silently break stored results and downstream consumers

View as markdown

License: Apache-2.0
Version: 1.0.0
Updated: 2026-08-25