# Pathrule Pattern: Model Routing and Cost Control (1.0.0) # ::pathrule:package:model-routing-cost-control ### [RULE] Route from a task policy, never a raw model name (path: /src/ai) Product code requests a named task class with capability, data, latency, quality, and budget requirements. Direct provider and model strings couple business behavior to a changing catalog and let callers bypass routing safeguards. Enforce this boundary in /src/ai so invalid work stops before it reaches another subsystem or creates an externally visible side effect. - Define task classes such as extraction, classification, chat, coding, or high-risk review with explicit required capabilities. - Resolve the route in one service using environment, tenant, region, data handling, and current health signals. - Reject a request when no candidate satisfies mandatory requirements instead of silently relaxing them. - Return the selected route and policy version in internal telemetry without exposing sensitive configuration to clients. Verification: Attempt each task with a missing capability, forbidden data region, exhausted budget, and healthy candidate; confirm only policy-compliant routes are selected. --- ### [RULE] Bound retries and fallbacks by one request budget (path: /src/ai) All attempts share a deadline, token allowance, monetary ceiling, and side-effect identity. Independent provider retries can multiply latency and cost, while a late fallback may repeat tools or return after the caller abandoned the work. Enforce this boundary in /src/ai so invalid work stops before it reaches another subsystem or creates an externally visible side effect. - Allocate attempt count, input and output token ceilings, time, and cost before the first call. - Deduct every failed and successful attempt from the same budget, including cached or speculative work. - Retry only failures classified as transient and stop when the remaining budget cannot complete a useful attempt. - Propagate cancellation and one idempotency identity through model calls, tools, storage, and streaming. Verification: Inject timeouts, rate limits, partial streams, and provider errors at every attempt; verify total time, tokens, cost, and side effects never exceed the route budget. --- ### [RULE] Fallbacks must preserve mandatory behavior (path: /src/ai/providers) A fallback is eligible only when it supports the task's required schema, tools, modalities, context, safety, and data controls. A nominally available model can produce a response that the application cannot parse, authorize, or legally process. Enforce this boundary in /src/ai/providers so invalid work stops before it reaches another subsystem or creates an externally visible side effect. - Maintain explicit capability declarations for every candidate rather than inferring from model-family names. - Run compatibility fixtures for structured outputs, tool calls, streaming events, and refusal handling before enabling a route. - Separate quality downgrade from emergency availability and require product approval for user-visible behavior changes. - Disable a candidate automatically when health or compatibility signals cross the route's policy threshold. Verification: Remove each required capability in a test candidate and confirm routing skips it; simulate degradation and verify the intended circuit and recovery behavior. --- ### [MEMORY] The capability registry is an observed contract (path: /src/ai/providers) Provider documentation describes support, but production routing also depends on the exact account, region, endpoint, model, and integration behavior. Store declared capabilities beside results from recurring compatibility probes and route from the intersection. Track schema subsets, tool semantics, event types, context limits, modalities, data handling, and known incompatibilities. Version registry changes and make route decisions reproducible from the policy and observation snapshot. Quarantine a capability when probes disagree with the declaration until a human reviews the difference. Keep the decision explicit at /src/ai/providers; moving it into an incidental caller makes behavior depend on which route happened to execute first. See /tests/ai for probes and /src/ai for task policy. That related boundary consumes this decision and carries the evidence that proves it still holds. --- ### [MEMORY] Cost belongs to the product operation (path: /src/observability) Provider invoices aggregate usage too late to explain which feature, tenant, route, retry, or prompt caused spend. Emit usage for every attempt against a stable product operation and route decision. Capture provider, model, task class, policy version, cache result, input, output, reasoning where reported, latency, finish state, and normalized cost. Attribute shared context and retry cost consistently so product metrics do not hide expensive failure paths. Keep raw prompts and personal content out of routine cost telemetry. Keep the decision explicit at /src/observability; moving it into an incidental caller makes behavior depend on which route happened to execute first. See /src/ai for budget enforcement and the routing review skill for decisions. That related boundary consumes this decision and carries the evidence that proves it still holds. --- ### [SKILL] review-model-routes (path: /) --- name: review-model-routes description: Review model routes against capability probes, quality evidence, reliability, latency, data policy, and cost. --- # Review Model Routes Run this procedure whenever the governed surface changes or its operational evidence becomes stale. 1. Inventory task classes, mandatory capabilities, candidate order, budgets, retry classes, fallback behavior, and product owners. 2. Run current compatibility and quality fixtures on every candidate using representative context sizes, schemas, tools, and refusal cases. 3. Compare observed latency, error rate, quality, and normalized cost by task and retry path; investigate missing or incomparable usage data. 4. Change one policy variable at a time, roll out by cohort, watch quality and budget guardrails, and preserve the previous policy for immediate rollback. Record the decision, failed checks, and follow-up owner with the change. A successful run leaves reproducible evidence that another reviewer can inspect without repeating the investigation from memory.