# Pathrule Pattern: AI Safety Guardrails (1.0.0) # ::pathrule:package:ai-safety-guardrails ### [RULE] Keep untrusted content outside the instruction channel (path: /src/ai/safety) User, retrieved, uploaded, web, tool, and memory content is labeled as data and cannot rewrite system policy. Attackers can place instruction-like text inside any content source the model reads, including documents that appear internal or previously summarized. Enforce this boundary in /src/ai/safety so invalid work stops before it reaches another subsystem or creates an externally visible side effect. - Build prompts from typed sections that preserve source, trust level, and intended use instead of concatenating strings. - Never interpolate untrusted text into system or developer instructions, tool definitions, or policy templates. - Tell the model the data boundary, but enforce it again with deterministic tool and output controls. - Treat remembered and retrieved content as untrusted unless a separate verified source grants authority. Verification: Seed conflicting instructions in every content source and confirm they cannot change allowed tools, data scope, output policy, or approval requirements. --- ### [RULE] Authorize every tool call outside the model (path: /src/api) A model proposal becomes an action only after schema, principal, tenant, resource, risk, and current-state checks pass. Valid JSON and a permitted tool name do not prove that the current user may act on the requested resource or amount. Enforce this boundary in /src/api so invalid work stops before it reaches another subsystem or creates an externally visible side effect. - Expose the smallest tool surface and use server-derived identity and tenant values rather than model-supplied authority. - Validate arguments with a strict schema and resolve resource identifiers against authorized current state. - Require confirmation or human approval for irreversible, financial, privileged, or externally communicative actions. - Give mutations idempotency keys and record proposal, policy decision, approval, effect, and result without leaking sensitive content. Verification: Attempt cross-tenant IDs, stale state, excessive amounts, hidden side effects, duplicate calls, and approval bypass; verify every rejection occurs before effect. --- ### [RULE] Validate output for its destination (path: /src/ai/safety) Output policy depends on whether content is displayed, stored, executed, indexed, sent externally, or used in a decision. A response safe to show as quoted text may be dangerous when rendered as HTML, executed as code, used as a URL, or treated as verified fact. Enforce this boundary in /src/ai/safety so invalid work stops before it reaches another subsystem or creates an externally visible side effect. - Apply structural and semantic validators for downstream machine consumers before handing them the result. - Escape or sanitize for the actual renderer and keep model text out of raw HTML, shell, query, and template contexts. - Require citations or source checks for factual claims where the product promises grounded answers. - Block, transform, or escalate according to the use-case risk policy and preserve an auditable reason code. Verification: Send adversarial markup, links, code, unsupported claims, secret-like data, and policy-sensitive content through every destination; confirm controls differ by destination. --- ### [MEMORY] Guardrail strength follows use-case risk (path: /src/ai/safety) A writing suggestion, internal search result, code change, support reply, payout decision, and medical summary have different impact, reversibility, and oversight needs. Classify each AI feature by affected people, data sensitivity, autonomy, scale, detectability, reversibility, and legal or product obligations. Bind the tier to allowed data, models, tools, output destinations, evaluation depth, monitoring, approval, and fallback behavior. Reassess the tier when a feature gains tools, new users, broader data, or automated action. Do not lower a control merely because model quality improved; quality and consequence are separate axes. Keep the decision explicit at /src/ai/safety; moving it into an incidental caller makes behavior depend on which route happened to execute first. See /src/api for action authorization and /tests/ai for tier-specific adversarial evidence. That related boundary consumes this decision and carries the evidence that proves it still holds. --- ### [SKILL] red-team-ai-feature (path: /) --- name: red-team-ai-feature description: Red-team an AI feature across prompt injection, data access, tools, output destinations, failure, and human escalation. --- # Red Team Ai Feature Run this procedure whenever the governed surface changes or its operational evidence becomes stale. 1. Map actors, trusted instructions, untrusted data sources, tools, destinations, protected assets, risk tier, and the exact boundaries expected to enforce policy. 2. Create attacks for instruction override, indirect injection, data exfiltration, cross-tenant access, tool argument abuse, unsafe rendering, and approval fatigue. 3. Run cases through production-equivalent retrieval, memory, provider, tool, and UI paths while capturing policy decisions and side effects. 4. Turn successful attacks into deterministic controls and regression fixtures, assign owners, and retest after model, prompt, tool, or data-source changes. Record the decision, failed checks, and follow-up owner with the change. A successful run leaves reproducible evidence that another reviewer can inspect without repeating the investigation from memory.