# Pathrule Pattern: Elasticsearch and OpenSearch (1.0.0) # ::pathrule:package:elasticsearch-opensearch ### [RULE] Define mappings before indexing production data (path: /search/mappings) The first document can choose a dynamic field type that later documents cannot use, and many mapping changes cannot be applied in place. Explicit mapping turns ingestion assumptions into reviewable schema. - Map identifiers, enums, and filter or aggregation fields as exact values; map human language as analyzed text with an intentional analyzer. - Use multi-fields when the same source value needs full-text relevance and exact sorting or aggregation, and name their roles consistently. - Map arrays of objects as nested when queries must preserve relationships within each object; ordinary object arrays flatten values and can create cross-object matches. - Reject or tightly control unexpected dynamic fields for untrusted or high-cardinality input so one malformed document cannot expand cluster mappings indefinitely. See /search/queries for the adjacent decision or procedure that completes this constraint. --- ### [RULE] Bound search work and result windows (path: /search/queries) A search request can be syntactically valid while consuming unbounded heap or CPU through deep pagination, broad wildcard expansion, large aggregations, or scripted work over many documents. - Search only fields relevant to the feature and apply filters in filter context when scoring is unnecessary. - Use cursor-style search-after pagination with a deterministic sort for deep traversal; keep offset windows small and product-bounded. - Set explicit aggregation sizes and control bucket growth, especially for user-selected high-cardinality fields. - Treat leading wildcards, regex, fuzzy expansion, scripts, and large highlights as privileged query features with validation, timeouts, and measured limits. See /ops/search for the adjacent decision or procedure that completes this constraint. --- ### [RULE] Keep indexing idempotent and externally versioned (path: /search/queries) Bulk ingestion, queues, and change streams can retry or deliver out of order. If each attempt creates a new document or an older event overwrites a newer one, the index diverges from the authoritative store. - Derive the search document ID from stable domain identity rather than generating a new value at indexing time. - Carry a monotonic source version or update sequence and reject stale writes where the indexing mechanism supports it. - Make delete and tombstone handling retryable so replay cannot recreate a document after the source removed it. - Inspect every bulk-item result and retry only failed items with bounded backoff; an accepted bulk request can contain individual failures. See /search/mappings for the adjacent decision or procedure that completes this constraint. --- ### [MEMORY] Analysis is part of the query contract (path: /search/mappings) Search behavior depends on how text becomes tokens at index time and query time. A synonym or analyzer change applied to only one side produces surprising matches, and existing indexed tokens do not rewrite themselves. - Choose analyzers from the content language and search experience, then inspect generated tokens for representative names, punctuation, identifiers, and mixed-language text. - Keep exact normalizers separate from full-text analyzers so sorting and filters do not inherit tokenization intended for relevance. - Version synonym and analysis configuration with the physical index definition and decide whether the change affects index time, search time, or both. - Evaluate relevance with a maintained query set and judgments before and after analysis changes rather than accepting anecdotal searches. See /search/queries for the rule or workflow that puts this decision into practice. --- ### [MEMORY] Shard count follows recoverability and data scale (path: /ops/search) Shards are independent Lucene indexes with memory, file, merge, and cluster-state overhead. More shards do not automatically mean more performance, especially when every query fans out across many tiny shards. - Estimate retained bytes, ingest rate, node count, recovery window, and representative query fan-out before selecting the initial primary count. - Use rollover or time-based boundaries only when retention and query behavior need them; avoid creating tiny daily indexes by habit. - Monitor shard size, segment count, merge pressure, heap, disk watermarks, and recovery duration as data grows. - Change an unsuitable primary count by creating a new index and migrating through an alias instead of pretending an in-place setting can rewrite physical partitioning. See /search/mappings for the rule or workflow that puts this decision into practice. --- ### [MEMORY] Applications address aliases, not physical index versions (path: /ops/search) Physical index names encode a schema and migration generation. If application configuration points directly at them, every reindex becomes a synchronized deployment and rollback is harder. - Give each logical collection a stable read alias and, where supported by the design, a single explicit write target. - Create new physical indexes with versioned names and complete settings, mappings, and analysis before data movement begins. - Switch aliases atomically after validation so readers never observe a half-migrated index set. - Retain the previous index for a bounded rollback window and prevent accidental writes to it after the alias moves. See /search/queries for the rule or workflow that puts this decision into practice. --- ### [SKILL] reindex-search-without-downtime (path: /) --- name: reindex-search-without-downtime description: Reindex Elasticsearch or OpenSearch after a mapping, analyzer, shard, or document-shape change. --- # Reindex Search Without Downtime Run this procedure when the affected surface changes, before the result is promoted to production. Record evidence for every step instead of accepting a plausible-looking result. 1. Create a versioned target index from committed settings, mappings, analysis, lifecycle, and alias configuration, then validate representative documents. 2. Start a backfill from the authoritative source or prior index with stable IDs and source versions while live changes continue through a replayable path. 3. Compare document counts, missing and extra IDs, field distributions, rejected items, query correctness, relevance judgments, and latency on the target. 4. Pause or fence the final change window, replay outstanding updates, atomically switch aliases, and monitor errors, freshness, and resource use. 5. Rollback the alias if acceptance fails; otherwise reconcile one final time, remove write access from the old index, and delete it only after the retention window. ## Exit criteria The change is complete only when the expected behavior, failure behavior, and rollback path have all been exercised with representative data. Preserve the evidence with the change so the next operator can repeat the same checks.