Elasticsearch and OpenSearch

Pathrule3 Rules • 3 Memories • 1 Skill

Search clusters become difficult to repair when dynamic mapping chooses the wrong type, analyzed and exact fields are confused, nested arrays lose relationship meaning, shard counts outgrow the data, or application code targets physical index names that cannot migrate. This pattern constrains explicit mappings, bounded queries, and safe writes; it records analysis, shard, and alias decisions and provides a zero-downtime reindex procedure. It differs from relational indexing and database-schema patterns because inverted indexes, analyzers, relevance scoring, nested documents, segment merges, and alias-based index replacement define the operational model.

Suggested path map

Pathrule places each piece on the matching path, so your assistant only sees it where it belongs. This is the scoping you get on import; you can adjust it in your workspace.

/ workspace root

reindex-search-without-downtime

search/

mappings/

Define mappings before indexing production data

Analysis is part of the query contract

queries/

Bound search work and result windows

Keep indexing idempotent and externally versioned

ops/

search/

Shard count follows recoverability and data scale

Applications address aliases, not physical index versions

Rules

Define mappings before indexing production data/search/mappingshighstrictDeclare field type, exact and analyzed variants, date formats, nested relationships, and dynamic behavior from the search contract.

1	The first document can choose a dynamic field type that later documents cannot use, and many mapping changes cannot be applied in place. Explicit mapping turns ingestion assumptions into reviewable schema.
2
3	- Map identifiers, enums, and filter or aggregation fields as exact values; map human language as analyzed text with an intentional analyzer.
4	- Use multi-fields when the same source value needs full-text relevance and exact sorting or aggregation, and name their roles consistently.
5	- Map arrays of objects as nested when queries must preserve relationships within each object; ordinary object arrays flatten values and can create cross-object matches.
6	- Reject or tightly control unexpected dynamic fields for untrusted or high-cardinality input so one malformed document cannot expand cluster mappings indefinitely.
7
8	See /search/queries for the adjacent decision or procedure that completes this constraint.

Bound search work and result windows/search/querieshighstrictLimit clauses, fields, buckets, scripts, highlights, and pagination depth according to the product query contract.

1	A search request can be syntactically valid while consuming unbounded heap or CPU through deep pagination, broad wildcard expansion, large aggregations, or scripted work over many documents.
2
3	- Search only fields relevant to the feature and apply filters in filter context when scoring is unnecessary.
4	- Use cursor-style search-after pagination with a deterministic sort for deep traversal; keep offset windows small and product-bounded.
5	- Set explicit aggregation sizes and control bucket growth, especially for user-selected high-cardinality fields.
6	- Treat leading wildcards, regex, fuzzy expansion, scripts, and large highlights as privileged query features with validation, timeouts, and measured limits.
7
8	See /ops/search for the adjacent decision or procedure that completes this constraint.

Keep indexing idempotent and externally versioned/search/querieshighstrictUse stable document identity and a source-of-truth version so retries and out-of-order events cannot resurrect stale search state.

1	Bulk ingestion, queues, and change streams can retry or deliver out of order. If each attempt creates a new document or an older event overwrites a newer one, the index diverges from the authoritative store.
2
3	- Derive the search document ID from stable domain identity rather than generating a new value at indexing time.
4	- Carry a monotonic source version or update sequence and reject stale writes where the indexing mechanism supports it.
5	- Make delete and tombstone handling retryable so replay cannot recreate a document after the source removed it.
6	- Inspect every bulk-item result and retry only failed items with bounded backoff; an accepted bulk request can contain individual failures.
7
8	See /search/mappings for the adjacent decision or procedure that completes this constraint.

Memories

Analysis is part of the query contract/search/mappingsVersion analyzers, normalizers, synonyms, and language behavior with the index because changing tokens usually requires reindexing.

1	Search behavior depends on how text becomes tokens at index time and query time. A synonym or analyzer change applied to only one side produces surprising matches, and existing indexed tokens do not rewrite themselves.
2
3	- Choose analyzers from the content language and search experience, then inspect generated tokens for representative names, punctuation, identifiers, and mixed-language text.
4	- Keep exact normalizers separate from full-text analyzers so sorting and filters do not inherit tokenization intended for relevance.
5	- Version synonym and analysis configuration with the physical index definition and decide whether the change affects index time, search time, or both.
6	- Evaluate relevance with a maintained query set and judgments before and after analysis changes rather than accepting anecdotal searches.
7
8	See /search/queries for the rule or workflow that puts this decision into practice.

Shard count follows recoverability and data scale/ops/searchChoose primary shards from expected index size, write throughput, query parallelism, and recovery behavior, then revise through a new index.

1	Shards are independent Lucene indexes with memory, file, merge, and cluster-state overhead. More shards do not automatically mean more performance, especially when every query fans out across many tiny shards.
2
3	- Estimate retained bytes, ingest rate, node count, recovery window, and representative query fan-out before selecting the initial primary count.
4	- Use rollover or time-based boundaries only when retention and query behavior need them; avoid creating tiny daily indexes by habit.
5	- Monitor shard size, segment count, merge pressure, heap, disk watermarks, and recovery duration as data grows.
6	- Change an unsuitable primary count by creating a new index and migrating through an alias instead of pretending an in-place setting can rewrite physical partitioning.
7
8	See /search/mappings for the rule or workflow that puts this decision into practice.

Applications address aliases, not physical index versions/ops/searchUse read and write aliases as the stable application contract so mappings and data can move behind them.

1	Physical index names encode a schema and migration generation. If application configuration points directly at them, every reindex becomes a synchronized deployment and rollback is harder.
2
3	- Give each logical collection a stable read alias and, where supported by the design, a single explicit write target.
4	- Create new physical indexes with versioned names and complete settings, mappings, and analysis before data movement begins.
5	- Switch aliases atomically after validation so readers never observe a half-migrated index set.
6	- Retain the previous index for a bounded rollback window and prevent accidental writes to it after the alias moves.
7
8	See /search/queries for the rule or workflow that puts this decision into practice.

Skills

reindex-search-without-downtime/rootCreate, backfill, verify, cut over, reconcile, and retire a search index through stable aliases.

1	---
2	name: reindex-search-without-downtime
3	description: Reindex Elasticsearch or OpenSearch after a mapping, analyzer, shard, or document-shape change.
4	---
5
6	# Reindex Search Without Downtime
7
8	Run this procedure when the affected surface changes, before the result is promoted to production. Record evidence for every step instead of accepting a plausible-looking result.
9
10	1. Create a versioned target index from committed settings, mappings, analysis, lifecycle, and alias configuration, then validate representative documents.
11	2. Start a backfill from the authoritative source or prior index with stable IDs and source versions while live changes continue through a replayable path.
12	3. Compare document counts, missing and extra IDs, field distributions, rejected items, query correctness, relevance judgments, and latency on the target.
13	4. Pause or fence the final change window, replay outstanding updates, atomically switch aliases, and monitor errors, freshness, and resource use.
14	5. Rollback the alias if acceptance fails; otherwise reconcile one final time, remove write access from the old index, and delete it only after the retention window.
15
16	## Exit criteria
17
18	The change is complete only when the expected behavior, failure behavior, and rollback path have all been exercised with representative data. Preserve the evidence with the change so the next operator can repeat the same checks.

Why this pattern

AI agents often rely on dynamic mappings, use analyzed text for exact filters, query nested objects as flat fields, or update an index mapping that actually requires a new index and reindex.

Built for Backend and search teams operating Elasticsearch or OpenSearch for product search, logs, or discovery.

Keeps your assistant from:

Locking an incorrect inferred field type into a production index
Returning false matches across different objects in an array
Creating excessive shards and cluster metadata overhead
Coupling applications to physical index versions during migration

View as markdown

License: Apache-2.0
Version: 1.0.0
Updated: 2026-08-25