Elasticsearch — Practical

Elasticsearch — Practical patterns

Index template (with ILM)

PUT _index_template/logs-template
{
  "index_patterns": ["logs-*"],
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "index.lifecycle.name": "logs-policy"
    },
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "@timestamp": { "type": "date" },
        "level":      { "type": "keyword" },
        "message":    { "type": "text" },
        "host":       { "type": "keyword" },
        "trace_id":   { "type": "keyword" }
      }
    }
  }
}

ILM policy (hot → warm → delete)

PUT _ilm/policy/logs-policy
{
  "policy": {
    "phases": {
      "hot":  { "actions": { "rollover": { "max_size": "30gb", "max_age": "1d" } } },
      "warm": { "min_age": "7d",  "actions": { "shrink": { "number_of_shards": 1 }, "forcemerge": { "max_num_segments": 1 } } },
      "cold": { "min_age": "30d", "actions": { "freeze": {} } },
      "delete": { "min_age": "90d", "actions": { "delete": {} } }
    }
  }
}

Search with filters + scoring

POST products/_search
{
  "size": 20,
  "query": {
    "bool": {
      "must":  [ { "multi_match": { "query": "rental dubai", "fields": ["title^3", "description"] } } ],
      "filter": [
        { "term":  { "status": "active" } },
        { "range": { "price": { "lte": 5000 } } },
        { "geo_distance": { "distance": "10km", "location": { "lat": 25.2, "lon": 55.27 } } }
      ]
    }
  },
  "sort": [ "_score", { "createdAt": "desc" } ]
}

Aggregation: facets + metrics

POST products/_search
{
  "size": 0,
  "aggs": {
    "by_category": {
      "terms": { "field": "category", "size": 20 },
      "aggs": {
        "avg_price": { "avg": { "field": "price" } },
        "p95_price": { "percentiles": { "field": "price", "percents": [95] } }
      }
    }
  }
}

Autocomplete (edge_ngram)

PUT autocomplete
{
  "settings": {
    "analysis": {
      "analyzer": {
        "edge_ngram_analyzer": { "tokenizer": "edge_ngram_tokenizer", "filter": ["lowercase"] }
      },
      "tokenizer": {
        "edge_ngram_tokenizer": { "type": "edge_ngram", "min_gram": 2, "max_gram": 15, "token_chars": ["letter","digit"] }
      }
    }
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "analyzer": "edge_ngram_analyzer",
        "search_analyzer": "standard"
      }
    }
  }
}

Bulk indexing

POST _bulk
{ "index": { "_index": "users", "_id": "1" } }
{ "name": "Alice" }
{ "index": { "_index": "users", "_id": "2" } }
{ "name": "Bob" }

Use the _bulk endpoint, batch 5-15MB, multiple parallel clients for max throughput. Disable refresh during bulk: index.refresh_interval = -1, then re-enable.

Reindex with alias swap (zero downtime)

# Step 1: write to alias
PUT products-v1
PUT products-v1/_alias/products

# Step 2: change mapping requires new index
PUT products-v2 { ... new mapping ... }
POST _reindex { "source": { "index": "products-v1" }, "dest": { "index": "products-v2" } }

# Step 3: atomic alias swap
POST _aliases {
  "actions": [
    { "remove": { "index": "products-v1", "alias": "products" } },
    { "add":    { "index": "products-v2", "alias": "products" } }
  ]
}

Optimistic concurrency

GET products/_doc/123          # returns _seq_no, _primary_term
PUT products/_doc/123?if_seq_no=42&if_primary_term=1 { ... }
# 409 conflict if changed

Highlight

{ "query": { "match": { "body": "rental" } },
  "highlight": { "fields": { "body": {} } } }

Vector / kNN

PUT items
{ "mappings": { "properties": {
  "embedding": { "type":"dense_vector", "dims": 768, "index": true, "similarity": "cosine" }
}}}

POST items/_search
{ "knn": { "field": "embedding", "query_vector": [0.1, ...], "k": 10, "num_candidates": 100 },
  "size": 10 }

Cluster diagnostics

GET _cluster/health                        # overall
GET _cluster/health/<index>?level=shards
GET _cat/indices?v&s=store.size:desc
GET _cat/shards?v&h=index,shard,prirep,state,docs,store,node
GET _cat/nodes?v&h=name,heap.percent,ram.percent,cpu,load_1m,role
GET _nodes/stats/jvm
GET _cat/recovery?active_only=true
GET _cluster/allocation/explain
GET _tasks?actions=*search*&detailed=true

Settings to know

indices.query.bool.max_clause_count   # default 4096; raise carefully
search.max_buckets                    # cap big aggs
indices.fielddata.cache.size          # 30% heap default

Anti-patterns to avoid

• Sorting/aggregating on text (use .keyword).
• Querying with leading * wildcard.
• Hot index with too many shards.
• Letting dynamic mapping run on user input.
• Searching across hundreds of indices without aliases / data streams.
• Using ES as primary DB without backup pipeline.