I've spent close to a decade running Elasticsearch clusters in production, from 3-node dev setups to multi-hundred-node deployments handling billions of documents a day.

One thing I've seen repeatedly in production is that teams either let shards balloon for years without noticing, or over-correct and split everything into tiny shards "to be safe."

Elastic101 – Best Practice #002 – Shard Size

Smaller shards increase overhead more cluster state to track, more per-shard costs (file handles, memory, translog), and more coordination work on every query.

Larger shards cause the opposite problem: recovery and rebalancing get painfully slow, and a single hot shard can bottleneck an entire index.

>The goal is to keep shard size around 50GB.

Pro tip: you cannot simply change the primary shard count of an existing index. If you need to change it, you typically need to use the Reindex API or the Shrink API, depending on the situation. Also, if you reindex a 200GB index, you should have at least 200GB of additional free disk space available in the cluster for the new index. Choose your primary shard count wisely from the start, fixing it later is expensive.

Previous Elastic101 best practices:

1. Elastic101 Best Practice #001 – HTTP traffic

Try here: Searchali Elasticsearch Monitoring Connect your cluster in 10 seconds. No agent. No data leaves your machine.

I’ve been working with Elasticsearch for almost 10 years, including 6 years of consulting and 4 years of training teams on Elasticsearch and related technologies. Over the years, I’ve seen the same mistakes come up again and again in production clusters, so I thought I’d share some of the best practices I’ve learned along the way.

For the first best practice, a simple misconception that is surprisingly easy to get wrong:

>Don’t send application HTTP traffic directly to master nodes.

This may come from architectures like Kubernetes, where the control plane has a central API endpoint. Elasticsearch is different: master nodes are for cluster management, not application traffic.

>Use dedicated coordinating nodes or data nodes for client traffic.

As a general rule of thumb, for clusters with less than 20 nodes, adding more data nodes is often more efficient than adding dedicated coordinating-only nodes. Of course, this depends on your workload, especially the size and complexity of aggregations and heavy queries.

Master ≠ API Server.

Want to see this traffic flow in real time on your own cluster?

You can try the tool I built it takes less than 10 seconds to connect and see which clients are hitting which Elasticsearch nodes.

Try here: No agent or server-side installation required; the connection is made directly from your browser. Searchali Elasticsearch Monitoring

If you’d like me to continue this series, an upvote would be appreciated. 🙂