Note: Qbox Inc. has deprecated the architecture used in this article. Supergiant has replaced this deprecated architecture and isn’t relevant for this article’s specifications.

Load balancing an Elasticsearch cluster is a good practice because a primary goal in ES configuration is a fault-tolerant system that’s resilient to single node failures. All of us here at Qbox know that the load-balancing features of Elasticsearch are a big part of what makes it such a great distributed computing platform. We also know that you’ll get the best performance when you take a bit more time to properly size your Elasticsearch cluster and optimize your computing resources to correspond well with your expected load.

In this article, we bring to mind some considerations that can help you optimize your cluster by improving distribution, performance, and reliability. We’re also aware that various notions about “load balancing” are heard throughout the community, so we also explore various meanings and applications of load balancing with respect to ES clusters.

Keep reading

Note: This blog describes resizing of classic Elasticsearch clusters no longer offered to new Qbox users. The information provided below is deprecated for new Qbox users with Supergiant AWS clusters.

Auto-scaling has been a frequent-request feature since the inception of the Qbox service because auto-scaling with Elasticsearch isn’t as easy as is commonly thought.

Horizontal scaling up is trivial, of course, and is one of the primary benefits of this technology. Automatic scaling down is typically more troublesome and-if not done carefully-rebalancing/reindexing carry an intrinsic computational overhead that dramatically affects performance. Meanwhile, our nodes-by-the-compute-hour model makes vertical scaling a potentially expensive prospect.

Taking all of this into consideration, today we announce a vertical resizing feature addition that is available on your dashboard. Now you can easily resize vertically by migrating to bigger VMs that have more resources. This gives you more flexibility beyond the horizontal scaling that is already available (adding nodes to an existing cluster).

Keep reading

Disk I/O is a major component of any database system, and Elasticsearch is no exception. Elasticsearch, along with most modern databases, makes heavy use of filesystem cache (RAM) to optimize performance. However, when persistence is needed — and it’s needed often — the disk is used. Heavy disk use under a high load can cause a bottleneck with I/O wait, which is a big factor in performance loss.

That being said, it can’t hurt to have blazing fast hard drives as a foundation. With Elasticsearch, faster disk writes = faster indexing, faster disk reads = faster searching.

Keep reading

In the first episode of this tutorial we’re going to explore some of the features of Elasticsearch, and in later episodes we’ll begin digging deep into more sophisticated features using hosted Elasticsearch with Thus begins our multi-part series giving some instruction on Elasticsearch and‘s service benefits for Elasticsearch. I’ll be showing some interesting queries, how to structure your documents for interesting projects, and much more.

For this first episode we will be installing Elasticsearch, creating a cluster, indexing some documents, and changing some settings. In my previous video I discussed a few of the several dozen major advantages Elasticsearch offers. Today we will begin an introduction into setting up a local instance of Elasticsearch. Let’s start by downloading Elasticsearch, using 0.90.9 since at this time it is the current stable release.

Keep reading