Monthly Archives: January 2014

Controlling Elasticsearch memory usage on os x

Posted on by

By default, Elasticsearch will use between 256MB and 1GB of memory. On my dev environment, memory is a premium and I don’t really have a lot of data to index and play with so I’d prefer this to be closer to 256m

To change this, navigate to
/usr/local/Cellar/elasticsearch/x.xx.x/bin

(where x.xx.x is the Elasticsarch version number)
open up elasticsearch.in.sh

then replace the memory size in the following lines

if [ "x$ES_MIN_MEM" = "x" ]; then
    ES_MIN_MEM=256m
fi
if [ "x$ES_MAX_MEM" = "x" ]; then
    ES_MAX_MEM=1g
fi

It is recommended that you set both ES_MIN_MEM and ES_MAX_MEM to the max size you want so that Elasticsearch doesn’t have to pause to resize memory, which can affect performance.

from the comments in that file …

min and max heap sizes should be set to the same value to avoid
stop-the-world GC pauses during resize, and so that we can lock the
heap in memory on startup to prevent any of it from being swapped
out.

I upgraded to Ruby 2.1.0 and all I got was this lousy T-shirt …

Posted on by

Ruby 2.1.0 was released about 2 weeks ago, and after seeing a notable speed performance improvement with my Rails app when going from 1.9.3 to 2.0.0 I was excited to see if the same would happen with Ruby 2.1.0.

I quickly ran another set of very casual tests, mainly running test suites and noting startup times of rails commands and rake tasks at the command line. What I saw was a 20% improvement in those tasks. Nothing to sneeze at …

Excited, I rushed to upgrade my tiny Rails app to ruby 2.1.0 and after doing it I tracked the difference in New Relic

This was my result (deployment happened about 18:00 on December 25th, app is setup to run with one puma worker and 8-16 threads proxied to nginx)

New Relic graph after upgrading to Ruby 2.1.0 (18:00 on the graph)As you can see, there wasn’t much of an improvement in performance at all. In fact here is the graph after a full day of running in the wild.

New Relic graph of my app after a full day of running with Ruby 2.1.0

 

So while you might get better startup times on Rails tasks, you probably won’t see too much of a speed boost on your servers.

Improved GC

What I found interesting, though, is the improved garbage collection. The brown part of the graph (GC execution) almost completely disappears in ruby 2.1.0. And its no wonder, a lot of work went into improving Garbage collection in ruby 2.1.0

The garbage collector in Ruby 2.1 implements a form of generational garbage collection, with Ruby calling their implementation RGenGC (Restricted Generational Garbage Collection).  This replaces the “Mark & Sweep” implementation used in previous versions of Ruby
…
Using RGenGC provides high compatibility with existing extensions while still bringing performance improvements.  Popular objects Array, String, Hash, Object, and Numeric are Write-Barrier protected, thus able to take advantage of the RGenGC system

The moral of the story? Ruby 2.1.0 has a small performance boost over Ruby 2.0 and upgrading to it should give you a bit of a boost with your Rails app, more if its a pretty big app with non-trivial time spent doing GC.

PS: Brian Hempel has created an awesome site that benchmarks ruby versions using Rails. His results are in line with what I found
PS2: Sorry there wasn’t actually a T-shirt, I just liked the title