Benchmarking fun with JRuby 1.1 RC2, glassfish, and Rails 2.0.2

Yesterday JRuby 1.1 RC2 was released and two days ago the glassfish gem v 0.1.1 was released. Lots of interesting stuff happening in JRuby land!

I decided to take JRuby and the glassfish gem for a spin with a simple Rails application.
Installing JRuby

First step was to download and install JRuby. This is pretty straightforward:
cd /tmp
wget http://dist.codehaus.org/jruby/jruby-src-1.1RC2.tar.gz
tar xvzf jruby-src-1.1RC2.tar.gz
cd jruby-1.1RC2/
ant
export JRUBY_HOME=`pwd`
export PATH=$JRUBY_HOME/bin:$PATH
jruby --version
ruby 1.8.6 (2008-02-17 rev 5944) [i386-jruby1.1RC2]

Yep, seems to work.

Installing gems

Next step was to install the Rails and glassfish gems:
unset GEM_HOME
unset GEM_PATH
gem install rails
gem install glassfish

Creating a Rails application

On to the Rails application… I used scaffold to have a simple application up and running quickly:
cd ..
rails glassfishtest --database=mysql
cd glassfishtest/
export RAILS_ENV=production
rake db:sessions:create
script/generate scaffold Book title:string
rake db:create
rake db:migrate
script/runner "Book.create(:title => 'JRuby Rocks')"

I use the database session store, so I added this line to the config/environment.rb file
config.action_controller.session_store = :active_record_store

Firing up glassfish

Let’s fire up the glassfish server:
cd ..
glassfish_rails glassfishtest -n 2

The -n 2 option will make glassfish start 2 Rails instances.

Benchmark fun!

Glassfish
I used the ab command to perform some simple benchmarks.
Each ab command was run twice with a freshly started glassfish server. The first run warms up the JIT in the JVM. The results listed below are for the second run (and the fifth run for some). All benchmarks were performed on my 2.33GHz MacBook Pro running Leopard 10.5.2 with Java version 1.5.0_13-b05-237.

The performance with respect to static files is impressive:
ab -n 5000 -c 10 http://localhost:3000/
Requests per second: 2705.63 [#/sec] (mean)

Now onto a page created by Rails:
ab -n 1000 -c 8 http://localhost:3000/books/1
Requests per second: 54.10 [#/sec] (mean)

JRuby can be tweaked a little bit with the -server parameter:
JAVA_OPTS="-server" glassfish_rails glassfishtest -n 2
ab -n 1000 -c 8 http://localhost:3000/books/1
Requests per second: 53.82 [#/sec] (mean) 2nd run
Requests per second: 63.06 [#/sec] (mean) 5th run

After a little warmup the performance is approximately 20% better than without the -server option.

Let’s try adding more Rails instances:
JAVA_OPTS="-server" glassfish_rails glassfishtest -n 4
Requests per second: 50.71 [#/sec] (mean) 2nd run
Requests per second: 60.69 [#/sec] (mean) 5th run

On my dual core machine this actually degrades performance a little bit. I guess it is a good idea to have the number of Rails instances match the number of cores in your server.

But what about one Rails instance:
JAVA_OPTS="-server" glassfish_rails glassfishtest -n 1
Requests per second: 31.56 [#/sec] (mean) 2nd run
Requests per second: 34.48 [#/sec] (mean) 5th run

That hurts!

Mongrel

How does mongrel compare to glassfish?
Single Mongrel – JRuby
JAVA_OPTS='-server' jruby script/server -e production
Requests per second: 54.99 [#/sec] (mean) 2nd run
Requests per second: 63.20 [#/sec] (mean) 5th run

Two Mongrels behind pen – JRuby
Requests per second: 58.39 [#/sec] 2nd run(mean)
Requests per second: 69.16 [#/sec] (mean) 10th run

Static files:
Requests per second: 313.57 [#/sec] (mean)

Mongrel and the glassfish server have comparable performance with respect to Rails generated pages.
With respect to serving static files, glassfish outperforms Mongrel significantly. That said, you shouldn’t really let Mongrel serve static content – it is better to leave that to nginx or Apache.

Mongrel – MRI

What is the performance when using MRI?
Single Mongrel – MRI
Requests per second: 120.79 [#/sec] (mean)

Two Mongrels behind pen – MRI
Requests per second: 123.42 [#/sec] (mean)

The MRI Mongrel seems to have a lot better performance for this (admittedly simple) benchmark.

Conclusion

With respect to ease of running a server the JRuby/glassfish combo is very appealing:

  • static files are served very fast
  • no need for a separate load balancer
  • the whole thing is started with just one command

For this particular Rails application benchmark, the performance of the JRuby stack is only half of the performance of MRI, which is kind of sad. I am pretty sure that this is not the case for all Rails applications. In fact, evidence from Mingle seems to indicate that JRuby is faster than MRI. So I guess the best thing is to try it out on your own Rails app – and please blog about your findings. If you decide to benchmark your own Rails app I highly recommend this peepcode screencast about benchmarking.

About these ads

6 thoughts on “Benchmarking fun with JRuby 1.1 RC2, glassfish, and Rails 2.0.2

  1. Did you watch the memory footprint of each deployment? How did they compare? These days with cheap VPS hosting starting at 256MB, I’m more interested in being able to run in 256MB than I am in subtle differences in CPU cycles. :-)

  2. Pingback: Interesting Ruby Tidbits That Don’t Need Separate Posts #17

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s