Enjoying Rails

October 16, 2008

Automatic Rails on Ubuntu 8.04 LTS

Filed under: rails — enjoyingrails @ 22:32
Tags: , , ,

A couple of weeks ago there was a post on the FiveRuns blog about automatically installing the Rails stack on an Ubuntu 8.04 VPS.

I prefer to use Passenger and Ruby Enterprise Edition when running my Rails app, so inspired by the FiveRuns script I wrote my own version – here is the gist on github.


#!/bin/bash
# Inspired by http://blog.fiveruns.com/2008/9/24/rails-automation-at-slicehost

apt-get update
apt-get upgrade -y
apt-get -y install build-essential libssl-dev libreadline5-dev zlib1g-dev
apt-get -y install mysql-server libmysqlclient15-dev mysql-client
apt-get -y install ruby ruby1.8-dev irb ri rdoc libopenssl-ruby1.8

RUBYGEMS="rubygems-1.3.0"
wget http://rubyforge.org/frs/download.php/43985/$RUBYGEMS.tgz
tar xzf $RUBYGEMS.tgz
cd $RUBYGEMS
ruby setup.rb
cd ..

# Install Ruby Enterprise Edition
wget http://rubyforge.org/frs/download.php/41040/ruby-enterprise-1.8.6-20080810.tar.gz
tar xvzf ruby-enterprise-1.8.6-20080810.tar.gz
yes '' | ./ruby-enterprise-1.8.6-20080810/installer

# Install Passenger
/usr/bin/gem1.8 install -v=2.0.3 passenger --no-rdoc --no-ri
apt-get -y install apache2-mpm-prefork apache2-prefork-dev
yes '' | passenger-install-apache2-module

# Create sample Rails app
/usr/bin/gem1.8 install rails --no-rdoc --no-ri
cd /var/www
rails -d mysql hello
cd hello
./script/generate controller welcome hello
echo "Hello World" > app/views/welcome/hello.html.erb
rake db:create RAILS_ENV=production

# Create the Apache2 Passenger module files
cat >> /etc/apache2/mods-available/passenger.load <> /etc/apache2/mods-available/passenger.conf <<-EOF

PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.3
PassengerRuby /opt/ruby-enterprise-1.8.6-20080810/bin/ruby

EOF
a2enmod passenger

# Create a site file for the sample Rails app
IP_ADDRESS=`ifconfig eth0 | sed -n 's/.*dr:\(.*\) Bc.*/\1/p'`
cat >> /etc/apache2/sites-available/hello <<-EOF

ServerName www.yourhost.com
DocumentRoot /var/www/hello/public

EOF
a2ensite hello

# That's it!
reboot

The script assumes that you have ssh access as root to a clean Ubuntu 8.04 install.

The script will install

  • Ruby 1.8.6
  • RubyGems 1.3.0
  • Passenger 2.0.3
  • Ruby Enterprise Edition 20080810
  • Apache 2.2.8
  • MySQL 5.0.51a
  • A sample Rails app

Note that the Passenger installer will install the latest Rails (2.1.1) and a bunch of other useful gems.

Assuming that your server IP address is 192.168.185.128 you can run it like this:

ssh root@192.168.185.128 "wget -O - http://gist.github.com/raw/16225/a6a16b3a38cd3486679b96fa0f3446e58f3b8423 | sed -e s/$'\r'//g > install.sh; /bin/bash install.sh; rm install.sh"

Sit back and enjoy – in less than ten minutes you will have the full Rails stack and a sample Rails app running. Take a look at it on http://192.168.185.128/welcome/hello

February 17, 2008

Benchmarking fun with JRuby 1.1 RC2, glassfish, and Rails 2.0.2

Filed under: rails — enjoyingrails @ 16:19
Tags:

Yesterday JRuby 1.1 RC2 was released and two days ago the glassfish gem v 0.1.1 was released. Lots of interesting stuff happening in JRuby land!

I decided to take JRuby and the glassfish gem for a spin with a simple Rails application.
Installing JRuby

First step was to download and install JRuby. This is pretty straightforward:
cd /tmp
wget http://dist.codehaus.org/jruby/jruby-src-1.1RC2.tar.gz
tar xvzf jruby-src-1.1RC2.tar.gz
cd jruby-1.1RC2/
ant
export JRUBY_HOME=`pwd`
export PATH=$JRUBY_HOME/bin:$PATH
jruby --version
ruby 1.8.6 (2008-02-17 rev 5944) [i386-jruby1.1RC2]

Yep, seems to work.

Installing gems

Next step was to install the Rails and glassfish gems:
unset GEM_HOME
unset GEM_PATH
gem install rails
gem install glassfish

Creating a Rails application

On to the Rails application… I used scaffold to have a simple application up and running quickly:
cd ..
rails glassfishtest --database=mysql
cd glassfishtest/
export RAILS_ENV=production
rake db:sessions:create
script/generate scaffold Book title:string
rake db:create
rake db:migrate
script/runner "Book.create(:title => 'JRuby Rocks')"

I use the database session store, so I added this line to the config/environment.rb file
config.action_controller.session_store = :active_record_store

Firing up glassfish

Let’s fire up the glassfish server:
cd ..
glassfish_rails glassfishtest -n 2

The -n 2 option will make glassfish start 2 Rails instances.

Benchmark fun!

Glassfish
I used the ab command to perform some simple benchmarks.
Each ab command was run twice with a freshly started glassfish server. The first run warms up the JIT in the JVM. The results listed below are for the second run (and the fifth run for some). All benchmarks were performed on my 2.33GHz MacBook Pro running Leopard 10.5.2 with Java version 1.5.0_13-b05-237.

The performance with respect to static files is impressive:
ab -n 5000 -c 10 http://localhost:3000/
Requests per second: 2705.63 [#/sec] (mean)

Now onto a page created by Rails:
ab -n 1000 -c 8 http://localhost:3000/books/1
Requests per second: 54.10 [#/sec] (mean)

JRuby can be tweaked a little bit with the -server parameter:
JAVA_OPTS="-server" glassfish_rails glassfishtest -n 2
ab -n 1000 -c 8 http://localhost:3000/books/1
Requests per second: 53.82 [#/sec] (mean) 2nd run
Requests per second: 63.06 [#/sec] (mean) 5th run

After a little warmup the performance is approximately 20% better than without the -server option.

Let’s try adding more Rails instances:
JAVA_OPTS="-server" glassfish_rails glassfishtest -n 4
Requests per second: 50.71 [#/sec] (mean) 2nd run
Requests per second: 60.69 [#/sec] (mean) 5th run

On my dual core machine this actually degrades performance a little bit. I guess it is a good idea to have the number of Rails instances match the number of cores in your server.

But what about one Rails instance:
JAVA_OPTS="-server" glassfish_rails glassfishtest -n 1
Requests per second: 31.56 [#/sec] (mean) 2nd run
Requests per second: 34.48 [#/sec] (mean) 5th run

That hurts!

Mongrel

How does mongrel compare to glassfish?
Single Mongrel – JRuby
JAVA_OPTS='-server' jruby script/server -e production
Requests per second: 54.99 [#/sec] (mean) 2nd run
Requests per second: 63.20 [#/sec] (mean) 5th run

Two Mongrels behind pen – JRuby
Requests per second: 58.39 [#/sec] 2nd run(mean)
Requests per second: 69.16 [#/sec] (mean) 10th run

Static files:
Requests per second: 313.57 [#/sec] (mean)

Mongrel and the glassfish server have comparable performance with respect to Rails generated pages.
With respect to serving static files, glassfish outperforms Mongrel significantly. That said, you shouldn’t really let Mongrel serve static content – it is better to leave that to nginx or Apache.

Mongrel – MRI

What is the performance when using MRI?
Single Mongrel – MRI
Requests per second: 120.79 [#/sec] (mean)

Two Mongrels behind pen – MRI
Requests per second: 123.42 [#/sec] (mean)

The MRI Mongrel seems to have a lot better performance for this (admittedly simple) benchmark.

Conclusion

With respect to ease of running a server the JRuby/glassfish combo is very appealing:

  • static files are served very fast
  • no need for a separate load balancer
  • the whole thing is started with just one command

For this particular Rails application benchmark, the performance of the JRuby stack is only half of the performance of MRI, which is kind of sad. I am pretty sure that this is not the case for all Rails applications. In fact, evidence from Mingle seems to indicate that JRuby is faster than MRI. So I guess the best thing is to try it out on your own Rails app – and please blog about your findings. If you decide to benchmark your own Rails app I highly recommend this peepcode screencast about benchmarking.

December 28, 2007

My first Rails Contribution

Filed under: rails — enjoyingrails @ 21:00
Tags:

Yeah! I am a Rails contributor!

In an application at work we are using a Rails REST application for the backend of the application and another Rails application as the frontend. The frontend application does not use the database at all but only the REST api provided by the backend application.

When sending lots of data between the two applications serializing to and from XML turned out to be a performance bottleneck. We turned to JSON and this improved performance significantly.

The JSON support in ActiveResource was added recently and there are still some areas where XML is better supported than JSON. So I submitted a patch to improve the JSON support. The patch got submitted to trunk in this changeset.

It feels really good to contribute back to Rails when Rails have brought me so many hours of joy :-)

October 27, 2007

My Rails presentations

Filed under: rails — enjoyingrails @ 19:59

A list of the presentations I have given in aarhus.rb (my local Ruby Brigade which I co-founded).

tv2.dk traffic costs

Filed under: rails,scaling — enjoyingrails @ 15:50
Tags: ,

In an earlier posting we saw that tv2.dk in August 2007 served 142.132.680 pages and that the average page size was 395K and the average number of requests per page was 54.

How much traffic is this?

Doing some math it turns out that approximately 54 TB traffic and 7.7 billion requests during August! That’s a lot of traffic!

Jay.net in their Grand National package say that 33000 GB traffic/month will set you back 10000 kr. which is approximately 1900$. Note that it is 33000 GB Danish traffic and that you have to pay for international traffic as well. According to Alexa 87% of the tv2.dk’s traffic comes from Denmark. My guess is that tv2.dk’s 54 TB traffic in the jay.net setup will cost them around 4000$.

What if the site was hosted on Amazon EC2/S3?

One option is to let Amazon S3 serve all static content. Using the AWS Simple Monthly Calculator the traffic is going to cost 8,699.44$ for the traffic itself and a request fee of 7,675.17$ totalling 16,374.61$! Letting the content from S3 removes the need for a dedicated server to serve static content but the request fee makes it rather costly.

Another option is to have a dedicated EC2 server for serving static content. The traffic is going to cost 8,699.44$ and the server itself 74.40$.

So jay.net turns out to be much cheaper than the Amazon offer for a high-traffic site like tv2.dk.

October 24, 2007

tv2.dk on Rails on Amazon EC2

Filed under: rails,scaling — enjoyingrails @ 20:53
Tags: , ,

What if tv2.dk (one of the most popular Danish sites) was running Rails and ran on Amazon EC2? This posting is a followup to this posting

Let’s assume that we on average during 24 hours have 50 requests/sec. The traffic during the 24 hours could for example be distributed like this

Hours Traffic
00 – 08 5 requests/sec
08 – 11 30 requests/sec
11 – 12 150 requests/sec
12 – 14 250 requests/sec
14 – 15 150 requests/sec
15 – 24 30 requests/sec

This works out to be 50 requests/sec on average. As we saw in the tv2.dk on Rails blog posting each quad core server will get you 80 requests/sec. Unfortunately we have to have servers that have enough power to satisfy the peak number of requests and not the average number of requests. In this example we need four quad core servers in order to be able to serve the 250 requests/sec needed from 12-14 (actually they are able to serve 320 requests/sec). So on average we need one server but in the peak hours we need four servers.

What if we were able to adjust the number of servers dynamically to satisfy the current load on the website?

Amazon EC2 to the rescue! Amazon EC2 is a service that allows you to run one or many virtual servers and you pay by the hour for each virtual server. So we can start and stop a number of virtual machines depending on the load on the website. Each EC2 instance has one (virtual) cpu core so with 10 mongrels doing 2 requests/sec it should be able to handle 20 requests/sec.
So how many servers do we need to run?

Hours Traffic # of servers
00 – 08 5 requests/sec 1
08 – 11 30 requests/sec 2
11 – 12 150 requests/sec 8
12 – 14 250 requests/sec 13
14 – 15 150 requests/sec 8
15 – 24 30 requests/sec 2

If we calculate the number of “server hours” it is 74. How much is this going to cost? Using the AWS Simple Monthly Calculator it turns out to cost 222$ for a 30 day month. Not too bad! Of course we have to add a server for static content (or use Amazon S3) as well as a couple of DB servers. With a small instance server for static content and two large instances (virtual quad cores with 7.5 GB ram) the monthly bill goes up to 811$ – still not too bad.

But you also have to pay for traffic. I will cover that in another blog posting.

October 22, 2007

tv2.dk on Rails

Filed under: rails,scaling — enjoyingrails @ 22:02
Tags: , ,

Recently I started wondering: What kind of server setup would it take to run one of the most popular sites in Denmark if it was running Rails?

According to FDIM (the association of Danish Internet media) tv2.dk was the thirdmost visited site in Denmark during August 2007:

Number of users 1.435.208
Number of visits 15.233.358
Number of page views 142.132.680

142 million page views! That’s a lot! But how much data is this?

According to Alexa the three most popular subdomains of tv2.dk are galleri.tv2.dk, nyhederne.tv2.dk, and vip.tv2.dk.
Using YSlow we get a picture of the total page size and the number of http requests for the main page of each subdomain:

galleri.tv2.dk Total size 140.1K HTTP requests 19
nyhederne.tv2.dk Total size 580.2K HTTP requests 85
vip.tv2.dk Total size 464.4K HTTP requests 58

The average page size is 395K and the average number of requests per page is 54.

The number of page views for August is 142.132.680. This translates to an average of 53 page views/sec. Given the average page size and number of requests this translates to an average bandwith usage of 20.5 MB/sec and an average of 2866 request/sec.

What kind of server setup do you need to handle this amount of traffic?

First of all, by far most traffic comes from static content (images, javascript files, stylesheets). Any decent web server (Apache, nginx and in tv2.dk’s case lighttpd) – this page shows that without too much hassle it is possible to achieve 10000+ requests/sec on a decent server with enough network bandwidth.

But what about the Rails part? If we assume that each page view corresponds to one Rails request we have to handle 53 Rails requests/sec. If we assume that the average request time is 500 ms, each Mongrel can handle 2 requests/sec. So we have to have 27 Mongrels to be able to handle 53 queries/sec. As noted in the excellent article Scale rails from one box to three, four and five by Courtenay a rule of thumb is that each cpu core can handle 10 Mongrels. So with a quad core machine we should be able to handle 2*10*4 = 80 Rails requests/sec. Each Mongrel uses in the range of 60-100 MB, so the machine has to have 4*10*80 = 3200 MB ram. Rails applications can be built to scale quite well, so adding additional quad core machines with 4 gigs of ram should get you another 80 requests/sec. With 4 of these machines you should be able to handle 320 requests/sec.

There must some database servers to handle all the data. There a several ways to setup these servers. Take a look at Courtenay’s article for an explanation of these.

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.