Enjoying Rails

November 6, 2007

Experimenting with Amazon S3 EU edition

Filed under: Uncategorized — enjoyingrails @ 23:42
Tags:

Today Amazon announced the availability of S3 in Europe.

Nice! Let’s play with it! Please notice that I am located in Denmark and that all tests were performed on my 2048/512 ADSL line.

Download the new version of Amazon S3 Authentication Tool for Curl

Unzip it and create an .s3curl file containing you AWS keys as described in the readme file.

Now let’s create some buckets – a US bucket and an EU bucket:

s3curl.pl --id personal --createBucket -- http://s3.amazonaws.com/erichsen.net.us
s3curl.pl --id personal --createBucket=EU -- http://s3.amazonaws.com/erichsen.net.eu

Fetch some test files a 50K file and a 10MB one:

wget ftp://ftptest1.tele.dk/pub/50Ktest.rnd
wget ftp://ftptest1.tele.dk/pub/10Mtestb.rnd

And upload them:

s3curl.pl --id=personal --acl public-read --put 10Mtestb.rnd -- http://erichsen.net.us.s3.amazonaws.com/10Mtestb.rnd
s3curl.pl --id=personal --acl public-read --put 50Ktest.rnd -- http://erichsen.net.us.s3.amazonaws.com/50Ktest.rnd
s3curl.pl --id=personal --acl public-read --put 10Mtestb.rnd -- http://erichsen.net.eu.s3.amazonaws.com/10Mtestb.rnd
s3curl.pl --id=personal --acl public-read --put 50Ktest.rnd -- http://erichsen.net.eu.s3.amazonaws.com/50Ktest.rnd

Try fetching the large file from the US bucket a couple of times

ab -n 1 http://erichsen.net.us.s3.amazonaws.com/10Mtestb.rnd
...
Time taken for tests: 50.325 seconds
...
Transfer rate: 208.37 [Kbytes/sec] received

ab -n 1 http://erichsen.net.us.s3.amazonaws.com/10Mtestb.rnd

Time taken for tests: 48.351 seconds

Transfer rate: 216.87 [Kbytes/sec] received
And the EU bucket

ab -n 1 http://erichsen.net.eu.s3.amazonaws.com/10Mtestb.rnd
...
Time taken for tests: 47.907 seconds
...
Transfer rate: 218.88 [Kbytes/sec] received

ab -n 1 http://erichsen.net.eu.s3.amazonaws.com/10Mtestb.rnd

Time taken for tests: 50.943 seconds

Transfer rate: 205.84 [Kbytes/sec] received
With respect to transfer rate they seem to perform about the same from my local machine’s point of view. But I guess that this is what is to expect. The EU bucket should give better response times and for large files the response times are only a small fraction of the total transfer time.

But what about the small file?

US bucket

ab -n 50 http://erichsen.net.us.s3.amazonaws.com/50Ktest.rnd
...
Time taken for tests: 60.308 seconds
...
Time per request: 1206.16 [ms] (mean)
...

EU bucket

ab -n 50 http://erichsen.net.eu.s3.amazonaws.com/50Ktest.rnd
...
Time taken for tests: 26.676 seconds
...
Time per request: 533.52 [ms] (mean)

Now we’re talking!

Summary: For large files you could just as well use the US variant of S3. If you use S3 for serving the static files of your web site and most of your visitors come from Europe switching to the EU S3 should give your users significantly better load times.

October 27, 2007

tv2.dk traffic costs

Filed under: rails, scaling — enjoyingrails @ 15:50
Tags: ,

In an earlier posting we saw that tv2.dk in August 2007 served 142.132.680 pages and that the average page size was 395K and the average number of requests per page was 54.

How much traffic is this?

Doing some math it turns out that approximately 54 TB traffic and 7.7 billion requests during August! That’s a lot of traffic!

Jay.net in their Grand National package say that 33000 GB traffic/month will set you back 10000 kr. which is approximately 1900$. Note that it is 33000 GB Danish traffic and that you have to pay for international traffic as well. According to Alexa 87% of the tv2.dk’s traffic comes from Denmark. My guess is that tv2.dk’s 54 TB traffic in the jay.net setup will cost them around 4000$.

What if the site was hosted on Amazon EC2/S3?

One option is to let Amazon S3 serve all static content. Using the AWS Simple Monthly Calculator the traffic is going to cost 8,699.44$ for the traffic itself and a request fee of 7,675.17$ totalling 16,374.61$! Letting the content from S3 removes the need for a dedicated server to serve static content but the request fee makes it rather costly.

Another option is to have a dedicated EC2 server for serving static content. The traffic is going to cost 8,699.44$ and the server itself 74.40$.

So jay.net turns out to be much cheaper than the Amazon offer for a high-traffic site like tv2.dk.

Blog at WordPress.com.