Poor Mans Edge-Cache

I have been playing around with the Amazon Web Services and S3 in particular over the last couple of days. It’s something I have planned to do for a long time now but never have come around till now. But I got a kick in the but to wake up and smell the Amazon Kool-Aid (ok, too many mixed metaphors and pop-cultural references) when I heard Jeff Barr’s presentation “Web Services: Fuelling Innovation and Entrepreneurship” from d.Construct 2006 about AWS, especially how easy it is to get started and play around with the services. One of the tools he mentioned, S3 Firefox Organizer, a Firefox add-on, had me up and running in matter of minutes.

S3, the Simple Storage Service, is indeed aptly named as Jeff Bezos usually puts it “You can do two things: put data in and get data out.” It’s a simple solution (to the user, me) to a difficult problem that can now be regarded as solved. But no matter how simple it is, S3 is incredible versatile — only your imagination sets the limits. Not only can you put data in and get data out via a REST API, you can also query for keys, download your data via bittorrent, through a regular web browser and get virtual host buckets branded with your own domain name.

Buckets are just like directories in a filesystem where you store the data, and because S3 otherwise has a flat namespace they’re needed to divide it up and ensure that you can create keys unique to you. So what about the “virtual” and “hosted” business then? Normally, you would access the data object bar in the bucket foo like http://s3.amazonaws.com/foo/bar. But if you own a domain name and can edit your DNS record, say inserting a CNAME alias that map foo.example.com to s3.amazonaws.com, makes it possible to access the data object as http://foo.example.com/bar instead (the bucket itself needs then to be renamed foo.example.com for it to work).

That feature is what I initially took advantage of to create a poor mans edge-cache. Edge-caching is a method intended to speed up sites (usually with high volume of visitors) by caching all static files (like javascript, css, images) on a network of caching servers and point their URIs to the cache. Since the Amazon S3 cloud replicate your data over several data centers, I assume they’re not just doing it for backup reasons but also for load balancing, it’s pretty safe to assume the S3 cloud will always serve your content quicker than your regular web server.

So last night I uploaded all my javascripts, css files and images for the WordPress themes and plugins, that are now being fetched from a server aptly named static.dahnielson.com pointing to the S3 cloud. My next step will be to upload rest of the static files and turn my regular web server into something more of an “application server”. Another benefit is the fact that both my blogs (in English and Swedish) now fetch static files from the same place.

This entry was posted in Besserwisser and tagged , , , , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

One Comment

  1. Posted August 15, 2007 at 01:00 | Permalink

    Thanks for detailed step-by-step writeup. I am trying to setup exactly the same thing with S3. This post helped me a lot.