self service super computing
Derek Gottfrid of the New York Times describes how he used Amazon’s EC2 and S3 to generate PDF versions of 11 million articles.
“I then began some rough calculations and determined that if I used only four machines, it could take some time to generate all 11 million article PDFs. But thanks to the swell people at Amazon, I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3.”