May 29, 2009
Sneakernet to the cloud
Recently, Amazon CTO Werner Vogels put up a blog post which suggested that, now and in the future, the best way to get large databases into the cloud is via sneakernet. In some circumstances, he is surely right. Possible implications include:
- When sending data to the cloud, you probably want to compress it to the max before sending. Clearpace’s new RainStor structured-data archiving service emphasizes that idea. RainStor marketing says cloud, cloud, cloud — but Clearpace thinks you really should have a bit of its software onsite too, to compress the data before sending it across the wire.
- Getting data from one cloud to another cloud could be problematic. I’m fond of saying that weblog data naturally lives in the cloud at your hosting company’s location, so you should analyze it there too. But this makes the most sense if you analyze it or at least filter/reduce it in place. (That said, the really, really big web companies have lots of different data centers, and presumably do move huge amounts of log data from place to place.)
But for one-time moves of data sets — sure, sneaker net/snail mail should work just fine.
Categories: Amazon and its cloud, Cloud computing, Database compression, EAI, EII, ETL, ELT, ETLT, Web analytics
Subscribe to our complete feed!
Comments
2 Responses to “Sneakernet to the cloud”
Leave a Reply
Yeah he makes a good point that network pipes cant keep up with data payloads. I think he’s right. I am actually surprised that someone has not yet come up with a business plan for just doing data “messengering” across the US specifically to ensure quick/secure delivery of data — like the FedEx of data. I’d charge by the TB 🙂 – When it absolutely positively _has_ to be there securely 🙂
[…] Getting data into the cloud in the first place is a lot slow. […]