Introducing s3tup

September 09, 2013Alex Guerra

After a few of weeks of work I'm finally kinda sorta finished with s3tup. Check it out here. In case you didn't follow that link, s3tup is a declarative configuration management and deployment tool for Amazon S3 that works with simple yaml configuration files.

Why? Don't you know about s3cmd?

I started on s3tup after trying to write a deployment script to push my own static site to S3. Doing it in boto was a nightmare. With the goal of increasing the speed of deployment and limiting unnecessary requests, I was mired deep in the boto source regularly trying to figure out how exactly all of the various methods mapped to their respective REST endpoints.

Eventually I decided that interacting with S3's REST api directly would greatly simplify my code, whipped up a quick make_request method using the requests library, and ported all of my old boto code over.

Over the past couple of weeks I've been iterating on the idea, writing tests, adding documentation, refactoring, etc, and I think that it's in a pretty decent state; the text you're reading right now was deployed with s3tup.

Though a few other packages offer 'rsync' like capability, they usually require outside scripting to function how you actually want. s3tup handles all configuration of the uploaded keys in an elegant way, and has the ability to sync configuration to keys that are already in the bucket as well, all through a simple yaml config file. Pretty slick.

It also does all that in a very low number of requests. "But Alex", you say, "requests to S3 are super cheap. Why does that even matter?" I don't know man. Stop asking so many questions.

Anyways, it's late, I'm delirious, and I should probably wrap this up. Hope that someone out there finds it useful. Goodnight.

PS: This is the first project I've ever registered with pip and (embarrassingly) the only project I've actively tried use git on, so my commits are shoddy. Don't judge. Gotta start somewhere, right?