S3 interface options

I’ve never really done much with S3 and now I am. Here are some tools I’ve tried to manage files on S3. None are very good, maybe someone can recommend better?

The Amazon web console works but is an awkward Web 1.8 app.

s3cmd is the common command line tool. It works fine. Despite it looking like an official Amazon product it’s not,  it’s “Copyright (C) 2014 TGRMN Software” and seems to be by Matt Domsch.

s4cmd claims to be a better s3cmd. Note that “pip install” installs the program as s4cmd.py. My one attempt to use it (“du”) did not work; said “1 task completed” but printed nothing.

s3 FUSE lets you mount an S3 bucket as a filesystem using userfs. The project comes with a lot of caveats, most notably that S3 doesn’t really have a notion of directories and while everyone mostly uses the same object naming convention to create a hierarchical namespace, it’s not entirely consistent. I didn’t try this one.

Cyberduck is a venerable graphical FTP client for MacOS that has S3 support. It works pretty well, it’s the main tool I’m using. You have to keep pressing “Refresh” if you want to see any updates to the bucket state. Also it complains about certificate names not matching when you run it.

Boto is the Python library everyone uses to write code to talk to S3. Here’s the docs. It does a lot more than S3, it’s a general AWS library.

Credential management is a nuisance. Two big opaque strings for keys and no standard on how to find them.

  • s3cmd uses $HOME/.s3cfg
  • boto uses $HOME/.boto
  • Cyberduck uses the MacOS keychain
  • The OpenAddresses tools I’m using take -a and -s command line flags.
  • Environment variables are popular, different tools use different ones. Sadly, environment variables are not private on shared Unix systems. Neither are command line arguments.