Nelson's log

Twitter API, tweepy, and error handling

I’m writing some Twitter API code. A few lessons learned in using the tweepy library.

The error handling is not awesome. All API errors throw tweepy.error.TweepError, which is fine, but that object has no structure. The docs have some stuff about getting codes that seems outdated. In the code there’s an api_code field , but it’s set to None. I’m doing stuff like e.reason.endswith(“401”) to detect the 401 error code Twitter throws for protected users. (This is also the code Twitter throws for an invalid token, so that’s not great.)

The rate limiting handling is pretty good. In particular there’s a wait_on_rate_limit option that is pretty handy for simply waiting for a rate limit to expire.

When paging through user_timeline the default fetch size is only 20 tweets. You can instantly get 10x the effective calls by bumping up to count=200.

Twitter’s rate limits are confusing. Nominally it’s “calls per 15 minutes” but there must be some sort of grace period or something built in because sometimes you get more calls than that. There’s a nice rate_limit_status API call that is very helpful but is sometimes out of date; in particular I’ve seen reset times that are now in the past, so apparently my limit reset (and calls are working) but the status call doesn’t tell me that. Anyway, here’s some Q&D Python code to print out which API calls you’re currently at less than maximum on.

r = twapi.rate_limit_status()
now = time.time()
for resource in r["resources"]:
    for name, status in r["resources"][resource].items():
        if status['remaining'] != status['limit']:
            print ("%4d of %4d left, reset in %ds %s" % (status['remaining'], status['limit'], status['reset'] - now, name))