IPython and inline matplotlib

I’m trying out IPython. I’m also playing around with machine learning which means I need some simple data visualization, so I’m playing around with matplotlib. Here’s how to display generated plots inline (as opposed to in a popup window); the docs are inconsistent.

QT: ipython qtconsole –matplotlib inline

Notebook: launch with “ipython notebook”, then run “%matplotlib inline” in a notebook.

Here’s a sample program that generates and shows a plot:

from matplotlib import pyplot as plt
years = range(1950, 2020, 10)
gdp = [300, 543, 1075, 2862, 5979, 10289, 14958]
plt.plot(years, gdp, color='green', marker='o', linestyle='solid')
plt.show()

Roku Twitch: variable bandwidth usage

The Roku got a new official Twitch client recently. It has a remarkably neat trick: variable adjustment of bandwidth usage while playing a stream without interruption. I’ll be watching the stream in what looks to be 480p quality. Then it drops to lower quality: 360p, or 240p, or even the impressionistic smeary stream for mobile devices. Then it pops back up to 480p when it can. All the while playback is uninterrupted. Crucially the audio never cuts out or skips, so the overall viewing experience is remarkably smooth.

Here’s a bandwidth graph of 10 minutes of me watching RiotGames’ official LCS channel yesterday.

Roku Twitch bandwidth graph

My network is capable of 2600 kbits/sec but only for a burst of ~30 seconds. Then I’m throttled back to 1024 kbps for awhile. Here you can see the stream is mostly consuming about 1000 kbps but occasionally enjoys the burst up to 1500 and once even dropped down to 660 or so. All the time that was happening my playback was quite smooth and solid, the only indication anything was going on was that the graphics got a bit fuzzy sometimes.

Most video streaming apps online seem to have one HTTP stream for each quality of video, and once you start streaming it you’re committed to that bitrate. The Twitch app must be doing something smarter about grabbing a few seconds of stream at a time and dynamically altering the bitrate depending on measured bandwidth. That may explain the 3s buffering time before the stream plays, too. I’ll gladly take that tradeoff.

In the long long ago of the early 00s Internet, video streaming was dominated by RealMedia’s proprietary protocol. That was UDP, so packet loss could be handled by the app. And it was a progressive codec. I think Twitch is doing TCP and not a progressive codec, so it’s more like it’s just dynamically altering which stream it asks for on the fly. Seems to work pretty well.

(BTW, Twitch is still coy about publishing details on video quality and bitrates. Their’ partners page has some samples though, which claims

  • 240p: 384kbps
  • 360p: 512kbps
  • 480p: 768kbps
  • 720p: 1161kbps
  • 1080p: 1500kbps

I’m suspicious of the correctness of these numbers. They’re too low, also too much like round numbers except 720p. Maybe the numbers are video only and they’re doing transcoding to exactly hit a video bitrate? Seems unlikely.)

Leap second fallout

I looked for evidence of major leap second bugs and didn’t find many. The big news was AWS went down around 30 minutes after the leap second, but that turned out to be an improperly published route, not directly a leap second thing.

But DynResearch published a great graph of BGP route instability around that time, a 15x increase in BGP advertisements over normal.

Leap second BGP spike

I wonder what the cause of that was and whether it caused any problems for civilians. My guess is that it is related to dynamic load balancing algorithms assuming a day only has 86,400 seconds.

More info in this article, which also notes that 2000 networks stopped working briefly.

Leap second 2015

Both my Mac and my Linux box seem to have survived the leap second, at least so far. Both systems just reported 23:59:59 taking two seconds. A friend with a MacOS 10.10 box says he saw 00:00:00 briefly. I saw that in 2012, and it’s a serious bug (the date flips too!) but didn’t today.

while :; do date; sleep 0.1; done

A screenshot from this Javascript viewer. (Which has a fun easter egg; the clock started running backwards after this image.)

23:59:60

League of Legends bandwidth usage increasing

I pulled some new stats out of Logs of Lag, the average bandwidth used by the game client. Each point is the average for all logs I have for the week. It shows that the client has roughly doubled its bandwidth usage over the past three years.

LoL bandwidth usage

The absolute usage is still very small; well under 50kbps, or the speed of good ol’ dialup. (Burst usage is significantly higher though, maybe 4x.) Mostly I find the trend interesting. Also surprised not to see a clear step function as patches are released, although maybe that pattern is hiding in the data.

As with my previous analysis the huge caveat with these reports is that I don’t have an unbiased sample of LoL players. I only have logs  uploaded to me by Logs of Lag users, which are undoubtedly biased towards people with network troubles. I think that bias won’t skew this bandwidth usage number much though, it has much more impact on any inferences I can draw about player ping times, packet loss, etc.

Speaking of those skewed data, here’s an update from last fall’s summary with significantly more sample logs now.

matchLength = 1736.0  (29 minutes)
bestPing = 80ms
medianPing = 90ms
averagePing = 103ms
stdDevPing = 53ms
bytespsIn = 4706 bytes/s
bytespsOut = 666. bytes/s
lostPerMin = 11.5 packets/minute

Running mocha tests

Dusting off some old Javascript, I’d forgotten how to run my Mocha / Chai test suite. Running “mocha” in the main directory is the basic thing, it finds tests in a subdirectory named “test”. The nice way to run it continuously is

mocha -w -R min

SmartTVs

I just bought a new TV. They are now plagued with a huge number of “smart” features you have to turn off to make the set useful. You pretty much can’t buy a TV without these.

First off is all the video processing. The motion interpolation / dejudder / etc that takes a lovely mastered Blu-Ray source and then makes it look like shit and 100ms out of sync with the audio. Fortunately TVs generally have a “game mode”, so-called because gamers want low latency displays. As a side effect it disables all the video processing.

Then there’s the Smart TV stuff. I’ve been buying Samsung TVs and watching their apps evolve. It’s not bad; you can legitimately watch Netflix or (probably) Youtube entirely on your TV now. Only problem is that there’s no simple way to get sound back out of the TV and in to your nice wall-mounted surround sound speakers. So all this smart TV stuff is useless unless you want to listen to 5W of shitty stereo. (Not strictly true; there’s a new HDMI 1.4 standard called ARC that does feed audio back up the cable.)

The Smart TV stuff is a little useful if you really just want a simple TV to watch movies in the kitchen. But for anyone who is at least a little serious about home theater, it’s useless. Worse than useless. Because now your TV has an operating system. So it takes a long time to boot, is vulnerable to security holes, and is popping up features and notifications when you don’t want them. I was particularly charmed to see this year’s Samsung TV includes a virus scanner. In the display. Awesome.

There used to be a class of flat panel displays called “monitors” that were really just a screen; no speakers, no “smart TV”, no bullshit. Unfortunately that market segment is gone. My suspicion is that the TV manufacturers are hoping to one day capture customers and turn them in to Internet service subscribers, so that they make more money than just selling the fancy glass panel. Ugh.

Bonus link: simple TV feature comparison matrix. Samsung’s marketing literature makes it impossible to understand what you are actually buying. What’s the difference between a JU6500 vs a JS8500? About $1000, but their web site does a terrible job informing you what that $1000 buys. This third party site has a simple feature matrix. May they make many dollars in Amazon referral fees.

Addendum

XKCD #463