Weird cable modem routing

Here’s a strange mtr report:

                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. router.nelson.monkey.org          0.0%   234    0.3   0.3   0.2   0.6   0.0
 2. 10.21.156.1                       0.4%   234   21.9  15.9   9.6  42.3   4.1
    104.220.254.153
 3. 10.21.156.1                       0.9%   234   24.4  16.5   9.6  71.3   5.6
    104.220.254.153
 4. 104.220.254.153                   0.0%   234   14.7  16.1  10.2  33.0   4.1
    cr1-200p-a-be-100.bb.spectrumnet.us
 5. cr1-200p-a-be-100.bb.spectrumnet  0.0%   234   23.3  17.2   9.8  42.5   4.6
    cr1-9greatoaks-te-0-5-0-13.bb.spectrumnet.us
 6. cr1-9greatoaks-te-0-5-0-13.bb.sp  0.0%   234   23.5  17.7  10.7  57.0   5.7
    72.14.204.109
 7. 72.14.204.109                     0.0%   234   13.9  17.6  10.5  37.9   4.4
    108.170.242.241
 8. 108.170.242.241                   0.0%   234   17.1  17.6   7.8  44.2   4.3
    216.239.49.83
 9. 216.239.49.83                     0.0%   234   15.0  17.2  11.4  36.6   4.5
    google-public-dns-a.google.com
10. google-public-dns-a.google.com    0.0%   233   12.5  17.3  11.1  47.6   4.8

It’s hard to read that at first, but what’s going on is the path is changing at hop 2. Sometimes my packets go directly to 104.220.254.153, sometimes they go through 10.21.156.1 first. And the packets take 4 or 5 hops before they get out of the local ISP and in to spectrumnet.us, suggesting sometimes they visit those other two addresses more than once. (I wish traceroute would break this out for me, show which fraction of packets take which route.)

That’s a capture from home in San Francisco, my ISP is Wave Broadband (aka Astound). They provide good clean 100Mbps without any of Comcast’s bullshit and have mostly been reliable. I had some problems yesterday (massive packet loss, lag) which got me looking. The problems went away when I rebooted my cable modem (which itself is WTF), but this weird routing remains.

I can’t tell if that reported 0.9% packet loss is real, or an artifact of mtr’s TTL trickery or reporting. A straight up ping to 8.8.8.8 showed 0 loss in 1500 packets, so it may not be real.

I have some vague theory that what’s going on is there’s local congestion and if stuff gets buffered, 10.21.156.1 becomes visible. But I have very little evidence for that.