Firefox GPU RAM leak

Update: upgrading from Windows 10 version 1903 to 2004 solved this problem. It looked like 2004 solved the problem, and it might have helped, but my computer just barfed again with Firefox holding on to 25GB of VRAM. And with only like 6 tabs open.

I’m chasing down a problem on my Windows system. Every few days it crashes, the GPU acts like its out of memory. I have a fancy 1080 Ti graphics card with 8GB of video RAM and don’t run things long term that use VRAM, so what’s the problem?


First trick was finding out what system processes were using video RAM. Process Explorer won’t show you. But the built-in task manager will. The “Performance” tab gives you a graph of GPU memory usage over time. And the “Details” tab will show you per-process if you add the display columns “Dedicated GPU Memory” and “Shared GPU Memory”.

When first launched Firefox allocates about 100MB of Dedicated GPU RAM. Then as I watched during the day it climbed; 2GB, 4GB… Even closing all tabs but one tab open to about:memory (as simple and passive a page as you could imagine) all that RAM was still open. After 24 hours it was 4GB. After 48 hours it was 17GB. (neat trick, since I only have 8GB of actual video RAM. I guess it pages?) Closing Firefox entirely does release the VRAM.

I’m still not sure what’s wrong with Firefox that it’s doing this. There are reports from users of this problem over the years: this bug is the best match, also this bug and that one. Not much resolution there. I also can’t find any tools to figure out what inside Firefox is holding the VRAM. That about:memory page is only for CPU RAM, not VRAM.

The most likely theory I’ve seen there is that hardware accelerated decoding of HTML5 video is the culprit. That the video decoder is somehow not releasing VRAM. FWIW my system has media.hardware-video-decoding.enabled set to true, the default. I think I can sort of reproduce that as a bug by watching Twitch and then closing it repeatedly. But not Youtube. I think a real culprit might be me browsing meme videos on Reddit using the Imagus extension (which opens media in a popup window). Another possibility is Reddit’s previews of RPAN streaming video. But I’ve not been able to reproduce it consistently in a way I understand.

Another theory I had was that it was related to the webrender experimental feature. But I have gfx.webrender.all set to false, so that’s not it. Also the leak seems to be triggered only by some kinds of pages, not all, and my impression is webrender would affect all pages.

One last bit of cargo cult advice gleaned from the Firefox bug reports is that Windows 1903 fix KB4497935 might improve things. Reading the Microsoft notes for that I don’t see how it’s related. But I am running 1903 without that patch. I think that’s the next thing I’ll try; install that patch and see if the problem goes away. (Also why am I not running a newer version of Windows?!)

Update: joke’s on me, I can’t install KB4497935 standalone. Installer refuses to run and says it’s not for my system. Maybe it was superseded by a later patch? In general I keep my systems updated, that seems likely. Anyway I’m going to just jump to Windows 10 version 2004 now, that’ll definitely mix everything up.

Update 2: I ran for a couple of days on 2004 and it definitely fixes the problem. At least, mostly. Firefox regularly returns big chunks if video RAM as soon as I close a tab. More importantly I kept it running for two full days and at the end it had maybe 500MB allocated. A bit more than the 100MB or so it started with, but well below the 8+GB that resulted in system crashes before. I don’t know what in particular fixed it, but something in the transition from Windows 10 1903 to 2004 either fixed or papered over the problem.

Update 3: I swear 2004 has helped. But just now I was playing World of Warcraft (uses 2GB of VRAM) with Firefox in the background with about 8 tabs. And WoW complained it had no RAM and when I checked, Firefox had 25GB of VRAM allocated to it. Quitting Firefox fixed it.

Computers are complicated.

UPS vs generator: PFC

I mentioned before my cheap UPS units didn’t work well on generator power. If something turned on (like a pump) that drew a lot of power the generator briefly didn’t keep up, the power dropped, and the UPS would switch to battery. Worse it would fail on my gaming PC, losing power entirely.

These are cheap UPS units, the minimal failover possible. My best guess why my gaming PC would shut down is it was drawing too much power. I never observed the thing taking more than 330W and the UPS was rated to 450W, but maybe that was enough to overload it. But the general unhappiness on generator power, the switching to battery, that seemed bad and fixable.

So far so good with a new UPS, a relatively expensive 1000W unit called the “CyberPower CP1500PFCLCD PFC Sinewave“. I think the key feature here is the PFC Sinewave, which is advertised as “provide sine wave output and correct minor power fluctuations without switching to battery”. That sounds like some form of power conditioning, even when not on battery. I think it’s a step up from basic Automatic Voltage Regulation, which “stabilizes the incoming AC signal to maintain output power at a nominal 120 volts by controlling high and low voltages without resorting to battery power”. Maybe AVR just clips the power at 120V while PFC actually creates a smooth sine wave? Dunno.

The PFC unit costs 33% more than a simple AVR. It may be overkill, I wonder if AVR would be enough to smooth out generator problems. Not sure I care, at this point I’m willing to pay a premium for things that work well.

The step up from this AVR/PFC stuff is a UPS that’s always running on battery. There’s AC power coming in to charge the battery, and AC power being generated off the battery, but AC never flows direct. These units are significantly more expensive (2x or more).

Oh, one downside of this CP1500PFCLCD is that it runs a fan when it’s on battery. Fairly noisy. Presumably a heat thing.

Ubuntu 20 vs Sonos: SMBv1

After I upgraded to Ubuntu 20.04 my music library stopped working on my Sonos. Turns out this is a Samba problem. Somewhere along the line Samba disabled SMBv1 by default; apparently it’s insecure and old. However some clients like Sonos only support SMBv1, so the result sucks.

The fix is easy. Add this to your smb.conf and restart Samba

server min protocol = NT1

I also have these lines lurking around for other obsolete clients, but I’m not sure they even work any more. They aren’t necessary for Sonos.

ntlm auth = yes
nltm auth = ntlmv1-permitted

Sonos sure could do a nicer job of error handling. The only error you get is “it failed” with a link to some irrelevant docs about Windows stack size. I figured out the real problem in this Reddit thread.

Apparently this is also a problem with Synology servers, which recently switched to SMBv2 or above only.

I don’t know if there’s a security risk in exposing SMBv1 this way. It’s all behind my firewall so I’m kind of hoping the Maginot line keeps me safe. Also it’s a freakin guest read only fileshare, why is there any login protocol at all? Ugh.

It’d be nice if Sonos upgraded to a newer SMB client, but apparently they’re not in a hurry to. And now given the major platform switch plus their emphasis on streaming services over local music, I’m a little afraid SMB may never be updated on Sonos.

Simple Unix file encryption

I want to password encrypt some Unix files. In the very old days you’d have used crypt(1) for this, but that’s deprecated now because the algorithm is junk. So many tools are deprecated now. The key thing is I need a simple tool that I can likely get working again in 10, 20 years. Also I want a file encryption tool, something that manages timestamps and deleting old versions and stuff.

I settled on using ccrypt. I don’t love that its on SourceForge but it was at least updated in the last couple of years, and the underlying cipher is good. Ubuntu 20 has it. One drawback is that it doesn’t compress files first, you’re on your own to do that.

Some other options:

  • mcrypt: the consensus choice for a crypt(1) replacement for a long time. Supports multiple algorithms. But it hasn’t been updated in 10+ years and is considered my many to be obsolete.
  • bcrypt: I used this for years, and it still works, but Ubuntu 20 stopped distributing it entirely. Still works when compiled from source. Not sure if the algorithm is still considered secure.
  • gpg: I’m not positive this has a symmetric file encryption mode but even if it does GPG has been such a hairball for so long, forget it.
  • openssl: apparently there’s a way to use this to encrypt files? No thanks.
  • zip: the original zip encryption algorithm is garbage, but various zip versions have replaced it. I don’t know there’s a standard one though.
  • 7z, rar, other archivers: same concerns as zip about being solid encryption and yet also being standard / stable.

Power monitoring generator, well

We have a new generator, since we live in a third world country where PG&E is unable to provide reliable electricity. It works pretty well, a fancy Generac standby unit with load shedding. Even has a decent website monitoring the generator via WiFi, also there’s hacker software for reading the generator’s state from a serial diagnostic port.

Anyway, when we’re on generator power the UPS units sometimes complain, switch to battery power briefly. I think I’ve also noticed the lights flickering. That got me worried the generator was putting out bad power so I hooked up my IoTaWatt to monitor the AC power.

The IoTaWatt’s main function is to monitor power draw through individual circuits. It directly measures current. But it also monitors voltage and frequency via an AC transformer sensor. Near as I can tell it samples that data many times a second but the most detailed data I can get from the status API is once a second. The most detailed graphs are every 5 seconds.

Generator load

Voltage and Frequency. Generator switches on at 12:06

As the above graph shows, the voltage and power doesn’t really change much when we switch over to generator power. Both PG&E and the generator are providing about 122V. The generator’s frequency isn’t quite as stable as PG&E was but a variance of ±0.1Hz isn’t that bad. (PG&E sometimes approaches that variance too.)

Well-related power blip

The graph above shows what happened as 12:17:30, one of the times my UPS kicked in. Turns out just then the well pump turned on (the red line), adding about 1.5kW load (or about double total power, the blue line) in that instant. The generator didn’t quite react fast enough. The power draw of some other loads (yellow and purple, basically room lights) dips briefly. More telling: the frequency of power, the black line, drops significantly, by nearly 0.15Hz. That’s the sign the generator is overloaded, it starts running more slowly and the frequency drops. It recovers quickly enough, but it’s enough to make a sensitive UPS unhappy and cause a visible flicker in the lights. Note the voltage (green line) doesn’t notably drop; I imagine the generator has a voltage regulator that smooths this out.

So that’s why the lights flicker and the UPS complains. I think that’s just a limitation of a consumer generator. It’s plenty of power to make everything run, but it’s not quite as clean as when you have utility power.


Looking at this reminded me to take another look at what the well pump was doing.

It’s still coming on every 30 minutes or so even when there’s no water demand at all, which suggests a leak somewhere. I know it’s not outside, because I turned that valve off. I’m pretty sure it’s not inside either, I’d have noticed. Maybe the well is leaking back into itself, against the check valve? Dunno. Like I said before it’s wasting about $0.50 of electricity a day, but otherwise seems mostly harmless.

Above is a cumulative graph of well power usage over 24 hours (blue) with transitory power (green). As you can see we use a lot of water overnight, when the irrigation is running, about 5kWH or $2 worth. The well doesn’t stay on continuously for much of this, other than one period about 6:15am. Our water system has no buffer of any kind; no tanks or reservoirs.

Ubuntu 19.10 to 20.04 upgrade failure

I seem to be cursed when it comes to Ubuntu updates. I just tried upgrading Ubuntu 19.10 to 20.04 (ie: eoan to focal) on a ordinary home server. Once again do-release-upgrade failed right as systemd was being updated. In the past that’s because the systemd upgrade broke my network configuration, because of course I’m doing the upgrade via ssh. (In a screen session though!) This time around it was worse, because the system wouldn’t boot:

- - -[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]- - -z

Well that’s exciting.

Turns out this is a well known problem with an easy solution. Basically if something interrupts an apt upgrade of the kernel at the wrong time, the kernel will be installed and chosen as the default but the matching initrd ramdisk to make it bootable won’t be properly installed. (ramdisk creation seems to be deferred to the very end of the apt process). The result is the system won’t boot with the new kernel. UEFI may be part of what makes this a challenge.

Fortunately there are usually other older kernels also installed that still work on the hard drive, so all you have to do is convince GRUB or whatever boot loader you’re using to boot an older kernel. Then you can go in and fix things.

The command that seems most clearly to fix things is
dpkg --triggers-only linux-image-5.4.0-29-generic
Of course you need to substitute the right package name there, for the latest kernel that’s not booting. You can find it this way by listing all packages installed with “triggers pending”:
dpkg -l | grep ^it
Another bit of advice on that Stack Overflow page is to run
dpkg --configure -a
This will force any packages that were installed but not configured to configure themselves. My aborted Ubuntu upgrade seems to have left me with ~40 unconfigured package, this fixed them all in a jiffy.

I sure wish I knew why upgrading systemd sometimes breaks networking on Ubuntu. It’s happened on two different systems of mine. And it’s an absolute disaster if you only have local access to the machine. Hilariously, do-release-upgrade still advises against running it if you’re ssh into the machine. How am I supposed to update the machine, fly to the datacenter and type at the keyboard?!

I also wish that apt or do-release-upgrade were a little more robust in leaving you with a bootable kernel.

The good thing here is that apt and dpkg are very robust themselves. This is the third time I’ve had do-release-upgrade fail on me now, but each time I can pretty much rely on apt and dpkg to fix whatever mess was created.

Bonus round: USB keyboards

This whole ordeal got a lot more complicated because I couldn’t find a keyboard to plug into the Intel NUC hardware to access the boot console. I use an old USB wired Apple Mac 101 key keyboard on my regular computer. For some bizarre reason that fails to work when plugged in to the NUC for accessing the BIOS. I ended up having to wait two days for Amazon to ship me a cheap $10 Chinese keyboard. That’ll teach me to not have a bunch of old computer parts lying around the house.

Making Amiibo for Animal Crossing

Nintendo has this system called Amiibo where you buy little figurines for a game and then scan them on your Switch console to do various in-game things. Breath of the Wild Amiibo unlock outfits, for instance. And Animal Crossing New Horizons Amiibo unlock new villagers to come live with you.

But Amiibo are expensive! New they are in the vicinity of $20 on Amazon. And there’s some collector / scarcity thing going on too so some of the rare ones are like $100. I’m not a big fan of paying real money just to unlock digital content. I mean sure I’ll pay for a major DLC, but I have no ethical problem creating my own unlock for a character.

Anyway, it turns out that Amiibo are just NFC 215 tags. And those are easy to write with pretty much any NFC-capable mobile phone, or with a $30 bit of NFC hardware. (Surprise! Your phone can write tags as well as read them!) And of course you can easily find the tag data online.

I ended up following this guide on Reddit. The FAQ is helpful too. In short:

  • Buy some blank NFC 215 tags on Amazon. They cost anywhere from $0.25 to $1.00 each depending on volume and shipping speed.
  • Get an NFC writer. I used TagMo on my Android Pixel 3 phone. It’s an unauthorized app, so you’re taking a small security risk installing it.
  • Get the special unfixed-info.bin and locked-secret.bin files installed on the tag writer. They’re only 80 bytes each. See the Reddit guides for where to find these.
  • Get the Animal Crossing tag data. This requires downloading files of dubious licensing status. A site called NFC Bank is a popular source.
  • Burn the tag. You can then scan it again to test it.
  • In game go to Isabelle and choose the Amiibo option for inviting visitors. Scan your new tag with the game controller.

The one small bummer is the cheap tags aren’t reusable. There’s alternate hardware which will spoof NFC tags for you. I think also some more expensive writable tags are re-writable.

Another option if this all seems too hard is you can apparently buy pirated Amiibo tags on Etsy, eBay, etc. Folks put a lot of effort into making nice cards or buttons for them too so they look good.

Firefox browser versions for Android

Summary: Firefox Browser is the stable consumer product. It’s about to be replaced with Firefox Preview, a major new release you may want to switch to early. There’s also a collection of betas and nightly builds.

One thing that’s nice about Android is that Firefox works nicely on it; there’s no iOS requirement that all browsers really just be reskinned versions of the vendor’s browser component. I’ve set up Firefox to be my default browser and that setting mostly sticks other than Google launching Chrome occasionally on me. I think Firefox is even used as an embedded browser in other things, but I’ve never been sure.

Anyway, the confusing thing is there’s so many things called Firefox on the Google Play store. Here’s a rundown of them all.

Firefox Browser (org.mozilla.firefox): this is the one you probably want, the mainline consumer app. This is currently version 68.8.0. This is the old Firefox that’s been around for awhile; this is not the new version announced in January 2020.

Firefox Preview (org.mozilla.fenix): this is the new hotness. Version 4.3.0 for me right now, but also says “GV: 76.0”. It’s a major update involving a port to GeckoView. See also these details of new features. Dark mode, more privacy, an option to put the toolbar on the bottom of the screen (near your thumb), a Private Browsing button right on the tab selector. Main change though is a full new rendering engine. Supports fewer addons, but does have uBlock Origin. This new stuff is slated to be released to the public as the Firefox Browser pretty soon.

Firefox for Android Beta (org.mozilla.firefox_beta): this looks like a slightly older version of Firefox Preview. Currently reports version 75.0.0-beta.6, also “GV: 75.0”. I would guess ordinarily this would be the cutting edge beta version of Firefox but right now during the transition to the new hotness, Firefox Preview is getting updates faster. Not positive.

Firefox Nightly for Developers (org.mozilla.fennec_aurora)
Firefox Preview Nightly for Developers (org.mozilla.fenix.nightly): I didn’t test these. I assume they’re just nightly builds of the two main development lines. FWIW the Preview Nightly currently reports “GV: 78.0a1” so it looks fairly far ahead of the main Firefox preview

Firefox Focus (org.mozilla.focus): a privacy-oriented experimental browser. Version 8.2.0 (75.0) right now. I think this was most relevant a couple of years ago and has taken a backseat to what is now Firefox Preview, which includes some of the same privacy features.

There’s also a bunch of stuff using the Firefox brand name that isn’t a browser at all. Firefox Private Network VPN (org.mozilla.firefox.vpn): is Mozilla’s new project selling VPN service to users. Firefox Lockwise (mozilla.lockwise) gives access to the Firefox password store. I think they’re trying to become a password agent like 1Password? Firefox Send (org.mozilla.firefoxsend) is a tool for sharing files. Last update was March 2019, not sure if the project is abandoned. Notes by Firefox is yet another text memo product. Firefox ScreenshotGo organizes screenshots with OCR and text search. And finally Mozilla Stumbler is a wardriving tool for building out Mozilla Location Service data. Last release Feb 2019.

Android Tablet: Samsung Galaxy Tab S5e

I just got a new Android tablet to replace my 2017-era iPad. Since moving to an Android phone last year (Pixel 3) I’ve come to realize I just like Android more. For ordinary stuff ordinary people do, Android is good, just as good as iOS. Sure there’s nits to pick with both systems, but they’re both great.

What I like about Android is that it also allows a level of more sophisticated stuff. There’s a brilliant capability for apps to interact with each other, for instance, setting things up like Unix pipelines. Android out of the box lets you customize the keyboards, the launcher, really most aspects of the system UI; no jailbreaking required. And a fair number of the Android apps seem more hacker friendly than their iOS equivalents.

Android’s ecosystem also does seem genuinely more open. Firefox integrates as a browser way better on Android than Apple ever allowed. Want to add more storage? There’s an SD card slot. (Why the F has Apple never allowed that?) Want to buy a book in the Kindle app? Of course you can; it’s not like iOS where Apple demands their 30% cut. Want to sideload an app from some random site? No problem, just disable the security check and you can. (A double edged sword, to be sure.)

The big downside to Android is that the vendors don’t distribute vanilla Android, they hack and munge and add crap. Worse they subtract value, particularly in how they sit on operating system versions for months, years, forcing folks to run out of date buggy versions. Google really needs to get more control of this problem.


Another problem with the Android ecosystem is the hardware quality varies. I dorked around with an Amazon Fire 7 briefly but the hardware is such garbage there was no joy in using it. (To be fair, it was super cheap.) And tablets are seen to be a losing market for some reason, particularly on Android, so there’s not a lot of options.

I read a bunch of guides online (this one was the best) and landed on the Samsung Galaxy Tab S5e. It has an excellent screen (2560×1600 AMOLED), is very thin and light, and at $350 is cheap enough to feel like an accessory / toy, not a whole laptop. The Galaxy Tab S6 is the new flagship, better in most every way, but at $650 it’s awfully pricy. The only thing it’s really missing is S Pen support but I had no desire to have that anyway. There’s an S6 Lite coming but it seems worse than the S5e for the same price, I don’t quite get it.

The S5e hardware seems great. Very solidly built. Nice camera I’ll never use, a working fingerprint reader for unlocking, wifi seems good. Screen is lovely although there’s some weird white balance software at work that’s not helping. Only complaint so far is I can’t find a magnetic cover just for the screen; all the cases are wraparound and cover the back too. Oh well.


So, what about the software? First, the big problem; it’s still only running Android 9. Android 10 came out eight months ago. Being on 9 puts me well ahead of a lot of crappier Android devices, but I hate being beholden to Samsung to get around to putting out an OS release.

Still it works nicely as a stock Android system, there’s none of the lockdown shenanigans that Amazon Fire does trying to keep you from customizing. The GBoard keyboard installs fine, including local voice recognition. Samsung’s launcher is OK but I switched to Nova Launcher mostly because it alphabetizes the apps tray view. Also it’s nicely customizable.

Samsung does install a bunch of its own crap, but it mostly stays out of the way. Bixby was the biggest annoyance but Nova Launcher mostly hides it. For some reason there’s two Clock apps and two Calendar apps and the like, one Google and one Samsung. Meh, I can live with it. It’s annoying Samsung won’t let me remove their preinstalled stuff.

But Android is hacker friendly, so you can install them! I followed this guide to uninstall a bunch of apps using adb to get a command line shell on the tablet. (Try that on an iPad, lol.) Another option is to use a guide like this one along with an app and config files which debloats the device for you. I wanted to control it myself, so I did it by hand.

I already had adb on my system thanks to having Android Studio. The adb.exe executable is in /Users/nelson/AppData/Local/Android/Sdk/platform-tools for me. Here’s a list of stuff I uninstalled by hand:


Special GSAM bonus notes

I use the GSAM Battery Monitor to track battery usage for apps. It works fine by default, but it can work better if you grant extra permissions to the app to let it get at some debugging APIs. There’s multiple conflicting instructions online about how to do this, including a very nice tutorial in the app itself that works right until the final crucial command. I think this worked fine on my Pixel 3? But it didn’t on this tablet, what finally worked was at the bottom of this discussion:

adb -d shell pm grant com.gsamlabs.bbm android.permission.BATTERY_STATS

TCL Series 6 TV

Just got a new TV, a TCL 6 Series and couldn’t be happier. I bought it after this glowing review. Pretty much every other review says the same thing; excellent picture quality for the price. It’s quite cheap even for an LCD TV, and definitely way way cheaper than an OLED.

I’m not able to judge picture quality except to say it looks great to me. It’s better than the 2013 era Samsung panel it replaced; brighter, bigger, sharper. The 4K 60fps demo reels look amazing. Of course the default picture settings are oversaturated garbage, but fixing that isn’t too hard.

The key picture setting for this screen is “movie mode”. That means “don’t blow out the colors”. However when you enable that it also for some dumb reason defaults to “warm color temperature”, which shifts all the colors to the red. So set that back to normal. Also it turns off Local Contrast which is one of the nice features of the set, so turn that on to high. Then turn off all the fancy motion interpolation type stuff and it looks pretty great.

I particularly like that there’s a quick setting for “darker / normal / brighter”, which means you might actually change that based on time of day and room lighting. There’s also a game mode for HDMI inputs that has decent latency (19ms, or just about one frame).

One nice feature for this set is that its operating system is Roku. Not some dumb badly designed TV manufacturer stuff, just good ol’ Roku with its open ecosystem of third party channels. I’ve been using Roku devices for years, mostly for Plex and Netflix and Amazon, it’s nice to have it native in the TV. I managed to get ARC (“Audio Return Channel”) working so the sound is still being sent to my fancy AV receiver for decoding.

Speaking of the receiver, I’m less clear on what to do with it now. It’s limited to 1080p (or 2160p @ 30 fps, lol). And seems to block HDR too, or at least my PS4 won’t do HDR through it. I guess I could just plug all my inputs in to the TV and use the receiver solely as a digital audio decoder + amplifier. The wiring is tidier if the receiver is a switch, but then I can’t pass 4K/HDR content through it. If I didn’t already have speakers installed in my walls I think I might just get a nice soundbar and call it a day.

Still haven’t considered the joy of making my old Logitech universal remote drive the new system. Need to settle down how the receiver fits in first.

4K is still such a bandwidth hog I can’t imagine I’ll be using it much if any for watching video. Would like it for games though. HDR interests me much more, it’s great we can finally get higher contrast ratios on TV sets. Apparently there’s competing standards for HDR, awesome. This panel supports both Dolby Vision and HDR10, which I guess are the two most common formats. Hopefully that’s good enough.