Black paper experiments

One of the dramatic effects you can do with a plotter is printing in bright inks on black paper. The ink has to be opaque for this to work well; you want to completely hide the black paper below it. Here’s some experiments with some new pens.

Three kinds of ink in the top sample: Uniball Signo 1mm, a Pentel brush in the middle that totally didn’t work, and Kuretake ZIG Metallic Calligraphy pens. Most all of these look good on black paper. The Uniball Signo is a pretty solid low cost choice, a rollerball. The white ink is clumpy though, you can sort of see it in the top left sample. Uniball Signo Silver works better; that’s what I used for the plot at bottom.

Photographing black paper is tricky. The above image is straight from my Pixel 3. Shooting in full sun was clearly a mistake. Needless to say it looks more like black in real life, and the lines pop pretty nicely.

White ink is particularly problematic: see here or here for example. That second post is from someone using Rotring Isograph Pens, fancy technical pens you fill yourself with drawing ink. That lets you choose from a wide variety of ink colors; Rotring sells 5 including (unusually) a white. Some folks use fountain pens for fancy white ink, too (including I think the first post.) There’s even a White Ink product experiment going on right now: Wink.

But I’m not messing with liquid ink yet. For cartridges see this plot featuring a Pentel hybrid gel pen. Or this one with a Sakura Gelly Roll. The silver Uniball Signo I used here works pretty well but the white is not reliable.

Fixing irrigation: all valves stuck open

More adventures in irrigation systems. We wired up a new controller (our third) for a few valves, programmed the system, and let it run. The next morning every irrigation valve was open, water running everywhere. We don’t have enough water pressure / well capacity to actually run that so more like things were trickling.

I think I found my answer on this web page. These valves are designed to close automatically if there’s no power. But if an air bubble gets trapped in the valve diaphragm it can get stuck open. I think many of my valves were in this state. The solution for that the pagesuggests is to manually open the valves (with the bleeder valve or by unscrewing the solenoid). I did that and I think managed to get all 23 valves to close.

How do air bubbles get in? The article talks about this happening after you’ve drained the system for winter and restarted it. In my case I think what might have happened is the new controller was programmed wrong (or clocks out of sync) so that two different high flow sprinklers were running at the same time. And that overwhelmed the water pressure and let some air in, but maybe only a little. The worst thing is this is a cascading failure; with valves stuck open there’s only going to keep being intermittent water pressure.

I asked two different irrigation experts if they could understand this, they were mystified. Both kept wanting to blame something electrical. Sort of makes sense; if your expertise is water and plants and dirt then the computers probably seem a little unpredictable. But all the controllers were literally off. Also 3 disconnected control systems, no way they would have all failed the same way at the same time. The only common connection was the water. Systems thinking helps here.

Now I have to carefully test all this before I trust it again; it’s not OK to leave town with an irrigation system that may fully fail open. I have the water supply turned on but all valves shut off and all controllers unplugged. Going to let that sit for 24 hours and verify it’s dry. Then watchfully turn controllers on again and run the valves manually and verify they turn off and on. Then turn on the programs again. Probably just the two old controllers first, verify that doesn’t break anything, then if that’s working re-enable the new controller last.

Distribution of 2d noise functions

So I’ve been using noise functions in my plotter art. One thing to be aware of with noise functions is the output distributions aren’t uniform. I don’t just mean the smooth variation of nearby points that you’re using a fancy noise function for, but even if you sample uniformly from a large space the values are still closer to a normal distribution than a uniform distribution. To put it more concretely; these noise functions all return values in the range [-1, 1]. (2d Perlin noise is [-0.7, 0.7] ish.) But results near 0 are more common than results near 1 and -1. See below for some histograms.

This isn’t necessarily bad. You can always scale the output up to a uniform distribution or whatever you want. Although that requires knowing what the original distribution actually is. Perlin noise looks like a real normal distribution but both Simplex and OpenSimplex noise have a weird boxy distribution where results are uniformish in the range of -0.5 to 0.5.

More than the usual caveats apply; this is sloppy code and I might have gotten it wrong. Histograms are for 1 million samples drawn from OpenSimplex and caseman’s Perlin and Simplex noise. 2d noise in each case; the coordinates were uniformly sampled in the range of [-100_000, 100_000]. (A small range because caseman’s code seems to be using 32 bit floats). I did not alter the scaling or octave parameters at all, library defaults. My code is in this gist.

Perlin noise
-0.95
-0.85
-0.75
-0.65
-0.55 ***
-0.45 ***********
-0.35 *****************
-0.25 *********************************
-0.15 *******************************************
-0.05 *************************************************
 0.05 **************************************************
 0.15 ********************************************
 0.25 **********************************
 0.35 ****************
 0.45 ***********
 0.55 ***
 0.65 *
 0.75
 0.85
 0.95
 
Simplex noise
-0.95
-0.85 *********
-0.75 **************
-0.65 *************************
-0.55 ************************************************
-0.45 *************************************************
-0.35 **********************************************
-0.25 **********************************************
-0.15 ***********************************************
-0.05 **********************************************
 0.05 ************************************************
 0.15 **************************************************
 0.25 ************************************************
 0.35 ************************************************
 0.45 **************************************************
 0.55 *************************************************
 0.65 ************************
 0.75 **************
 0.85 *********
 0.95
 
OpenSimplex noise2d
-0.95
-0.85
-0.75 *****
-0.65 ***************
-0.55 **************************
-0.45 *****************************************
-0.35 **************************************************
-0.25 **************************************************
-0.15 *************************************************
-0.05 *************************************************
 0.05 **************************************************
 0.15 *************************************************
 0.25 *************************************************
 0.35 *************************************************
 0.45 ****************************************
 0.55 **************************
 0.65 ***************
 0.75 *****
 0.85
 0.95

Update: I asked about this on the Plotter Art Discord and someone suggested empirically measuring the distribution then doing a correction. They offered this matlab snippet:

p = randn(1000,1);
p2 = (p-min(p))/(max(p)-min(p));
x = sort(p2);
CDFp = linspace(0,1,numel(p2));
pUnif = interp1(x,CDFp,p2);

CO2 sensing and air circulation

Ever since I got a CO2 sensor I’ve been low-key interested in my CO2 level. Partly just because it’s interesting, partly because CO2 is related to cognitive ability (here and here), and partly theoretically because CO2 is a proxy for air circulation in general and thus, potentially, Covid prevention. That latter point is apparently making waves in schools.

Here’s some benchmark numbers for CO2 levels. These numbers are all ppm, parts per million. The important number is 420ppm, which is what you get outside right now.

Recent research has found a significant cognitive effect for all levels of CO2. Even going from 400 to 600 to 800ppm had a few percent impact on Stroop color/word test performance. Fresh air matters! (Corollary: global warming is directly making us all dumber.)

CO2 as a proxy to air circulation for disease prevention is a newer idea. Honestly it used to be too hard, but now that you can carry a $100 CO2 sensor with you (or in your kid’s backpack) it is much easier. It’s really not clear what the “right” level is. CDC’s 800ppm recommendation seems reasonable but also possibly challenging.

CO2 in my house in Grass Valley

My house is well sealed and insulated. I’ve learned that during the day CO2 at my desk gets to about 800ppm. The wacky thing is it doesn’t drop to 420 overnight, even if I’m in another part of the house sleeping peacefully.

Image
24 hours of CO2 with no one home.

I left the house at 11:30am, came back the next day. CO2 had just about returned to 420.

Anyways, now I open a window when I can. It doesn’t keep the CO2 down much at my desk while I’m in my office, but the air clears out pretty quickly when I’m away for a bit.

This is all for summer / fall. The air conditioner may be running and doesn’t seem to affect things. But the heater isn’t. I believe last winter I noticed the CO2 fell every night even with all the windows closed and I suspect that might have something to do with the heater or outside temperature, but I don’t have the data to back that up. I’ll know in a few months.

Home HVAC systems typically don’t have anything designed to supply fresh air. The air systems are closed to prevent having to condition outside air. Older houses are so drafty it’s not a big problem. But it is possibly a small problem for a new, well insulated house. I asked my HVAC guy and he said an outside air intake vent could be added for $2500 or so. Presumably it’s not open all the time (like a window), that has a significant cost in temperature and humidity.

Commercial buildings are designed to standards including air changes per hour. Nominally, an ACH of 1 means that all the air is replaced once an hour. ACH standards are 4 ACH at the low end to 20 or more for places that generate lots of pollution, like kitchens and paint shops. I’m not quite sure how to convert my CO2 information into ACH; this presentation suggests it’s possible but maybe a novel idea. Presumably there’s some sort of half-life calculation involved.

The main conclusion I draw from all this is it’s good to open a window now and again.

Python static blog generators

Once again contemplating moving my blog away from the ancient Blosxom setup I have. Static generators are clearly the way to go. Here’s what I found among Python options.

Pelican looks to be the most popular. Nikola is also compelling. Lektor may also be an option. Cactus and Hyde are abandonware.

I may also consider using Hugo. Go, not Python.

USB charging in 2021

USB charging is even more of a mess than ever before. I thought USB-C and USB Power Delivery was the be-all and end-all of charging. One common protocol, buy a charger that claims 45W and a USB-PD device would charge at 45W. Ha!

I bought a nifty little USB charging tester. It shows you in real time what the current voltage, amperage, and thus wattage of the charging current is. Finally a reliable way to tell what’s going on! Assuming the presence of the tester doesn’t disrupt the USB charging, that is.

The charger Amazon sent me is called a KWS-1902C, there’s a nice review here. It claims to support up to 150W, 4-30V, and 0-5A. The first look on the blog is from October 2018, for a sense of product age. I’m assuming it does not support USB 3.1 gen 2. But it may not need to, maybe it will do that with the right cable actually providing the E-marker chip. It apparently has a display bug if you go over 65.536W.

My Samsung Galaxy Tab S7+ (aka SM-T970) claims to support up to 45W of charging. But when I plug it in via USB-C to an Anker Power Port Atom III, advertised as a 60W charger, I only get 15W. 9V at 1.67A. Actually a little less than that, like 14.3W. I’ve tried two USB-C cables, also an A-to-C cable using the A port on the charger. Always 15W. So what’s going on?

First off, the Anker charging port is not actually 60W. It’s 45W. And that only at 20V. At 9V the max is 3A, or 27W. But still I’m only getting 15W.

Second, Samsung’s support for “fast charging” relies on a USB spec called “Programmable Power Supply” or PPS. That’s a newer thing where the device just requests an arbitrary voltage and amperage and the charger provides it. At least one other Samsung device wants to be charged at 10V and 4.5A. Samsung’s own 45W charger will do PPS up to 11V at 5A. But if it falls back to ordinary USB-PD 3.0 then it maxes out at 9V at 3A.

But that’s still 27W. I don’t know why the Anker charger is only putting out 15W. A failure to negotiate a higher amperage? Even basic USB PD 3.0 supports 9V/3A but I’m not getting it. Maybe the device deliberately charges itself slowly to preserve battery life? I haven’t tried charging from a dead battery.

My motherboard’s USB ports charge the tablet at 5V/1.5A. I believe that’s the minimum to be a USB-C port.

Back to PPS, Anker does make a couple of 65W PPS chargers but they seem to be newer and/or more expensive. This review suggests it’s capped at 3.25A so at 9V you’re limited to 29W. Every charger brand seems to have one or two PPS chargers but most chargers don’t support it.

How about charging other devices?

My Google Pixel 3 phone should support 5V/3A or 9V/2A. I get 9V at 1.2A. So that’s confusing. It’s variable too; 1.1-1.3A. I’m beginning to doubt the accuracy of my tester. Although again, maybe this is a battery preservation strategy? (Not USB, but wireless charging tops out at 10W.)

My PS5 controller can’t be charged by the Anker charger, there’s no orange light. The official specs say it can do 5.1V at 2.8A. When plugged in to the Anker charger I see it’s getting 5V and 0.1A, which is the basic USB 2.0 draw for a low power operating device. Plugged in to the PS5 in rest mode (with the meter) it does charge, at least the orange light is throbbing. But it’s still only 5V and 0.125A, very low power. So it may not be charging well. (Update: after six hours it was still looking like it was charging. Maybe the tester screwed it up?) Also I note the D- data line is at 2.76V; for most devices both D+ and D- have been at 0V when charging. Presumably that’s related to why the controller can’t be charged by the Anker charger, must be some odd protocol.

Bottom line: absolutely none of my devices are charging at the rated spec. Anker in general makes good products, so I’m not inclined to blame the charger. Although at least for the tablet it’s clear I need a PPS charger to get maximum charging.

I wonder if it’s my cable to blame. I’ve tried several decent cables, including an Amazon Basics cable. But they’re a couple of years old. Apparently the new hotness is USB-3.1 Gen 2, which has an active chip in the cable itself for negotiating USB-PD and supports 10Gbps data, 20V/5A charging. (Keyword: E-marker chip.) I am going to buy this one and see if it makes a difference. Amazon Basics makes one too. Not clear it will make a difference, from what I’ve read it only becomes important when you want to go over 5A.

Update: the new cable makes no difference.

SourceSinkVoltsAmps
Anker Power Port Atom IIIGoogle Pixel 391.2
Anker Power Port Atom IIISamsung Galaxy Tab S7+91.67
Anker Power Port Atom IIIPS5 Controller50.1
PS5 Console (with meter)PS5 Controller50.125

Syncthing status from the command line

I’ve written a little tool to show me the status of Syncthing on the command line. I now rely on it to get my files off of computers that will then be turned off and inaccessible. I didn’t have any way to know if it was really working. I don’t have a reliable outage monitoring system for all my personal tech crap. (I mean, who does? Hmm..)

Anyway this turned into a bit of a yak-shaving exercise. Syncthing does not really have a command line tool. There is a syncthing cli command (and formerly the syncthing-cli third party tool) but they don’t report the status I need. What syncthing does have is a very nice REST API for querying status, so I wrote a tool to check that.

The other tricky thing is defining what it means for syncthing to be synced. I’m not sure I got this right. The algorithm I’m using is it enumerates all folders and all remote devices and checks completion on each one. In practice folders seems to catch outgoing changes, remote devices catches incoming changes. It ends up being O(n) HTTP calls, not ideal, but here we are. It might be possible to get it down to 3; I didn’t quite understand how the bundling in rest/db/completion worked.

Anyway, the Python script below mostly works. I wonder if someone else has already done this and I reinvented the wheel? One last wrinkle; when to run this. I was originally thinking in MOTD (hence my earlier post) but for now I’m running it in .bash_login.

#!/usr/bin/env python3
"""
Status report for syncthing, suitable for including in /etc/motd.d. By Nelson Minar <nelson@monkey.org>

Note that this will show "completely synchronized" if a file was very recently created or altered and
syncthing is not yet aware the change has been made. See the scanning and fsnotify docs in Syncthing for details.
"""

import json, urllib.request, platform, itertools, sys
from pprint import pprint

"Configure the URL endpoint and API key based on hostname. API key is in Settings."
HOST = None
API_KEY = None
config_table = {
	'ub': ('http://127.0.0.1:8384', 'XXXXX'),
    'Nelson-Win10': ('http://127.0.0.1:2103', 'XXXXX')
}


def st_api(command):
	"Call the SyncThing REST API and return as JSON"
	req = urllib.request.Request(f'{HOST}/{command}')
	req.add_header('X-API-Key', API_KEY)
	resp = urllib.request.urlopen(req, timeout=2).read()
	return json.loads(resp)


def main():
	global HOST, API_KEY
	try:
		HOST, API_KEY = config_table[platform.node()]
	except:
		# Exit silently if we're not configured
		sys.exit(0)

	folder_report = []
	device_report = []
	try:
		# Explicit error handling for first REST connection. Others are YOLO.
		devices = st_api('rest/config/devices')
	except Exception as e:
		print(f'Syncthing: Error connecting to {HOST}.')
		sys.exit(0)

	# Check completion for all remote devices
	for device in devices:
		completion = st_api(f'rest/db/completion?device={device["deviceID"]}')
		device_report.append((device["name"], completion["completion"]))

	# Check completion for all folders
	for folder in st_api('rest/config/folders'):
		completion = st_api(f'rest/db/completion?folder={folder["id"]}')
		folder_report.append((folder["label"], completion["completion"]))

	# Show a short status message
	if all([c == 100 for n, c in itertools.chain(folder_report, device_report)]):
		print('Syncthing:', f'Completely synchronized.')
	else:
		incompletes = ((n, c) for n, c in itertools.chain(folder_report, device_report) if c != 100)
		print('Syncthing:', 'Incomplete.\n ', '\n  '.join(f'{n}: {c:.2f}%' for n, c in incompletes))
		print(f'Details at {HOST}')


if __name__ == '__main__':
	main()
	sys.exit(0)

Ubuntu 20.04 motd

I was trying to figure out how Ubuntu systems print the Message of the Day at login. In ye olden days this was a static file, /etc/motd. But it’s dynamic now.

The online references I found talk about the update-motd package. That was indeed how it worked in 2008 or so but is no longer the case in 2021. I don’t even have it installed on my Ubuntu 20.04 system.

The newer Debian docs seem to be correct. The motd is being created and shown by pam-motd. PAM is the code that defines what all happens when a user logs into a Linux system. It has this configuration:

# Print the message of the day upon successful login.
# This includes a dynamically generated part from /run/motd.dynamic
# and a static (admin-editable) part from /etc/motd.
session    optional     pam_motd.so  motd=/run/motd.dynamic
session    optional     pam_motd.so noupdate

That first line means “run the scripts in /etc/update-motd.d and write their output to the file /run/motd.dynamic. Then display the output.” The second line means “don’t run anything, just display the contents of the default file /etc/motd.”

The actual messages come from scripts in /etc/update-motd.d. You can use run-parts to run them all dynamically. This most fork like 100 processes every login shell, what a waste! But it’s sure flexible. 10-help-text is the one that prints the URLs advertising Ubuntu stuff. I removed mine.

There are some other motd things too.

sshd has an option for PRINTMOTD. The docs say it only prints /etc/motd.

systemd does not have any MOTD code, surprisingly. Ubuntu does have a systemd timer configured to run 50-motd-news a couple of times a day.

login has a couple of options for .hushlogin files to suppress the MOTD.

The ubuntu package show-motd drops a script in /etc/profile.d which will display the MOTD once a day. It does this using update-motd, not PAM, but it seems to work the same way with the /etc/update-motd.d scripts. It has various triggers to stop it from printing the MOTD:

  • .hushlogin
  • a MOTD_SHOWN environment variable (PAM sets this.)
  • Timestamp on $HOME/.motd_shown: only show if it’s the first shell after 00:00
  • If the shell is not interactive. (It does not check if it’s a login shell.)

WSL doesn’t run login or PAM for shells so some of this stuff doesn’t happen in the WSL context. But WSL seems to install show-motd by default and so you get a MOTD once a day.

Automatic irrigation

I’ve got a bunch of automatic irrigation in my lawn, all Irritrol equipment. (What a brand name!) Some notes on how it works. This little video is very good.

The valve is a diaphragm ordinarily held in place by finely balanced water pressure. There’s a solenoid as part of the system that ordinarily is in a closed position so water can’t flow out a dump port. When the solenoid gets electricity it raises that stopper so a small amount of water is released. That lowers the pressure on the diaphragm, allowing the main water flow to go down the pipe. I think a spring is involved somewhere to hold the solenoid valve closed.

The key thing here is that the solenoid isn’t doing a lot of work. It’s moving a small part that controls a small water flow; it’s not turning a big valve or somehow manipulating full water pressure.

The solenoids are controlled via a 24V AC wired signal coming from an irrigation computer. When the computer turns the water on it supplies continuous power at about 0.5 amps. That’s a lot of power! But I’m not sure how many watts it really ends up being; I’m finding things that suggest it’s about 2W, not the 12W I’d naively assumed.

The nice thing is if the power is cut the valve automatically closes, there’s no need to energize it to get it to close. Also the 24VAC solenoids are standard; any ordinary irrigation controller can control any brand’s solenoids and valves.

The wires are usually direct buried in your lawn. You know, right under all the shovels and other tools. Hopefully they’re buried deep enough that they aren’t broken but in practice they get broken frequently. Guess why I’m reading about this? (Although in my case I think it’s a gopher, not a shovel.) PVC conduit is about $50-$100 for 100 feet, so next time I run a wire I might splurge for some protection. Irrigation wire tends to be fairly hardy plastic but not metal armored. Mine has 5 strands; 4 wires for signal, one common neutral. I believe these systems are grounded through literal ground.

Irrigation wiring is often a mess. I paid someone to trace a lot of wires. We found one run goes maybe 800 feet through four splices. That we found, those four were in boxes.

Unfortunately irrigation valves require power, which in practice means a wire. There are very few consumer solar controller options and what I could find is expensive; $100 or more for a single valve. There are multivalve solar controllers but they seem hard to find and priced for commercial use.

Battery powered irrigation controllers are more readily available. But two experts I talked to don’t like them. Partly because just being outdoors means they aren’t going to last forever. Also the realities of batteries. One guy suggested you need to change the batteries at least twice a year. And if you forget and don’t notice, your plants die. Also the battery powered controllers require special 9V solenoids.

TIL about OpenSprinkler, a modern open source sprinkler controller with a web interface. It looks to be a serious supported product, too.

Time zones that are UTC

Sometimes it’s useful to set a clock to UTC. Say, your camera when you’re travelling; better to label all the photos with one consistent time IMHO. Some consumer UIs don’t let you choose UTC or GMT since they aren’t attached directly to a geolocation. In that case I always set the clock to Atlantic/Reykjavik time; they haven’t done daylight savings since 1967.

The full list of UTC-like timezones are Africa/Abidjan, Africa/Bissau, Africa/Monrovia, Africa/Sao_Tome, Atlantic/Reykjavik, and America/Danmarkshavn. (Not clear many places would recognize that last one, it’s pretty obscure.) I found these by searching the database for ^\s+0:00\s+-

The timezone database is a thing of beauty as Jon Udell so nicely articulated. Sadly the text contents of all the fascinating comments aren’t easily available as a web page; I should do a project to make a static site of them.