- Shipped
- September 23, 2026 at 9:09 AM UTC
- Author
- Kamo
- Commit
- bc8eb06
The build had never been run. Running it found one bug per attempt, and the first three would each have produced a plausible-looking image with something missing. 1. packages.txt says "blank lines and # comments are stripped" and nothing stripped them: `tr '\n' ','` handed apt every comment line as a package name. apt answered with forty "Unable to locate package" lines quoting English prose. 2. `qemu-img resize` grows the qcow2 container and leaves the partition table and the root filesystem alone — about 2.5 GB in Canonical's image. A cloud image normally grows itself on first boot through cloud-init growpart; virt-customize never boots it, so 914 MB of desktop ran out of space midway through dpkg --unpack. virt-resize --expand now moves the partition before anything is installed into it. 3. `firefox` on Ubuntu 24.04 is a transitional deb whose only job is to install the snap. With no snapd in an image build its postinst prints "System doesn't have a working snapd, skipping" and exits ZERO — so apt succeeded, the build succeeded, and the image every tenant clones from had no browser on it. Mozilla's apt repository is added instead, with the signing key's fingerprint verified rather than trusted because it arrived over HTTPS. 4. `virt-customize --run` executes a script with /bin/sh whatever its shebang says. On Ubuntu that is dash, and all four guest scripts open with `set -euo pipefail`, so every one died on "Illegal option -o pipefail". They are copied in and run with an explicit bash now, matching what the agent installer already did. 5. `--copy-in src:dest` needs dest to be an existing DIRECTORY and copies src into it. `agent:/opt/kamo-hc-agent-src` was naming the result rather than a destination and failed with "target is not a directory" — only reachable once the build got far enough to install the agent at all. And one that was hiding rather than failing: install-agent.sh ends by restarting the unit and waiting for /healthz, neither of which can work in an image build because virt-customize edits a filesystem and never boots it. The build therefore called it with `|| true`, which also swallowed any real failure in the install steps — an image could ship with no agent and say nothing, and a computer with no agent never sleeps, never takes its organization's branding and never syncs a password. It now tests /run/systemd/system and does an honest offline enable, so the caller can fail loudly. The wrapper itself had one too: it followed `kubectl logs -f` the moment the Job object existed, which returns instantly against a pod still pulling its image, then read .status.succeeded on a job ten seconds old and reported failure. That is a worse kind of wrong than a broken build, because it tells you to delete a claim that is being written correctly. Built and verified: noble-20260911, 1.7 GB, sha256 ****************