Sprites on 0.0.2-beta.5 reboot repeatedly without a request and recover their journals each boot

The Sprites in our org run the same tenant container. The ones on runtime 0.0.2-beta.5 reboot far more often than the one on 0.0.1-rc48, and every boot recovers an ext4 journal.

  • expansion-tenant-t005 (0.0.2-beta.5, created 2026-10-10 17:02Z): 11 boots in 15 hours, read from a per-minute log on the Sprite’s own root.
  • expansion-tenant-t007 (0.0.2-beta.5, created 2026-10-11 05:22Z): 10 boots in 2.5 hours.
  • expansion-tenant-t001 (0.0.1-rc48, created 2026-09-24): 5 boots in 9 days.

None of these reboots was requested through the API or the dashboard. Memory before them sat near 7.5 GB available with swap unused, and the load was near zero.

Each boot’s kernel log carries EXT4-fs (vdb): recovery complete, and on the beta.5 Sprites also EXT4-fs (nbd0): recovery complete. On 2026-10-10 21:25Z t005 logged block nbd0: Dead connection, failed to find a fallback followed by 804 write I/O errors on nbd0, and on 2026-10-11 06:54Z ext4_end_bio: I/O error 10 writing to inode ... potential data loss! on nbd0, about 30 seconds before a reboot. Files written just before those events came back empty.

Nothing on the Sprites drives these reboots. On 2026-10-11, expansion-tenant-t005 and expansion-tenant-t007 rebooted 16 seconds apart (08:50:35Z and 08:50:51Z), both idle. t007’s last memory sample before its boot read 7,999 MB available, swap unused, load 0.01.

They also land under work in progress. On 2026-10-12 between 10:30Z and 12:00Z, expansion-tenant-t005 booted 12 times in 85 minutes, while t001 booted once. Several of those lives ended within seconds of an exec that started work, and the work died with the machine. On expansion-tenant-t009, a reboot at 2026-10-11 16:28:18Z landed while an inbound stream connection through the Sprite’s proxy was held open.

An exec that reaches a frozen Sprite gets a cold boot, and the exec itself dies with shell exit 137 (t007, 2026-10-11 about 09:08Z; boot ec383c80 at 09:10:31Z).

The Docker disk can be re-attached on a new device under a running Sprite with no reboot. On t007, boot ec383c80, the kernel logged EXT4-fs (nbd1): recovery complete at 09:19:20Z, 530 seconds into a boot where the same disk had been nbd0. An exec at that moment died with 137 and the container restarted.

On 2026-10-12, after three more boots that morning, t005’s nbd0 log for 12:05Z to 12:41Z carried 2 dead connections, 90 write errors, 9 I/O errors, 5 potential-data-loss lines and 2 read-only remounts.

On 0.0.2-beta.5 /var/lib/docker sits on a network block device (nbd0); on rc48 it is vdb. The reboots and the dead nbd connections are on the beta.5 Sprites.

We need to know what reboots these Sprites, whether a Sprite can be created on, or held at, a runtime that does not, whether the nbd disk’s dead-connection behaviour is known, and whether the nbd device behind /var/lib/docker can be swapped under a running Sprite.

@epiphanylab I investigated the machines you’re having issues with and it looks like the swap on /var/lib/docker/swapfilewas causing issues with our storage swap. We have a fix coming shortly.

Thanks, Ben. That matches what we see, and the file is ours. A boot service on each Sprite creates a 4 GB swap file at /var/lib/docker/swapfile and turns it on, because on 0.0.1-rc48 a Sprite with no swap wedged when memory ran out. On 0.0.2-beta.5 that path sits on the network block device, so we were swapping onto nbd0.

Two answers would help while the fix is on its way:

  1. Should we turn our swap file off on the beta.5 Sprites now, or leave them as they are so the fix meets the same state that broke?
  2. Does beta.5 give a Sprite swap of its own? On 2026-10-11 we read a 4 GB /swapfile on one. Today expansion-tenant-t005 lists only ours in /proc/swaps.

We can remove ours whenever you say.