Region: lhr · Machine: e82d10e6f3e778 · Size: shared-cpu-1x, 512MB
On 7 Oct 2026, every stdout line from this machine between 09:46:22 and 10:06:46 UTC is missing, from both fly logs and the Grafana/Loki log search.
The process was running normally during that window. Our database shows it handling webhooks and writing rows at 09:58:20 and 10:00:44 UTC, and it logs every HTTP request it serves. The gap starts mid-boot: Starting scheduled tasks… (09:46:21.984) is delivered, but the lines our app always writes a few milliseconds later (Starting account deletion check…, the credit checks, Scheduled tasks started) never appear. Logging resumes at 10:06:46 without any of the missing lines being delivered late.
Separately, Loki is missing lines that fly logs did show (the boot lines at 09:46:20 to 09:46:22 UTC).
From about 10:00 UTC the machine also ran out of CPU quota balance (throttling ~90%, steal ~0.9). We think that’s our own usage and unrelated, since the log gap started 14 minutes earlier. Fly’s VM operations were slow afterwards too: a deploy at 10:19 took 2m6s to recreate and start the machine (“Machine created and started in 2m6.136s”).
Questions:
- Was there a known log-delivery problem on host 1dad / lhr in that window?
- Is stdout delivery best-effort, and is there any way to detect dropped lines on our side?
- Does a shared-CPU machine build up quota balance while it’s stopped (auto_stop_machines = ‘stop’), or does the balance reset when it starts or is recreated by a deploy?