We would like to monitor and alert on application restarts (into a warning slack channel that is less than critical).
How would we do this? If we could find out within the application via an environment variable the number of restarts then we could send that telemetry data to our telemetry system (Honeycomb).
However it seems Fly doesn’t currently expose such an environment variable.
Alternatively and maybe better, if Fly exposed a metric about this we could alert on in Grafana consuming those metrics, that would be great too.
I don’t think we have anything baked in to make this easy. We’re planning to add notifications for machine events, but aren’t working on it yet.
The simplest thing to here might be to have your app send a message when it boots. You can’t exactly detect restarts, but you can derive that from bootups.
A suggestion: collect the fly logs of your app and send it to a log analytics platform and set that up to alert on reboots (certain sequence of log lines)
@kurt IIUC this info is already available in the Fly UI. Wouldn’t it be trivial to expose an environment variable to apps at runtime that lets them know the restart number? On the first ever boot for that deployment it would be 0. Easy?