I’ve put together a simple app to store Fly logs on an attached volume. This uses Vector to read the Fly NATS endpoint and Loki as the datastore. It offers configurable log retention and should help with troubleshooting app issues ( think OOM alerts ). I know many of you have already come up with your own log solution at Fly but this might help others looking for a quick solution. One thing to note is that this will not backfill any prior logs.
The README guides you through a quick setup and how to query but further customization is possible. I believe the default settings should work for most but adjustments may be required for heavy log flows.
Let me know if you run into issues or have any additional thoughts.
Will this do anything about the main problem I have with Fly.io logging, which is the fact that it doesn’t log everything? There’s a big delay between when a message is printed to the console, and most are not printed, for some reason.
I run a Twitch bot which connects to Twitch using the twitchio library. Most print statements aren’t in the fly logs logs/monitoring on the website. If I send a message to the bot, it should print to the console, which it does when running locally.