My app experienced a spike of HTTP 422 errors this morning causing it to crash. After I restarted the machines, the app worked as expected. When I checked Grafana to see what caused the 422 errors, there was no logging from before the restart. Now I’m unable to troubleshoot what may have caused the app to crash.
Is this expected behavior, or can the logs for that instance be viewed elsewhere?
Hi… fly logs -a app-name should give you the most recent 100 entries, in principle.
(However, that will tend to be missing the same time slots the Grafana view lacks, since they’re based on the same persistent store, as I understand it.)
There is also typically a delay in log entries reaching the historical store. I.e., often, you can see them in the live NATS stream but not in the output of a freshly started fly logs invocation yet.
Overall, gaps are not really unusual in the retrospective (i.e., historical) log entries. The following thread has several reports from a few days ago:
It seems unlikely that the restart itself caused the gap that you saw, though. I tried a fly app restart on a test app here and it didn’t lose anything. (Although it did take a minute or so for the new boot to appear in fly logs --no-tail.)