I’m having trouble deploying a change (probably an issue with my code), so I was trying to check the logs… And now my logs aren’t displaying for the two apps I checked.
For example
% flyctl logs --app nikola-sharder
Update available 0.0.134 -> 0.0.137
… then nothing
Friends at Fly, any ideas what might be going on here? Thanks!
p.s. just upgraded my client to 0.0.137 and still have the same issue (minus the upgrade message)
We are arm wrestling with our log cluster, should be back soon!
good luck!
Should be back now (and faster!)
Yep, I’m now back in business thanks!
Logs broken again? On 0.0.138-beta-8.
Logs should be back now.
Cheers, works now.
It seems logs are stalling again… I only see logs from 30 mins ago.
Is there any other way to download logs?
Edit:
I got everything from the last 30 mins in one shot now 
Yep, I regularly get stalled logs. Mine are again from what I can see.
[Back to the pricing discussion: I take it all back if you can get redis and logs working 100% Scrap that free tier in exchange.]
There’s almost one hour of delay now in the logs for me 
I’m not sure how you’re handling this. I think it would be better to only have the realtime logs when one is using flyctl logs and not handle all logs there. Then have a cache of files or something we could download for the past 24 hours or so.
We have a capacity problem, basically a bunch of Graylog servers that can’t keep up with processing. flyctl logs just polls the index. We’re at peak log traffic right now (we have some especially chatty apps that get busy about now).
We’re working on it, but it’s a big project that will take a while.
If you want the dirty technical details, we’re running 7 pretty beefy Graylog servers to process logs. They currently have 4 to 8 million unprocessed messages queued up, each.
Some apps are sending 10s of thousands of lines per second. We’re going to start throttling those for now. The long term plan is to allow it, but charge for sustained, high volume logs. 
Good news everyone, flyctl logs should be much quicker starting today: New Logging Infrastructure