Searchable application logs in Grafana

My worry is that in your original post you said the log retention is 30 days and some of my logs are now outside that window. I don’t want them to be deleted as there is some valuable data in there.

If you could download all the logs from my application to a txt file then send it to me that would be ideal. Alternatively if you could extend the log retention duration until we can figure out how to download them that would also be all good.

Thanks for working with me on this!

Unfortunately there’s no easy way to download logs or extend the retention. This log service is provided as a convenience. If you need longer term storage, you can send your logs to a service of your choice using the Fly Log Shipper.

Meanwhile, we might look into a way to retrieve paginated results for a query, but there’s no ETA for that.

Dam, I made the site tenthousandpixels.com (pretty much an r/place clone). It ended up getting a bunch of people coming to the website (20k+) and getting over 4 million pixels placed.

I ended up just using the provided logging solution to keep track of all the pixels that had been placed by people as it seemed like an easy approach at the time.

I really want to generate a big timelapse of all the pixels placed to cap off the project. Unfortunately, to do that I need the downloaded logs.

Are you sure there’s nothing that can be done to get those downloaded logs to me? I’d be happy to pay for one of the premium plans, or anything else that needs to be done. I’ll also make sure to ship the logs in my next project. That’s my mistake.

If you can upgrade to paid plan, you’ll be able to submit a support ticket where we can help you further on this.

Hey y’all, so I had a ton of trouble the other day searching my logs. Sure, parsing json is cool and all, but I would be just fine with a text search on a pattern. I can’t seem to find something that works. I’m looking for any log line that contains both “7115” and “monthly”

*7115*monthly* -> Wildcard iquery contains wildcard in non final position (whatever that means)
7115 AND monthly -> Returned lots of lines that don't match at all (`[ActiveJob] [AccountStatsUpdateJob] [487463eb-5f9e-4289-a1bc-986896a285f6] Performed AccountStatsUpdateJob (Job ID: 487463eb-5f9e-4289-a1bc-986896a285f6) from Sidekiq(user_metrics) in 15321.19ms`)
7115, monthly -> somehow this worked! even though the actual matching string contains double quotes  - '7115, "monthly"'

Can we beef up the docs here a bit? Btw, could someone show me how they typically move forward from here? I now have a single line that matches, and I’ve gotta use the weird context thing, which pops up in this awkward modal but preserves the original search text, so I have to remove some of the search criteria to look around that line at the given time.

I mean, my normal process here is, find something relevant to pull on in the logs, then expand the search a bit in terms of time to see what happened in the leadup to that log line, or what happened after. I must be missing something?

1 Like