Fly log shipper only receives logs from some apps

1

I’ve followed the instructions to install fly-log-shipper with a Papertrail sink in an organization with ~10 apps including Elixir apps and Fly Postgres instances across a production and staging environments. I’m seeing logs for a few of the apps, but not all of them, and I can’t figure out what’s going wrong.

  • When I shell into the log shipper machine and run vector tap nats, I see the same log lines that are making it to Papertrail, and nothing from those other apps.
  • I can’t find a pattern between the apps whose logs are missing and those who are present. The phenomenon seems to be spread across the board across: DB and web instances, prod and staging, older and newer apps.
  • I’ve tried generating a new ACCESS_TOKEN and filling in some of the optional env variables that I’d originally left empty (e.g. QUEUE and SUBJECT), and don’t see any change
  • I just discovered there’s at least one app (one of our staging DBs) whose logs were being sent over until a few hours ago, a time that doesn’t appear to coincide with a release of either the log shipper or the DB app in question (I set up the log shipper yesterday and didn’t get back around to debugging it until ~1 hour ago)

Any help would be appreciated!

2

Hello,

I’m seeing the same thing on my side. On 16th f April logs stopped flowing and I could not get them back. I’m using an organization token that does not expire, and I’m receiving the logs from the log shipper app, but nothing else - and we haven’t changed anything.

Any idea what may be happening?

3

I saw an incident affecting logs on 15th, may it be related?

4

Fixed: Fly logs are not flowing through Log Shipper - #4 by oscarjt

5

Hmm… I have added a machine to my log shipper app as well as to one of the apps whose logs are missing, but I’m still seeing the same behavior. :confounded:

6

We have found the issue and will be rolling out updates over the next few hours.

7

Awesome, thanks! Looks like things are working correctly now

8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.