Streaming app logs from one Fly app to another for analysis

Hello! I am running a personal website on Fly.io using Caddy to serve a generated static site. I wanted to do a bit of analysis over the server logs, using something like Goatcounter or Goaccess.

My initial attempt with Goatcounter ran into issues: I wanted to self-host Goatcounter, so I had to run it as its own app. I attempted to use Caddy’s net log handler to write to goatcounter-app.internal:9090, while adding ncat -l 9090 > /data/caddy.log & to the launch script of my Goatcounter instance to try and read that stream to a file. I could see logs from Caddy on stdout, but none showed up in the caddy.log file on the Goatcounter host, so I imagine I must have done something incorrect there.

My next plan is to try running Goaccess locally on the same VM as my Caddy instance, so that it can inspect a logfile directly.

I am curious if anyone has attempted anything similar, and whether or not there are any recommendations for sharing log data between Fly apps.

1 Like

You can subscribe to logs from another app in the same organization using NATs: Fly Logs over NATS

@dusty built an example app that subscribes to NATs and stores logs with Loki, which sounds pretty close to what you’re trying: GitHub - fly-apps/fly-log-local: Store Fly app logs locally.

3 Likes

Neat! Too bad we already impl a custom log-sink with AppRunner and Firehose + S3.

Btw, there needs to be “one-click” deploy button for (such uber useful) fly-apps!


edit: Just read the source – a vector.toml, a dockerfile, a loki.yml and that’s pretty much it; so succinct. The pieces just seem to fit together like lego blocks!

2 Likes