fly-log-shipper: staging vs. prod & then hiding individual server logs

Questions / Help
1

Hi everyone,

We just got fly-log-shipper working. Great product! I have two questions:

  1. It seems like all the logs in our org are going to the same instance. We have our staging environment set up in the same org, which I imagine will also funnel those logs.

Is there a way to choose which logs we want to go where?

  1. It seems like we’re still seeing logs when I run flyctl logs -a myserver. Is there a way to avoid stdout and have those logs strictly go to our log-shipper instance?

IIRC, logs slow down servers or something along those lines?

Thanks!

EDIT: It seems like we can use NATS subject-based messaging for this, but there doesn’t seem to be regex support, just the wildcard *?

We’re considering moving our staging environment to a different region as a workaround for now.

2

I solved this on the receiver side. I’m shipping my logs to LogDNA. In my Vector config for LogDNA, I add the Fly instance name. Then in my LogDNA config, I discard all of the extra logs and create filters to get what I want.

When I looked into the NATS subject-based messaging, it didn’t fully support my needs because of the lack of Regexp.

Here is my LogDNA sink config:

[sinks.logdna]
  # General
  type = "logdna" # required
  inputs = ["log_json"] # required
  api_key = "${LOGDNA_API_KEY}" # required
  hostname = "{{fly.app.instance}}"
3 Likes