I’m using fly-log-shipper to ship logs to Datadog. Seems to be working quite nicely, especially after I figured out how to get my Spring Boot app to output JSON logs; Datadog now detects the log level correctly.
I want to ship logs from my production and staging apps. So, at the moment, I have two fly-log-shipper apps. One has subject
logs.<production-app-name>.*.* and the other has subject
Is there a way to set the subject, or some other way, so that I can achieve this with only one fly-log-shipper app? I would particularly like to exclude logs from the free builder, which is a little noisy.
If you don’t set the subject, logs from every app in the organization will ship by default. Could you exclude specific log patterns in Datadog?
I could do that. Datadog (and other services?) charge based on how much data they receive. So that’s why I’ve chosen to filter at source.
Some services (like LogDNA/Mezmo) will discount logs that you explicitly filter out. Is there a different price for logs that don’t get indexed?
If I understand you correctly, this vector filter should work. Unset your $SUBJECT and modify 18-23 with something like:
type = "remap"
inputs = ["nats"]
source = '''
. = parse_json!(.message)
type = "filter"
inputs = ["remap"]
condition = '.fly.app.name == "production-app-name" || .fly.app.name == "staging-app-name"'
Yeah, I’d like to know the answer to this. It wasn’t clear in the docs I read so I have contacted their support. Thanks!
This was exactly what I was looking for. I’ve just confirmed it works and I’ve also learned more about vector transforms. Thank you!