Fly log shipper to datadog sets source to "undefined"

Questions / Help
1

Yo, so we’ve got the Vector config running from the fly-log-shipper repo. It’s working great except the source attribute in Datadog is “undefined”

image

This results in every single app running in our Fly.io org being categorised under this source, instead of the app name which would be ideal.

It would be great to set this to something in the Vector config that would use the Fly app name but I can’t find this anywhere because the word “source” seems quite overloaded in the context of log aggregation!

Would appreciate some guidance on this, thanks!

2

See: Vector Remap Language (VRL)

3

Progress…

I’ve tried configuring Vector to transform the Fly.io app name into service which according to Vector+Datadog’s documentation, should result in the desired behaviour:

[transforms.log_json]
    inputs = ["nats"]
    source = '''
. = parse_json!(.message)
.service = string!(.fly.app.name)
'''
    type = "remap"

However this didn’t work BUT ddtags does:

[transforms.log_json]
    inputs = ["nats"]
    source = '''
. = parse_json!(.message)
.ddsource = string!(.fly.app.name)
'''
    type = "remap"

Now it’s working well in the DD interface:

image

It seems this in the Vector docs isn’t accurate:

image
image920×119 6.79 KB

As these just appeared as regular JSON fields:

image
image

2 Likes
4

What if you remap it to .message.service instead?

Anyway, maybe it’s easier to remap on Datadog’s side.

Option 1 (Preprocessing)

  • Preprocessing for JSON logs
    • Service attributes – prepend fly.app.name

Option 2 (Custom Pipeline)

  • Service Remapper, use fly.app.name
1 Like