Fly Logs over NATS

37

@steveberryman Thanks again for the feedback. Managed to make it all work nicely. One little thing i noticed, if you run vector validate vector.toml you get a warning that data_dir = "/var/lib/vector/"doesn’t exist. Creating the dir solves that.

Here is the transform and sink for LogDNA:

[transforms.log_json]
type = "remap"
inputs = ["fly_socket"]
source = '''
    .= parse_json!(.message)
    . |= parse_regex!(.message, r'^\[(?P<app>\w+)\]\s+(?P<message>.*)$')
    del(.log)
    if ends_with(.fly.app.name, "-staging") {
        .env = "staging"
    }
    '''

[sinks.logdna]
type = "logdna"
inputs = ["log_json"]
api_key = "${LOGDNA_APIKEY}"
hostname = "{{fly.app.name}}"
default_app = "system"
tags = ["${ORG}"]

I am running an S6 multi-process setup with multiple services in a single APP. The logs for each service are prepended with the service name in between brackets. This is done in the S6 log script.

I’m applying following parsing rules for specific fields recognised by LogDNA

  • hostname: Set based on app name
  • app: Parsing out the service name from the log message prefix
  • env: Parsing out from app name based on convention (app name ends with either -staging or -production)
  • tags: Set based on the ORG name

I’m also deleting log.level as LogDNA handles this out of the box and it seems the info by fly is always info ?

Keep up the great work!

5 Likes