I was reading the “Shipping Logs” (Shipping Logs · Fly) blog post and decided to try out the logtail.com service. I followed the instructions to setup fly-log-shipper and it seems to work great with JSON-formatted logs out of the box.
However, some of my services output NGINX logs in a combined format, which logtail.com ingests as is, without further destructuring of the log message. I looked through fly-log-shipper code. I’m not very familiar with NATS and Vector, but as far as I can understand fly-log-shipper expects logs from apps in JSON format and otherwise treats them as plain text.
NGINX log format seems pretty common to me, is there an easy way to process it via fly-log-shipper somehow? I suppose I can make NGINX spew out JSON logs instead, but I’m wondering whether I can do that centrally on the fly-log-shipper.
The other way to go about this is to customize the Vector configuration, which would mean forking the log shipper repository and editing the Vector toml config. This is likely a bunch more work than the previous option, mostly because it involves googling Vector configs and it seems like there’s a bunch of opinions/ways to go about that. (for example, figuring out wtf is going on here).
I ended up going the route #1, thankfully it turned out to be pretty simple: I use Kong and all I needed to do is to add new env var (KONG_NGINX_HTTP_LOG_FORMAT="json_format '{…}'") and update a couple of other env vars to pass the new log format name to them. This still took me half a day to figure out though because documentation is not great.
I tried looking at route #2 and yeah, you’re right that it looks like way more work so I bailed out.
).
Based on some research, I think getting Nginx to output JSON logs is your best bet. Here are two examples of how to do that:
The other way to go about this is to customize the Vector configuration, which would mean forking the log shipper repository and