Fly + Datadog Vector generated configuration broken

Southclaws · November 30, 2022, 2:24pm

Hey! Loving fly so much we’re moving prod to it! But there’s one last bit to solve: Datadog integration.

I’ve spun up the fly-log-shipper but it seems to be generating an invalid Vector config when the Datadog site is set to the EU site.

Configuration error. error=duplicate key: `site` for key `sinks.datadog` at line 35 column 1

I think the issue is here:

github.com

superfly/fly-log-shipper/blob/main/start-fly-log-transporter.sh#L6


      
          #!/bin/bash
          set -euo pipefail
          
          template() { eval $'cat <<_EOF\n'"$(awk '1;END{print"_EOF"}')"; }
          sponge() { cat <<<"$(cat)" >"$1"; }
          filter() { for i in "$@"; do template <"$i" | sponge "$i" || rm "$i"; done; }
          filter /etc/vector/sinks/*.toml 2>&-
          echo 'Configured sinks:'
          find /etc/vector/sinks -type f -exec basename -s '.toml' {} \;
          
          exec vector -c /etc/vector/vector.toml -C /etc/vector/sinks

It’s duplicating the site key in the output config. Not sure what the shell script is doing, I can never remember the weird syntax for if-statements.

I’ll probably roll my own config for now but just wanted to throw this out there in case there’s anything missing from the docs or I’ve done something wrong.

Config is all set, DATADOG_SITE is set, it’s just somehow ending up with two site entries in the generated toml.

dusty · November 30, 2022, 4:45pm

Nice find. I’ll take care of this bug.

dusty · November 30, 2022, 5:08pm

@Southclaws give it a try now, I’ve merged https://github.com/superfly/fly-log-shipper/pull/33

Southclaws · December 13, 2022, 7:05pm

Perfect, works fine, thanks!

Topic		Replies	Views
Issues connecting `fly-log-shipper` to Datadog	6	462	September 23, 2022
Fly log shipper to datadog sets source to "undefined" Questions / Help	3	402	January 11, 2023
fly-log-shipper error on launch/deploy - nats: Authorization Violation Questions / Help logs	1	462	July 25, 2022
fly-log-shipper fails on LogTail: Authorization Violation Questions / Help	3	760	October 26, 2023
http sink for fly shipper logs	2	112	February 1, 2024

Fly + Datadog Vector generated configuration broken

Related Topics