Fly + Datadog Vector generated configuration broken

Southclaws · November 30, 2022, 2:24pm

Hey! Loving fly so much we’re moving prod to it! But there’s one last bit to solve: Datadog integration.

I’ve spun up the fly-log-shipper but it seems to be generating an invalid Vector config when the Datadog site is set to the EU site.

Configuration error. error=duplicate key: `site` for key `sinks.datadog` at line 35 column 1

I think the issue is here:

github.com

superfly/fly-log-shipper/blob/main/start-fly-log-transporter.sh#L6


      
          #!/bin/bash
          set -euo pipefail
          
          template() { eval $'cat <<_EOF\n'"$(awk '1;END{print"_EOF"}')"; }
          sponge() { cat <<<"$(cat)" >"$1"; }
          filter() { for i in "$@"; do template <"$i" | sponge "$i" || rm "$i"; done; }
          filter /etc/vector/sinks/*.toml 2>&-
          echo 'Configured sinks:'
          find /etc/vector/sinks -type f -exec basename -s '.toml' {} \;
          
          exec vector -c /etc/vector/vector.toml -C /etc/vector/sinks

It’s duplicating the site key in the output config. Not sure what the shell script is doing, I can never remember the weird syntax for if-statements.

I’ll probably roll my own config for now but just wanted to throw this out there in case there’s anything missing from the docs or I’ve done something wrong.

Config is all set, DATADOG_SITE is set, it’s just somehow ending up with two site entries in the generated toml.

dusty · November 30, 2022, 4:45pm

Nice find. I’ll take care of this bug.

dusty · November 30, 2022, 5:08pm

@Southclaws give it a try now, I’ve merged https://github.com/superfly/fly-log-shipper/pull/33

Southclaws · December 13, 2022, 7:05pm

Perfect, works fine, thanks!

Topic		Replies	Views
Issues connecting `fly-log-shipper` to Datadog	6	539	September 23, 2022
fly-log-shipper failing in fly app Questions / Help logs	1	593	February 24, 2023
Fly + Datadog vector error Questions / Help logs	8	239	July 4, 2024
Fly log shipper to datadog sets source to "undefined" Questions / Help	3	560	January 11, 2023
fly-log-shipper error on launch/deploy - nats: Authorization Violation Questions / Help logs	1	543	July 25, 2022

Fly + Datadog Vector generated configuration broken

Related topics