Do I need to run multiple log shipper for multiple sinks?

1

I want to send logs to logtail and newrelic, do I need to run multiple log shippers or can one log shipper send to both?

2

A single shipper should be able to ship to multiple destinations — the log shipper uses Vector under the hood, so you should be able to modify this file fly-log-shipper/vector.toml at main · superfly/fly-log-shipper · GitHub to add all the destinations you need, based on the Vector docs: Configuring Vector | Vector documentation

3

It doesn’t really work out of the box because every template toml file in fly-log-shipper/vector-configs/sinks at main · superfly/fly-log-shipper · GitHub DOES not end in a new line at the end.

So when multiple are concatenated into the final config toml, it’s messed up.

Please add a new line at end of every sink template file.

1 Like
4

Another question, when I ship logs from all my apps over to logtail, they all get jumbled together.

Is there a way for fly log shipper to append the “application name” and “app instance id” to the beginning of every log message? so that on logtail side, I can parse things out into different apps?

Or do I have to run multiple fly log shippers, one for each fly app’s log stream. (right now its just listening to *)

5

Nvm, i realized that the fly log shipper does send the right json data to Humio, Logtail just kind of sucks at parsing the messages.

1 Like
6

HI,

Vector is quite powerful in terms of doing stuff to the logs before they’re shipped out.
They’re also working on Vector Remap Language now to increase what’s possible in this interim stage.

This link may not go to the exact thing you’re looking for but just generally to the Vector docs → VRL example reference | Vector documentation

More than learning the tools themselves, it’s understanding the various documentation approaches these tools/companies take is what drains me lol.
So much variety!