fly-log-shipper - NATS Connect Error - Authorization Violation

1

I am trying to set up log shipper for an app that runs iin an org other than personal. The sink is ClodWatch. I set the following secrets:

fly secrets set \
  ORG="<name of the org under which the main app runs>" \
  ACCESS_TOKEN="<output of fly tokens create readonly personal>" \
  SUBJECT='logs.<name of the main app>.>' \
  AWS_ACCESS_KEY_ID="..." \
  AWS_SECRET_ACCESS_KEY="..." \
  AWS_REGION="..." \
  CLOUDWATCH_LOG_GROUP_NAME="..." \
  CLOUDWATCH_ENCODING_CODEC='raw_message' \
  -a <name of the log shipper app>

In the logs, I see the following error:

2026-09-07T14:37:53.305847Z ERROR vector::topology: Configuration error. error=Source "nats": NATS Connect Error: unexpected line while connecting: Err("Authorization Violation")

Below is fly.toml as can be seen on the Web UI:

app = "fly-log-shipper-tr-api-prod"
primary_region = "fra"

[build]
image = "flyio/log-shipper:latest"

[http_service]
auto_start_machines = true
auto_stop_machines = true
force_https = true
internal_port = 8_080
min_machines_running = 0
processes = [ "app" ]

[[services]]
internal_port = 8_686

[[vm]]
cpus = 1
memory = "1gb"
memory_mb = 1_024

What have I done wrong, and how do I fix it?

Thanks a lot in advance!

3

Hm… The token needs to be created for ORG, though, not for personal (which is an organization name in this context).

Try fly tokens create readonly $ORG, like in the official docs.

(I.e., be sure to use the same ORG as the one specified in the secrets list.)

4

Thanks! That problem is now fixed.

Next issue is: My app’s logs are already JSON messages. So I was hoping that setting

CLOUDWATCH_ENCODING_CODEC=‘raw_message’

would cause the raw messages to be logged. I see in CloudWatch that this is not the case. I.e., my app’s JSON Log messages are embedded in another JSON log. How do achieve the desired effect?

5

CloudWatch I don’t know that well, but, from your earlier thread, it looks like support for raw_message might not be fully tested.

I’d suggest marking the current thread as Solved and then creating a new top-level thread for that separate (and possibly much thornier) problem. The Questions / Help category is the best place for such things.

(If you link to the March thread from the new one, then casual readers will have more context. This will improve the odds that someone will happen to see the solution while glancing through, etc.)

6

Thanks for the comment! I will follow it. One quick note: It would be good to update the docs in GitHub, which is the instructions I had originally followed.

7

I see that starting on the 8th of September, the messages are ingested in raw format.