Output of 'fly logs' different on dev vs Fly instance

I have a Fly app deployed in ewr that manages other Fly apps in the same organization.

I’ve noticed that running fly logs -a ${APP_NAME} on my local machine gives me the log output for all regions. However, running the same exact command on the Fly app only gives the output for a single region (ewr in this case).

Local output:

2022-09-08T13:39:43Z runner[f99eff15] hkg [info]Unpacking image
2022-09-08T13:39:44Z runner[f99eff15] hkg [info]Preparing kernel init
2022-09-08T13:39:45Z runner[f99eff15] hkg [info]Setting up volume 'storeData'
2022-09-08T13:39:45Z runner[f99eff15] hkg [info]Opening encrypted volume
2022-09-08T13:39:45Z runner[f99eff15] hkg [info]Configuring firecracker
2022-09-08T13:39:45Z runner[f99eff15] hkg [info]Starting virtual machine
2022-09-08T13:39:45Z app[f99eff15] hkg [info]Starting init (commit: 249766e)...
2022-09-08T13:39:45Z app[f99eff15] hkg [info]Mounting /dev/vdc at /data w/ uid: 0, gid: 0 and chmod 0755
2022-09-08T13:39:45Z app[f99eff15] hkg [info]Preparing to run: `docker-entrypoint.sh node server/dist/index.mjs` as root
2022-09-08T13:39:45Z app[f99eff15] hkg [info]2022/09/08 13:39:45 listening on [fdaa:0:65b2:a7b:7f07:1:d144:2]:22 (DNS: [fdaa::3]:53)
2022-09-08T13:39:46Z runner[6d7b3925] sin [info]Unpacking image
2022-09-08T13:39:47Z app[f99eff15] hkg [info]Connected to coordinator at coordinator.hathora.com with appId 9d3ebf4b605a82bbc49f9a5fa941e94f366aff841fefadd97350d5df9058962c and storeId 659c0730-106f-4d21-a74e-f34441697100
2022-09-08T13:39:47Z runner[6d7b3925] sin [info]Preparing kernel init
2022-09-08T13:39:47Z runner[6d7b3925] sin [info]Setting up volume 'storeData'
2022-09-08T13:39:47Z runner[6d7b3925] sin [info]Opening encrypted volume
2022-09-08T13:39:47Z runner[6d7b3925] sin [info]Configuring firecracker
2022-09-08T13:39:47Z runner[6d7b3925] sin [info]Starting virtual machine
2022-09-08T13:39:47Z app[6d7b3925] sin [info]Starting init (commit: 249766e)...
2022-09-08T13:39:47Z app[6d7b3925] sin [info]Mounting /dev/vdc at /data w/ uid: 0, gid: 0 and chmod 0755
2022-09-08T13:39:47Z app[6d7b3925] sin [info]Preparing to run: `docker-entrypoint.sh node server/dist/index.mjs` as root
2022-09-08T13:39:47Z app[6d7b3925] sin [info]2022/09/08 13:39:47 listening on [fdaa:0:65b2:a7b:ead:1:d141:2]:22 (DNS: [fdaa::3]:53)
2022-09-08T13:39:49Z app[6d7b3925] sin [info]Connected to coordinator at coordinator.hathora.com with appId 9d3ebf4b605a82bbc49f9a5fa941e94f366aff841fefadd97350d5df9058962c and storeId 7c53d31f-978d-446d-b25a-f96643f2af71
2022-09-08T13:39:53Z runner[ef43c8bb] maa [info]Unpacking image
2022-09-08T13:39:54Z runner[ef43c8bb] maa [info]Preparing kernel init
2022-09-08T13:39:54Z runner[ef43c8bb] maa [info]Setting up volume 'storeData'
2022-09-08T13:39:54Z runner[ef43c8bb] maa [info]Opening encrypted volume
2022-09-08T13:39:54Z runner[ef43c8bb] maa [info]Configuring firecracker
2022-09-08T13:39:55Z runner[ef43c8bb] maa [info]Starting virtual machine
2022-09-08T13:39:55Z app[ef43c8bb] maa [info]Starting init (commit: 249766e)...
2022-09-08T13:39:55Z app[ef43c8bb] maa [info]Mounting /dev/vdc at /data w/ uid: 0, gid: 0 and chmod 0755
2022-09-08T13:39:55Z app[ef43c8bb] maa [info]Preparing to run: `docker-entrypoint.sh node server/dist/index.mjs` as root
2022-09-08T13:39:55Z app[ef43c8bb] maa [info]2022/09/08 13:39:55 listening on [fdaa:0:65b2:a7b:14bf:1:d13a:2]:22 (DNS: [fdaa::3]:53)
2022-09-08T13:39:57Z app[ef43c8bb] maa [info]Connected to coordinator at coordinator.hathora.com with appId 9d3ebf4b605a82bbc49f9a5fa941e94f366aff841fefadd97350d5df9058962c and storeId 6b49d986-6e28-4eda-8d65-2c21c4166d87

Output from Fly app (via SSH)

2022-09-08T12:57:18Z runner[c7549f92] ewr [info]Starting instance
2022-09-08T12:57:21Z runner[c7549f92] ewr [info]Configuring virtual machine
2022-09-08T12:57:21Z runner[c7549f92] ewr [info]Pulling container image
2022-09-08T12:57:21Z runner[c7549f92] ewr [info]Unpacking image
2022-09-08T12:57:22Z runner[c7549f92] ewr [info]Preparing kernel init
2022-09-08T12:57:22Z runner[c7549f92] ewr [info]Setting up volume 'storeData'
2022-09-08T12:57:22Z runner[c7549f92] ewr [info]Opening encrypted volume
2022-09-08T12:57:23Z runner[c7549f92] ewr [info]Configuring firecracker
2022-09-08T12:57:23Z runner[c7549f92] ewr [info]Starting virtual machine
2022-09-08T12:57:23Z app[c7549f92] ewr [info]Starting init (commit: 249766e)...
2022-09-08T12:57:23Z app[c7549f92] ewr [info]Mounting /dev/vdc at /data w/ uid: 0, gid: 0 and chmod 0755
2022-09-08T12:57:23Z app[c7549f92] ewr [info]Preparing to run: `docker-entrypoint.sh node server/dist/index.mjs` as root
2022-09-08T12:57:23Z app[c7549f92] ewr [info]2022/09/08 12:57:23 listening on [fdaa:0:65b2:a7b:ab2:1:d137:2]:22 (DNS: [fdaa::3]:53)
2022-09-08T12:57:25Z app[c7549f92] ewr [info]Connected to coordinator at coordinator.hathora.com with appId 9d3ebf4b605a82bbc49f9a5fa941e94f366aff841fefadd97350d5df9058962c and storeId d27d9469-b5c2-40b9-b6ac-02d31f94e245
2022-09-08T13:39:28Z runner[c7549f92] ewr [info]Shutting down virtual machine
2022-09-08T13:39:28Z app[c7549f92] ewr [info]Sending signal SIGINT to main child process w/ PID 524
2022-09-08T13:39:29Z app[c7549f92] ewr [info]Starting clean up.
2022-09-08T13:39:29Z app[c7549f92] ewr [info]Umounting /dev/vdc from /data
2022-09-08T13:39:33Z runner[97ad1fc4] ewr [info]Starting instance
2022-09-08T13:39:35Z runner[97ad1fc4] ewr [info]Configuring virtual machine
2022-09-08T13:39:35Z runner[97ad1fc4] ewr [info]Pulling container image
2022-09-08T13:39:36Z runner[97ad1fc4] ewr [info]Unpacking image
2022-09-08T13:39:37Z runner[97ad1fc4] ewr [info]Preparing kernel init
2022-09-08T13:39:37Z runner[97ad1fc4] ewr [info]Setting up volume 'storeData'
2022-09-08T13:39:37Z runner[97ad1fc4] ewr [info]Opening encrypted volume
2022-09-08T13:39:37Z runner[97ad1fc4] ewr [info]Configuring firecracker
2022-09-08T13:39:37Z runner[97ad1fc4] ewr [info]Starting virtual machine
2022-09-08T13:39:38Z app[97ad1fc4] ewr [info]Starting init (commit: 249766e)...
2022-09-08T13:39:38Z app[97ad1fc4] ewr [info]Mounting /dev/vdc at /data w/ uid: 0, gid: 0 and chmod 0755
2022-09-08T13:39:38Z app[97ad1fc4] ewr [info]Preparing to run: `docker-entrypoint.sh node server/dist/index.mjs` as root
2022-09-08T13:39:38Z app[97ad1fc4] ewr [info]2022/09/08 13:39:38 listening on [fdaa:0:65b2:a7b:ab2:1:d137:2]:22 (DNS: [fdaa::3]:53)
2022-09-08T13:39:39Z app[97ad1fc4] ewr [info]Connected to coordinator at coordinator.hathora.com with appId 9d3ebf4b605a82bbc49f9a5fa941e94f366aff841fefadd97350d5df9058962c and storeId f92cbcfd-50bd-4cae-888e-7e25329f55ac

What is causing the output to be different when it’s the same exact command that’s executing?

Hi @dsiddharth,

I’m thinking its because you have the FLY_REGION environment variable set. Try unset FLY_REGION and see if you start seeing all regions.

1 Like

Awesome, that was it – thanks!

1 Like