lava
February 14, 2024, 11:40am
1
whenever I run flyctl
or fly
command in my terminal, I got something like this:
2024/02/14 12:36:02 traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 149.248.216.54:4317: connect: network is unreachable"
The command still executes, but at the same time this error message pops up constantly.
I’m not sure what does it mean. Having been googling around but no similar issues.
Also I’m not able to let the deployment passes health checks:
[[services.tcp_checks]]
grace_period = "10s"
interval = "10s"
restart_limit = 0
timeout = "2s"
Must be some network issue I’m facing, right? How to deal with it then? Please help, thanks!
Hello, the network issue might be different from your deployments being unable to pass health checks.
Do you see any errors in fly logs
?
I see this as well, on a network that blocks outgoing port 4317.
This is the new telemetry package trying to phone home, if I understand correctly.
I was able to disable it with the (non-obvious) environment variable setting OTEL_TRACES_SAMPLER=always_off
.
Hope this helps!
(I agree with @somt0xhi that the health checks aspect is most likely unrelated.)
1 Like
lava
February 14, 2024, 1:06pm
4
Yes, I agree that health checking failures are another thing.
Here is my fly logs:
fly logs
2024-02-14T12:37:11Z runner[6e82936b217048] ams [info]Pulling container image registry.fly.io/02-docker:deployment-01HPKSW775Z1JTWEWVGTDCF4JQ
2024-02-14T12:37:22Z runner[6e82936b217048] ams [info]Successfully prepared image registry.fly.io/02-docker:deployment-01HPKSW775Z1JTWEWVGTDCF4JQ (11.504149513s)
2024-02-14T12:37:23Z runner[6e82936b217048] ams [info]Setting up volume 'data'
2024-02-14T12:37:23Z runner[6e82936b217048] ams [info]Uninitialized volume 'data', initializing...
2024-02-14T12:37:23Z runner[6e82936b217048] ams [info]Encrypting volume
2024-02-14T12:37:27Z runner[6e82936b217048] ams [info]Opening encrypted volume
2024-02-14T12:37:29Z runner[6e82936b217048] ams [info]Formatting volume
2024-02-14T12:37:30Z runner[6e82936b217048] ams [info]Configuring firecracker
2024-02-14T12:37:30Z app[6e82936b217048] ams [info][ 0.043902] PCI: Fatal: No config space access function found
2024-02-14T12:37:31Z app[6e82936b217048] ams [info] INFO Starting init (commit: bfa79be)...
2024-02-14T12:37:31Z app[6e82936b217048] ams [info] INFO Mounting /dev/vdb at /mnt/data w/ uid: 0, gid: 0 and chmod 0755
2024-02-14T12:37:31Z app[6e82936b217048] ams [info] INFO Resized /mnt/data to 3217031168 bytes
2024-02-14T12:37:31Z app[6e82936b217048] ams [info] INFO Preparing to run: `docker-entrypoint.sh npm start` as root
2024-02-14T12:37:31Z app[6e82936b217048] ams [info] INFO [fly api proxy] listening at /.fly/api
2024-02-14T12:37:31Z app[6e82936b217048] ams [info]2024/02/14 12:37:31 listening on [fdaa:5:a52d:a7b:c988:60ad:d730:2]:22 (DNS: [fdaa::3]:53)
2024-02-14T12:37:31Z runner[6e82936b217048] ams [info]Machine created and started in 20.032s
2024-02-14T12:37:32Z app[6e82936b217048] ams [info]> node-sqlite-fly-tutorial@1.0.0 start
2024-02-14T12:37:32Z app[6e82936b217048] ams [info]> node ./start.js
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]Prisma schema loaded from prisma/schema.prisma
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]Datasource "db": SQLite database "sqlite.db" at "file:/mnt/data/sqlite.db"
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]SQLite database sqlite.db created at file:/mnt/data/sqlite.db
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]1 migration found in prisma/migrations
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]Applying migration `20230118213146_init`
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]The following migration have been applied:
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]migrations/
2024-02-14T12:37:34Z app[6e82936b217048] ams [info] └─ 20230118213146_init/
2024-02-14T12:37:34Z app[6e82936b217048] ams [info] └─ migration.sql
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]All migrations have been successfully applied.
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]npm notice
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]npm notice New minor version of npm available! 10.2.3 -> 10.4.0
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.4.0>
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]npm notice Run `npm install -g npm@10.4.0` to update!
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]npm notice
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]Starting app...
2024-02-14T12:37:34Z app[6e82936b217048] ams [info]Server listening at http://localhost:8080
2024-02-14T13:02:51Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:02:53Z app[6e82936b217048] ams [info]GET /favicon.ico
2024-02-14T13:03:14Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:26Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:26Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:27Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:28Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:28Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:29Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:29Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:30Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:30Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:30Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:30Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:31Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:31Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:31Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:31Z app[6e82936b217048] ams [info]GET /
2024-02-14T13:03:31Z app[6e82936b217048] ams [info]POST /
2024-02-14T13:03:31Z app[6e82936b217048] ams [info]POST /
2024/02/14 14:04:23 traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 149.248.216.54:4317: connect: network is unreachable"
Like what I’ve mentioned this error is poping up everything (virtually every fly command I ran).
lava
February 14, 2024, 1:14pm
5
Thanks for your repy. I think you’re right about outgoing port being blocked. I’ve tried to switch to another wifi and the err msg stops.
Where should I put the env, could you give some hints? Would like to see those errs go away
On Linux and macOS, you’d do…
export OTEL_TRACES_SAMPLER=always_off
…within the same terminal that you’re using for flyctl
.
(And if that does turn out to work, then you can add that line to .bash_profile
, .zshrc
, or the like—so that it gets set every time.)
PowerShell I’m not very familiar with, but it looks like it might be…
$env:OTEL_TRACES_SAMPLER="always_off"
1 Like
Are these all the logs? At first glance I don’t see any error. What’s the output of fly status -a <app-name>
In the case you are seeing incomplete logs due to the network error, you can also check logs logs at the monitoring tab in the web dashboard.
lava
February 15, 2024, 9:45am
8
the error shows up in the end, sorry for pasting lengthy stuff there.
In this screenshot, I’ve ran two fly commands, and each of them introduces the error msg I was talking about.
2024/02/15 10:43:27 traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 149.248.216.54:4317: connect: network is unreachable"
lava
February 15, 2024, 9:46am
9
Thanks, got it!
After I reboot my terminal, the error msg is gone.
I don’t see any failed checks for your app?
lava
February 16, 2024, 3:28pm
11
yeah…the error message doesn’t affect the app per se.
And maybe I should open another post for not able to pass any health checks in deployment.
system
Closed
February 23, 2024, 3:29pm
12
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.