So please try out these commands again.
fly doctor
DEV=1 fly agent probe personal
fly dig TXT _apps.internal
1 Like
Cool. So now can you update your CLI with fly version update, then:
fly ping -o personal
1 Like
Can you try running ping again?
1 Like
You should be able to post again. Sorry for the trouble!
To help debug further, I’d like you to try setting up a direct Wireguard connection for your organization. This will help us determine if the problem has to do with the ‘agent’ mentioned above.
The process is outlined here: Private Networking.
If you’re able to get a tunnel setup and connected, then we can isolate the issue more easily.
Hi Joshua,
having similar problems and following along with your troubleshooting steps. Should I start a new topic or post here?
thx for the help!
Feel free to post here.
nice.
facing the same problem and trying to follow along.
(1) fly doctor
TEST STATUS
Agent PASS
Docker (local) PASS
Probe (app) failed probing personal: failed probing "personal": context deadline exceeded
Token PASS
Unix socket PASS
(2) fly agent stop and fly agent start
(3) log
2022/02/08 15:59:54.068974 srv config change at: 2022-02-08 15:59:54.068281344 -0800 PST
2022/02/08 15:59:54.068987 #4 <- ( 4) "ping"
2022/02/08 15:59:54.069012 #4 -> ( 57) "7\x00ok {\"PID\":42542,\"Version\":\"0.0.296\",\"Background\":true}\n"
2022/02/08 15:59:54.069034 #4 dropped.
2022/02/08 15:59:54.069091 #5 connected ...
2022/02/08 15:59:54.069116 #5 <- ( 18) "establish personal"
2022/02/08 15:59:54.423411 #5 -> ( 728) "\xd6\x02ok {\"WireGuardState\":{\"org\":\"personal\",\"name\":\"REDACTED\",\"region\":\"sea\",\"localprivate\":\"l/zREDACTEDr/fmSBA=\",\"localpublic\":\"qrsQqopN6JNsghWfOn9J0DLMJ01LHJC/bWJ7wo4N+w4=\",\"dns\":\"\",\"peer\":{\"peerip\":\"fdaa:0:4152:a7b:177d:0:a:2\",\"endpointip\":\"sea1.gateway.6pn.dev\",\"pubkey\":\"SN9V6BfrC9Dn+g84LGSDI+3VHBw3piOyaF2gZNuVNF4=\"}},\"TunnelConfig\":{\"LocalPrivateKey\":\"REDACTED/REDACTED+w4=\",\"LocalNetwork\":\"fdaREDACTED0/120\",\"RemotePublicKey\":\"SN9V6BfrC9Dn+g84LGSDI+3VHBw3piOyaF2gZNuVNF4=\",\"RemoteNetwork\":\"fdaa:0:4152::/48\",\"Endpoint\":\"sea1.gateway.6pn.dev:51820\",\"DNS\":\"fdaa:0:4152::3\",\"KeepAlive\":0,\"MTU\":0,\"LogLevel\":0}}\n"
2022/02/08 15:59:54.423514 #5 dropped.
2022/02/08 15:59:54.423842 #6 connected ...
2022/02/08 15:59:54.423907 #6 <- ( 14) "probe personal"
2022/02/08 15:59:54.423924 srv probing "personal" ...
2022/02/08 15:59:59.425155 #6 -> ( 58) "8\x00err failed probing \"personal\": context deadline exceeded"
2022/02/08 15:59:59.425427 #6 dropped.
fly version
Darwin 20.6.0 Darwin Kernel Version 20.6.0: Wed Jan 12 22:22:45 PST 2022; root:xnu-7195.141.19~2/RELEASE_ARM64_T8101 arm64
fly v0.0.296 darwin/arm64 Commit: 25eb4c5 BuildDate: 2022-02-08T18:17:06Z
[16:20:44]phoenix/phx166» fly ping -o personal
It looks like it’s going to time out… it’s just sitting there…
with debugger
DEBUG Loaded flyctl config from/Users/kme/.fly/config.yml
DEBUG determined hostname: "kme-"
DEBUG determined working directory: "/Users/kme/Projects/phoenix/phx166"
DEBUG determined user home directory: "/Users/kme"
DEBUG determined config directory: "/Users/kme/.fly"
DEBUG ensured config directory exists.
DEBUG ensured config directory perms.
DEBUG cache loaded.
DEBUG config initialized.
DEBUG initialized task manager.
DEBUG skipped querying for new release
DEBUG client initialized.
DEBUG app config loaded from /Users/kme/Projects/phoenix/phx166/fly.toml
DEBUG --> POST https://api.fly.io/graphql {{"query":"query($slug: String!) { organization(slug: $slug) { id internalNumericId slug name type } }","variables":{"slug":"personal"}}
}
DEBUG <-- 200 https://api.fly.io/graphql (675.38ms) {"data":{"organization":{"id":"60L7BO3NPGLMXHvjvN6nmoY6nAFq08","internalNumericId":"29800","slug":"personal","name":"kim.me@gmail.com","type":"PERSONAL"}}}
DEBUG --> POST https://api.fly.io/graphql {{"query":"mutation($input: ValidateWireGuardPeersInput!) { validateWireGuardPeers(input: $input) { invalidPeerIps } }","variables":{"input":{"peerIps":["fdaa:0:4152:a7b:177d:0:a:2"]}}}
}
(edited…was running in wrong folder)
still sitting there … will probably timeout eventually
@kim I’m not seeing any handshakes to the wireguard gateway from your local CLI agent. Will you try stopping the agent again, then run ps aux | grep "fly agent" and make sure it’s actually stopped?
Here you go.
[19:06:33]phoenix/phx166» fly agent stop
[19:06:37]phoenix/phx166» ps aux | grep "fly agent"
kim 44535 0.0 0.0 408103312 1360 s001 S+ 7:06PM 0:00.01 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox fly agent
[19:06:46]phoenix/phx166»
Context:
I am planning to use fly.io for a personal phoenix project. I got it deployed the other day but had trouble tunneling it via WireGuard and livebook. I am using this project to stand up a vanilla 1.66 phoenix environment following the fly.io docs.
This is not urgent btw! Thx for the halp!
We should have some slightly better diagnostics for wireguard through our agent soon. For now, try setting up a wireguard connection with a real client and fly wg create. It’ll show you a bit more info about handshake success, at the very least.
fly wg create
Automatically selected personal organization: kim.sfsdfm@gmail.com
Creating WireGuard peer "interactive-kim-C02F23SSQ05N-kim-imsdfsdf-gmail-com-714" in region "dfw" for organization personal
!!!! WARNING: Output includes private key. Private keys cannot be recovered !!!!
!!!! after creating the peer; if you lose the key, you'll need to remove !!!!
!!!! and re-add the peering connection. !!!!
? Filename to store WireGuard configuration in, or 'stdout': wireguard
Wrote WireGuard configuration to wireguard; load in your WireGuard client
LOG_LEVEL=debug fly ping -o personal
DEBUG Loaded flyctl config from/Users/kim/.fly/config.yml
DEBUG determined hostname: "kim-C02F23SSQ05N"
DEBUG determined working directory: "/Users/kim/Projects/phoenix/phx166"
DEBUG determined user home directory: "/Users/kim"
DEBUG determined config directory: "/Users/kim/.fly"
DEBUG ensured config directory exists.
DEBUG ensured config directory perms.
DEBUG cache loaded.
DEBUG config initialized.
DEBUG initialized task manager.
DEBUG skipped querying for new release
DEBUG client initialized.
DEBUG app config loaded from /Users/kim/Projects/phoenix/phx166/fly.toml
DEBUG --> POST https://api.fly.io/graphql {{"query":"query($slug: String!) { organization(slug: $slug) { id internalNumericId slug name type } }","variables":{"slug":"personal"}}
}
DEBUG <-- 200 https://api.fly.io/graphql (579.42ms) {"data":{"organization":{"id":"60L7BO3NPGLMXHvjvN6nmoY6nAFq08","internalNumericId":"29800","slug":"personal","name":"kim.il.com","type":"PERSONAL"}}}
DEBUG --> POST https://api.fly.io/graphql {{"query":"mutation($input: ValidateWireGuardPeersInput!) { validateWireGuardPeers(input: $input) { invalidPeerIps } }","variables":{"input":{"peerIps":["fdaa:0:4152:a7b:177d:0:a:2"]}}}
}
DEBUG <-- 200 https://api.fly.io/graphql (124.21ms) {"data":{"validateWireGuardPeers":{"invalidPeerIps":[]}}}
Sorry, fly ping won’t work with a proper wireguard setup (it might, but I think it would require an env var).
The best way to test the wg peer is to run fly ips private on your app, then just ping it like normal (possibly ping -6 depending on your OS).
[19:24:31]phoenix/phx166» fly ips private
ID REGION IP
just to clarify:
I currently don’t have an app running or deployed. I nuked the project up to start over with a generic phoenix app. I just wanted to rule out any config errors on my end.
So my “main” problem is that I can’t deploy… and then I found this thread
[19:22:43]phoenix/phx166» fly launch --remote-only
An existing fly.toml file was found for app solitary-snow-2344
App is not running, deploy...
Deploying solitary-snow-2344
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
Error error connecting to docker: failed building options: failed probing "personal": context deadline exceeded
FYI this is still a thing. Not critical though… but would be great to deploy eventually
fly launch --remote-only
An existing fly.toml file was found for app ancient-flower-5218
App is not running, deploy...
Deploying ancient-flower-5218
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
Error error connecting to docker: failed building options: failed probing "personal": context deadline exceeded
fly doctor
TEST STATUS
Agent PASS
Docker (local) PASS
Probe (app) failed probing personal: failed probing "personal": context deadline exceeded
Token PASS
Unix socket PASS
Hey! Please install the latest prerelease:
curl -L https://fly.io/install.sh | sh -s pre
Then try:
fly wireguard status
Select your personal org and, assuming you only have one ‘peer’ on the list, pick it and paste the output here.