UDP and SSH issues

Questions / Help
1

I’m trying to launch a DNS server but was having issues. I deployed a UDP echo server to troubleshoot/make sure the issue wasn’t on my side. I’m using terraform (i.e. app, machines api, etc)

  • The echo server is listening on 10053 (I also tried port 53 earlier)
  • I exposed a udp service with internal port 10053 and external ports 10053 and 53 (I also tried just 53)

Here’s what I observe:

  1. Requests on port 53 never reach the server
  2. Requests on port 10053 reach the server (logs Received "hi"), but the responses never come back to me

App terrars-helloworld, app ip 109.105.221.83, testing with (echo hi; sleep 5) | ncat -u 109.105.221.83 10053

Is this a v2 issue? Is there a way to switch back to v1?

Additionally when I try to ssh I see

$ ~/.fly/bin/fly ssh console  -a terrars-helloworld -A fdaa:1:125f:a7b:9ada:6f7a:6352:2
Connecting to fdaa:1:125f:a7b:9ada:6f7a:6352:2... complete
Error: error connecting to SSH server: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

and in the logs I see

 2023-05-31T08:59:59.085 app[5683934b749e68] ord [info] 2023/05/31 08:59:59 user: 'root' does not exist

2023-05-31T08:59:59.243 app[5683934b749e68] ord [info] 2023/05/31 08:59:59 unexpected error: [ssh: no auth passed yet, user: 'root' does not exist]
2

Hi @andrewbaxter

For UDP on fly.io to work, internal and external port need to be the same:

  • The UDP side of your application needs to bind to the same port that is used externally. Fly will not rewrite the port; Fly only rewrites the ip address for UDP packets.

I just tried a simple UDP echo server and got a response from it. Can you share the code for your echo server?

As for SSH problems, do you have a /etc/passwd file with root user in your Docker image?

3

Yeah sure! Thanks for taking a look.

use std::{
    net::UdpSocket,
    thread::sleep,
    time::Duration,
};

#[tokio::main]
async fn main() -> Result<(), std::io::Error> {
    let mut buf = [0u8; 4096];
    loop {
        match UdpSocket::bind("0.0.0.0:53") {
            Ok(socket) => {
                let mut error_number = 0;
                loop {
                    match socket.recv_from(&mut buf) {
                        Ok((n, addr)) => {
                            let msg = &buf[0 .. n];
                            println!("Got message {}", String::from_utf8_lossy(msg));
                            match socket.send_to(msg, addr) {
                                Ok(_) => { },
                                Err(e) => println!("Send to {} error: {:?}", addr.to_string(), e),
                            }
                            error_number = 0;
                        },
                        Err(e) => {
                            error_number += 1;
                            println!("Receive error #{}: {:?}", error_number, e);
                            if error_number >= 10 {
                                break;
                            }

                            // 2 ^ 9 = 512 ms
                            sleep(Duration::from_millis(2u64.pow(error_number as u32)));
                        },
                    }
                }
            },
            Err(e) => println!("UDP bind error: {:?}", e),
        }
        sleep(Duration::from_secs(1));
    }
}

It works locally.

And no, it’s a musl bare docker image (only file in the image is the binary).

4

You need to bind to fly-global-services:53 instead of 0.0.0.0 for UDP (another quirk of the implementation).

And no, it’s a musl bare docker image (only file in the image is the binary).

You need at least a /etc/passwd and a shell binary for SSH to work.

5

Ah thanks! I’ll try that out! I think the documentation could be more clear that UDP has Fly-specific requirements to work - I actually clicked on that guide, but the intro doesn’t say anything about Fly-specific quirks. Having it at the bottom of a guide mixed up with TCP stuff, non-fly specific things like MTU, and beginner tutorial-like content makes it hard to find.

Yeah, I’d expect an error like /bin/sh not found if I didn’t have a shell binary, but I didn’t get that error. It’s not clear why /etc/passwd is needed for docker exec and it doesn’t appear to be documented anywhere.

In any case, thanks again.

6

Is the fly-global-services address fixed? I’m using software that requires an ip address for binding and I don’t want to wrap it in a script if I don’t have to.

7

No. Each VM gets its own, but it’s written by our init to /etc/hosts on boot:

❯ fly ssh console
Connecting to fdaa:1:d615:a7b:141:d703:cf8e:2... complete
148ed433f09158:/# cat /etc/hosts
127.0.0.1       localhost localhost.localdomain
::1             localhost localhost.localdomain

127.0.0.1       localhost

# Address in the 6PN private network for this app
fdaa:1:d615:a7b:141:d703:cf8e:2 fly-local-6pn

# Private address for this instance
172.19.165.106  148ed433f09158

# Address used for global traffic routing
172.19.165.107  fly-global-services

# Private address for this instance
2605:4c40:119:ed57:0:d703:cf8e:1        148ed433f09158
1 Like
8

Perfect, fly-global-services worked. Thanks again!

1 Like
9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.