Opening Port Range for WebRTC

Questions / Help
,
1

Hi All,

i’m working on a streaming application which is being hosted on fly.io machine. I have exposed the needed ips in my Dockerfile and configured fly.toml file to have the port ranges. I’ve also allocated a shared public IP address to the app, yet the port ranges exposed are not available, as a result, webrtc connections from the client are being cancelled.

Could someone perhaps point me in the right direction what I may be doing wrong?

//..other docker config
EXPOSE 30001
EXPOSE 30002
...
EXPOSE 30100

// -- fly.toml other config
  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["http", "tls"]
    port = 443  

  [[services.ports]]
    handlers = ["http"]
    start_port = 30000
    end_port = 30100
2

From General to Questions / Help

3

Can you do EXPOSE 30001-30100?

Have you tried udp to see if that does anything?

4

@khuezy Yes I exposed 30001 - 30100. I’ve also set the udp protocol and start_port and end_port.

This is what fly.toml file looks like:

 [[services]]
  protocol = "udp"
  internal_port = 4000 

  [[services.ports]]
    start_port = 30000
    end_port = 31000

As for udp, looks like I need to bind to fly-global-services according to this but don’t know where to obtain this.

Any idea what could be going on?

5

udp in the handlers, try that.

6

@khuezy Setting the handlers = ["udp"] returns this deployment error.

Failed to update machines: failed to update machine 3dxxxx: failed to update VM 3dxxxx: Services [1] Ports [0] Handlers must be one of: tls, pg_tls, http, proxy_proto, edge_http…

Looks like handlers can only be of the specified types.

7

Try that? I’m not familiar but it looks like they configured both udp and tcp for the port.

8

Yeah, this is exactly what Ive got

app = "app"
kill_signal = "SIGINT"
primary_region = "lhr"
kill_timeout = 5
processes = []

[build]
  dockerfile = "Dockerfile"

[env]
  file = "vars.env"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 5000
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["http", "tls"]
    port = 443  
 
  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

 [[services]]
  protocol = "udp"
  internal_port = 5000 

  [[services.ports]]
    start_port = 30000
    end_port = 31000  

 [[vm]]
   size = "shared-cpu-2x"
   memory = "2gb"
9

On the Fly.io platform, the internal port has to exactly match the external port for UDP, so it’s really unclear that a range would work.

The UDP side of your app needs to bind to the same port that is used externally. Fly will not rewrite the port; Fly only rewrites the IP address for UDP packets.

E.g., a packet that came in for port 30543 on the Anycast address would (at most) be forwarded by the edge node kernel’s eBPF :honeybee: (hereafter referred to as “magic”) to port 30543 on your internal machine.

But I don’t see any indication that it would honor this configuration syntax.

How about starting with a simpler case—and then working upwards in complexity?

Even the near-minimal UDP example that @khuezy linked to has given a lot of people trouble, at first…

1 Like
10

proxy, since that’s the closest forum tag to eBPF magic.

11

Good point you raised!

I’ve actually just changed from the port_range to a single internal port, my fly.toml file now looks like so:

app = "app"
kill_signal = "SIGINT"
primary_region = "lhr"
kill_timeout = 5
processes = []

[build]
  dockerfile = "Dockerfile"

[env]
  file = "var.env"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 4000
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["http", "tls"]
    port = 443  
 
  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

 [[services]]
  protocol = "udp"
  internal_port = 4000 

  [[services.ports]]
    port = 4000

 [[vm]]
   size = "shared-cpu-2x"
   memory = "2gb"

The above doesnt work.

I saw this Your app needs to bind to the fly-global-services address, any idea how to bind to fly-global-services? Can’t seem to find this as part of environment variables.

Perhaps someone from Fly.io can also chip in

12

Hm… What does fly ips list show?

13 
v4      137.6.x.x.x           public (dedicated, $2/mo)       global
v6      fdaa:x.x.x.x         private                                        global
14

Thanks… This part looks good.

How about trying the simpler nc test from this other recent UDP thread?

https://community.fly.io/t/issue-with-udp-connection-simple-test-with-netcat/22238/7

In general, the way to bind to fly-global-services is very application-dependent, and not all UDP apps have such a knob (unfortunately).

15

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.