Replay headers with private networks

We’re using machines with private networks enabled.
Can one machine accept traffic (public IP, default network name) and then route traffic to another machine (no public IP) in the same org, but with a different network name.

Yup, you can use fly-replay to send requests to apps/machines that are in the same org, but on different networks.

This is useful for cases where you’d like a machine to only be reachable by fly-replay or other hosts on its network.

3 Likes

Hi @eli, I’d consider this a security issue. The idea of a private network is isolation. An app in a private network can redirect a request to any app instance in the same organization. That allows scenarios to spy out apps. Instance IDs like 00bb33ff aren’t so hard to guess / brute force. The crux is that apps in private networks are usually not protected. Is this by design? AFAIK The only way to protect against it would be to ignore requests with an incoming The Fly-Replay Header · Fly Docs header.

Replay only works to apps that have service ports defined. You can’t replay to arbitrary ports to machines, they have to be configured to accept “public” traffic with a service definition.

This is by design. You definitely want to sanity check traffic you receive, though. You can do a lot to shoot yourself in the foot with replay.

1 Like

Thanks, @kurt for clarifying. I think in our scenario, this creates a security problem that needs addressing. We’re running multi-tenant applications on behalf of our users. That is, we run user code and want to isolate it. Each user has their machines in their own “network”, so that’s isolated. However, this construct breaks with the replay header as a user might write code to use the replay functionality to “guess” other users’ applications/instances. They might even use the fly internal DNS to discover apps. I’m not sure if the internal DNS respects just the org or networks though.

Anyways, I think the best solution would be to add a flag to the machine config which enables or disables the replay header. For our routing app, which we own ourselves, we enable this flag, so the fly router will respect replay headers. On customer apps, we will let this flag in the default state (disabled) so that apps from customers cannot use the replay header feature. This way, we’re back to true “network isolation” without loopholes.

What do you think of adding such a flag to “explicitly” enable the replay header feature?