Static egress IPs for machines

I’ve deployed an app with egress-ip to hkg. but all egress connections are failing.

@akshit-fly which region does work best for this?

Is any region working with egress-ip list for anyone?

Original post: egress IP - not reflecting even after using the command - #3 by abeeshake1

1 Like

I caught the update in this thread - egress IPs are working for me in DEN and DFW :tada:

2 Likes

@emilyemorehouse Thanks for referencing the other thread. We’ve got it working in GRU and MIA :tada::tada::tada:

@akshit-fly can I set egress ip for LHR location?

@akshit-fly would it be possible to get IPs from the jnb region?

It seems like that egress statis IP is still http-proxy based outbound IP mapping. It doesn’t work for non-http traffic. Can someone from fly let us know if support for non-http calls is in the works?

1 Like

I just lost a week on this issue, pulling my hair out. Simply hadn’t faced it before, but it’s clear now.

  • Could you PLEASE document this platform constraint more clearly, adding catch terms for MongoDB Atlas and other common cases.
  • Could you please doc the static, and static egress IP functionality better. Add line items on the pricing page too? This thread was my first contact with the topic after 10-20+ search & LLM sessions.
3 Likes

How is that billed for stopped machines? Is it per hour with egress IP assigned or per hour with running machine using that egress IP?

Also, please update the official pricing documentation. It took me a while to land here only searching for the egress IP pricing info.

Thanks!

The price is updated here

Any chance offering a less expensive version for just having a fixed IPv6 address?

These seem like serious downsides to not use egress. How can I use this feature if my app re-deploys after a change?

Seems like a pre-mature feature.

Any plans on having a stable egress ip offering?

Static egress IPs are per-machine, not per-app.

  • IPs are released when a machine is destroyed.

  • IPs don’t automatically transfer across deploys.

  • Blue/green deployments will replace machines—and their IPs.

  • Deployment-time jobs may bypass egress routing.

  • Extra latency and connectivity issues are possible in some regions.

It was a good addition to the platform’s core capabilities, but there’s a broad consensus that the current incarnation is a bit low-level and inconvenient for many users, indeed. Last I heard there was interest in providing a higher-level alternative (which would essentially do the proxying mentioned above behind the scenes, for you):

I seem to remember that there was also a more recent reference to work actually currently being in progress on this (as opposed to just being a good idea), but I haven’t been able to find that post again yet, :magnifying_glass_tilted_right:

Unless you’re using the blue-green mode, this part is often fine. Each Machine retains its identity during a rolling deploy.

The release_command causes problems, though, since that’s a new, ephemeral Machine each round…


Edit: Ah, the passage I was recalling earlier was really a recent commit to the docs themselves.

App-scoped egress IPs are in development. These will simplify routing and avoid per-machine binding.

They will inherently also eliminate the present blue-green and release_command caveats, of course, since those both stem from per-Machine binding.

1 Like

Some additoional feature i’d like to have

  • Use the same IP for all machine inside apps (or even orgs ? would be better) (i use proxy squid for now )
  • DO NOT Delete my ip when i try fly scale count 0 (can we have like volumes here guys , where i can re attach, i have nightmare incident because of this, morning migration i want to rescale the website i try to fly scale count 0 then update it again now my IP is gone, the thing is my app is sending request to atleast 10 different third party apis and half of them use IP Validation)
1 Like

Ouch, :adhesive_bandage:… The app-scoped IPs in the quote above would be like the Squid approach, except that the proxies would be managed by Fly.io instead of by you. (That’s my understanding, anyway.)

You would no longer be able to accidentally release the IP by scaling, I wouldn’t think.


These might end up being two IPs per region, though, for HA…

that would be better if they manage the squid config or give us sample , tbh i have skill issue problem with this network thing sometimes my squid lag, die, wont work i have no idea why

I am using rolling strategy. So this should work without any problems then?

I am using rails and the release_command is `rake db:prepare` which runs migrations if any.

I will give it a try.

And they do say in the docs:

App-scoped egress IPs are in development and may simplify this in the future.

Not sure when it’s scheduled to be released.

If you just need the egress IP for non-db: external APIs, then, yep, it typically would be fine.

However, if it’s the database itself that’s external, then the release_command will fail, since it’ll be running on a new Machine each time (and hence won’t have a static egress IP of its own, :dragon:).

Yeah, that’s a lot of moving parts to keep track of, on top of the 10 external APIs, etc.

Squid is primarily a caching proxy, though, so maybe there’s an easier substitute… in the interim?

@akshit-fly posted an example of using Smokescreen upthread, although I haven’t looked at that in detail myself yet, :turtle:

It’s easy if you try! What you want is to run your server-side HTTP through a proxy.

[…]

Smokescreen is an egress proxy that was built at Stripe to help manage outgoing connections in a sensible and safe way.

(edit: that quote is from the second, older link.)

Why can we not just assign a dedicated IP and use that for inbound and outbound requests? That is what I would expect when I see dedicated IPv4. This is a serious problem that could have cost us hundredds of thousands of $ if we hadn’t caught it. There should just be a single IPv4 address used for egress and ingress, and it should be used for all machines in the app. You can obviously use a NAT to decide which machine to route to.

Your plea (along with the related ones of @erlangga, @zwhitchcox, and others) has been heard, it seems! The beta of per-app egress IPs was just announced:

https://community.fly.io/t/beta-feature-try-out-app-scoped-egress-ips/26536

There are still some limitations, like really being per-region rather than strictly per-app, still a distinction between ingress and egress(?), etc., but it looks like a much closer approximation of the ideal…