It works. Thanks for opening this up, we’re really excited about using Fly!
3 Likes
Port 25 is looking good for me too! Thanks.
I looked into tls briefly. My understanding is that a typical expectation for mail servers is:
- port 25: try to upgrade via starttls; server decides whether non-upgraded sessions are allowed
- port 465: smtp over tls
- port 587: like port 25, but require tls upgrading
The port 465 standard does seem like it could work if the port was available: I set up a tls handler on another port and got mail through with swaks --tls-sni myapp.fly.dev --tlsc .... That flag was only recently added to swaks, though, so I have no idea how common it is for mail clients to actually do sni.
Ports 25 and 587 seem like they would work if I managed certs myself. Conceptually it seems simple to have something periodically write certs to a fly secret, but I’m fuzzy on the details. Doing an acme http challenge from my app on fly seems hairy since I’d need to serve smtp and http from one image, right? Or maybe I could run something externally that uses a dns challenge? Access to the raw TLS certs via the Fly API would also solve this, I think.
1 Like
It seems like SNI might work with port 465 with our built in TLS handler. Postfix sends SNI too, it might be well supported.
You don’t have to run an HTTP server on the same app, you can do DNS based verification.
I’d been meaning to set up cert-manager anyway, so I’ve now got that running outside of fly with dns challenges. I’m still working on the best way to deploy the certs.
I was planning to put the key + cert into secrets so that my external cron could run flyctl secrets set instead of a build/deploy. But, the cert (in pem format) turns out to be too big for that: it’s ~5kb and secrets are limited to 4kb. For now I’ve set it via an env var in flyctl deploy.
I think could avoid a build by finding the current image via fly’s registry and passing it to flyctl deploy --image, but that seems like a hassle. My best idea right now is to just chop up the cert into 4kb chunks and concat them at startup. Anyone got a better idea?
I wonder why we restricted secrets to that size. We’ll see if we can up that limit tomorrow.
1 Like
Just investigating this, and noticed that the client IP (eg 93.187.216.173) that connects on port 25 is the edge proxy IP (I assume anyway, since it’s always the same regardless of which client I actually connect from).
Is there an option to setup a SMTP handler, like the HTTP ones, that support the SMTP proxy functionality, so that we can get the real client IP ?
Just a stab, but: getting the original source is the use case for the proxy_proto handler:
https://fly.io/docs/reference/services/#proxy-protocol
Your SMTP software would need to grok the HAProxy proxy protocol to make that work.
I’m glad this works for you but sad that this collapses a giant ambitious nightmare network feature I just proposed to Kurt.
2 Likes
Has the limit been raised?
To close the loop here: I finally got back to this project and everything is working beautifully! The secrets limit has been raised, and now that all ports are allowed I can run on 25, 465, and 587.
In case it’s helpful for others working on smtp, here’s the relevant bits of my fly.toml:
[[services]]
internal_port = 25
protocol = "tcp"
[[services.ports]]
port = 25
[[services.ports]]
handlers = ["tls"]
port = 465
[[services.ports]]
port = 587
Thanks for all the improvements along the way! I’m really excited to cut traffic over from my old server.
3 Likes
I’d be interested in seeing your postfix config & dockerfile if you are able to share. I’m struggling right now with starttls specifically, and am wondering if you have solved that.
Thanks for sharing 
oh, haha. me too, I just assumed you were using postfix and was going to try to adapt it to haraka. Would you be able to share what you have for haraka? Thanks
Hah, that makes it easy. I’m running more or less stock config.
tls.ini: key and cert are set
smtp.ini: listen=0.0.0.0:25 to match internal_port
plugins: tls is included
My dockerfile installs haraka and then runs a script that copies my cert over from secrets:
mkdir -p /var/lib/acme/devnull.noreply.zone/
echo "${DEVNULL_KEY}" | base64 -d > /var/lib/acme/devnull.noreply.zone/key.pem
echo "${DEVNULL_CRT}" | base64 -d > /var/lib/acme/devnull.noreply.zone/fullchain.pem
exec npx --no-install haraka -c /app/devnull
I’ve also since switched my fly.toml from the earlier example to use port 465 without the tls handler. This makes it do starttls with my cert rather than smtp-over-tls with fly’s. I’m not sure it really matters since that port seems to be deprecated for smtp usage.
1 Like
I’ve got A/AAAA records to my fly ips and an MX record pointing at the domain of the A/AAAA records. It’s been a while since I set it up so I don’t remember if there was any big tradeoff involved here, but it seems to work fine.
Is this working for people with v2? My config looks like this:
[[services]]
internal_port = 25
protocol = "tcp"
[[services.ports]]
port = 25
[[services.ports]]
handlers = ["tls"]
port = 465
[[services.ports]]
port = 587
[[services]]
internal_port = 8080
protocol = "tcp"
[[services.ports]]
handlers = ["http"]
port = "80"
force_https = true # optional
[[services.ports]]
handlers = ["tls", "http"]
port = "443"
Then to test:
> telnet example.com 25
Connected to example.com
Connection closed by foreign host
Note that this is the same response you get from any random port so I’m pretty sure it’s not getting to my app (also not seeing anything in the logs).
Also tested telnet with port 80 and that is getting through.
Here’s some very simple Go code:
l, err := net.Listen("tcp", ":25")
if err != nil {
panic(err)
}
for {
c, err := l.Accept()
fmt.Printf("Got a connection - %v - %v\n", c, err)
}
Where the print never occurs. Pretty sure it’s not getting to my app. Any pointers are appreciated.
For me (using Elixir and Phoenix ) I have to make sure the internal port is 2525 (below 1024), then mapping it to external 25 etc.
[[services]]
# have to use 2525 otherwise will get this error
# o]"Starting SMTP server at port 25"
# 2023-09-29T17:41:58Z app[683d3d6f777528] lax [info]17:41:58.656
# [error] Failed to start Ranch listener :gen_smtp_server in :ranch_tcp:listen([{:cacerts, :...}, {:key, :...}, {:cert, :...}, {:port, 25}, {:ip, {0, 0, 0, 0}}, {:keepalive, true}, :inet]) for reason :eacces (permission denied)
internal_port = 2525
protocol = "tcp"
# mapping these three ports ALL to internal 25
# https://community.fly.io/t/receiving-email-inbound-smtp/1033/24
[[services.ports]]
port = 25
[[services.ports]]
handlers = ["tls"]
port = 465
[[services.ports]]
port = 587
Maybe that’s a constraint when using shared IP? Not tested if I can do 25 directly using dedicated IP.