I have 2 apps on fly, both are single machine and use caddy as a server.
So my fly.toml file doesn’t list anything http, just tcp going straight to caddy which handles http / https and getting certificates.
Today as I redeployed an app with some changes, I lost ability to get the site to load.
Since I was unable to figure it out, as a test, I redeployed the second app where there are no code changes and it has since also become fully unavailable.
When I deploy those with Caddy, it tries to setup certs and outputs error messages saying that lets encrypt could not reach the app.
Right now I have dumbed down the app as much as possible to just validate that it receives traffic.
The fly.toml only has:
app = 'testapp'
primary_region = 'sjc'
kill_signal = "SIGINT"
kill_timeout = 5
[[services]]
internal_port = 8080
processes = ["app"]
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
type = "connections"
[[services.ports]]
port = 80
[[services.ports]]
port = 443
[[services.tcp_checks]]
grace_period = "1s"
interval = "15s"
restart_limit = 0
timeout = "2s"
So the app listens to TCP connections on ports 80 or 443 and both of those should arrive on port 8080 within the machine.
The app has been reduced to a minimal Go app that listens for a TCP connection, writes “Welcome to the server” and shuts off:
package main
import (
"fmt"
"net"
"os"
)
func main() {
listener, err := net.Listen("tcp", "0.0.0.0:8080")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer listener.Close()
fmt.Println("Server listening on :8080")
for {
conn, err := listener.Accept()
fmt.Println("new connection")
if err != nil {
fmt.Println(err)
continue
}
fmt.Println("handling connection")
go handleConnection(conn)
}
}
func handleConnection(conn net.Conn) {
defer conn.Close()
fmt.Println("received connection:", conn.LocalAddr(), conn.RemoteAddr())
fmt.Println("writing to connection")
_, err := conn.Write([]byte("Welcome to the server\n"))
fmt.Println("wrote to connection")
if err != nil {
fmt.Println("error writing to connection")
fmt.Println(err)
}
}
Once deployed, I see periodic connections that seem to be fly’s healthcheck:
2025-02-08T00:21:43.121 app[d8d9e35a22e678] sjc [info] received connection: 172.19.2.114:8080 172.19.2.113:48322
Common IP prefix makes me think this is internal fly traffic.
I can also run nc localhost 8080 from within the machine and see my connection:
2025-02-08T00:23:23.588 app[d8d9e35a22e678] sjc [info] received connection: 127.0.0.1:8080 127.0.0.1:39192
And I do get the Welcome to the server response.
However, nc <public ipv4 of app> 80 results in nothing. The app doesn’t receive a connection at all. Tried on 80, 443, 8080 all give no results.
Very confused at what has happened or changed in fly’s config that suddenly makes my apps unavailable.
.