TCP connection close

[imaai]

I have a simple server written that handles TLS termination.
I have a dedicated IPv4.

When I try to reach my server via dedicated IPv4 something closes connection. And I get no connection logs.

curl -vi https://<dedicated_ip>:443 -k

*   Trying <dedicated-ip>:443...
* Connected to <dedicated-ip> (<dedicated-ip>) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <dedicated-ip>:443 
* Closing connection
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <dedicated-ip>:443 

When I proxy the traffic to local everything works okay.

curl -vi https://localhost:3010 -k

Healthcheck also passes. It seems like request never reaches server. Any ideas?

[build]

[[services]]
  internal_port = 443
  protocol = "tcp"

  [[services.ports]]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

[[vm]]
  cpu_kind = 'shared'
  cpus = 1
  memory_mb = 1024

[pavel]

Hey @imaai

This looks like a bug on our side. Looking into this.

[imaai]

Thanks Pavel, please let me know when you figure it out.

Some additional information.

I didn’t touch service configuration.

I changed server implementation temporary to handle pure HTTP to exclude any issues related to TLS on my side.

I configured CF DNS to proxy custom domain to dedicated ip and it RESPONDS.

Wierd.

How can server be accessed through dedicated IP if that’s the only thing CF proxy knows about?

❯ curl -vi http://<dedicated-ip>:443 
*   Trying <dedicated-ip>:443...
* Connected to <dedicated-ip> (<dedicated-ip>) port 443
> GET / HTTP/1.1
> Host: <dedicated-ip>:443
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
* Empty reply from server
* Closing connection
curl: (52) Empty reply from server

Empty response? Let’s try default port 80.

❯ curl -vi http://<decicated-ip>    
*   Trying <decicated-ip>:80...
* Connected to <decicated-ip> (<decicated-ip>) port 80
> GET / HTTP/1.1
> Host: <decicated-ip>
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
< location: https://<decicated-ip>/

Permanent redirect to https? Huh.

* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <dedicated-ip>:443 
* Closing connection
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <dedicated-ip>:443 

Nah still no luck.

[imaai]

One more observation.

Afterwards I have changed services.port to 80 and CF could no longer retrieve response.

Based on that I suppose that request coming from CF are sent via https and somehow accepted by fly.

I think application is receiving TLS terminated traffic.

[khuezy]

@imaai

I have a simple server written that handles TLS termination

What exactly do you mean by this? Fly automatically terminates TLS, so maybe you’re double terminating it? A wild guess.

[imaai]

I’m not using any handlers in service configuration so according to documentation I should receive raw TCP connection.

[pavel]

@imaai

I believe this should be fixed now.

We had a bug that, under certain conditions, caused proxy to keep both old and new service config. After your service config got changed (http + tls handlers removed), the proxy had both old and new configurations for port 443 of your app and got confused whether or not it needs to terminate TLS.