How to use fly.io to manage the multiple domains for our project

42

This is Cloud Flare rejecting connections. You can make it work if you change your nginx config to:

server {
  listen 80;
  listen [::]:80;

  server_name whitelabel.inlinecheckout.com;

  location / {
      proxy_pass https://payments-staging.inlinecheckout.com/;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_ssl_server_name on;
  }
}

Usually it’s better to bypass Cloud Flare for this kind of thing. They are very hostile to reverse proxies, and you might run into issues with it blocking your nginx VMs. If you can point directly to the GCP load balancer that will work better.

43

Hey Kurt, thanks for babysitting me, I still didn’t manage to find the solution :roll_eyes:

I ditched Cloudflare and trying to set this up with Fly hostname to begin with but I face issues:

nginx.conf:

worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /tmp/nginx.pid;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    log_format main '$remote_addr [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';
    access_log /var/log/nginx/access.log main;

    server {
        listen                        443 ssl;
            server_name               whitelabel.fly.dev;
            ssl_certificate           /etc/nginx/whitelabel.crt;
            ssl_certificate_key       /etc/nginx/whitelabel.key;

        location / {
            proxy_pass                https://payments-staging.inlinecheckout.com/;
            proxy_ssl_server_name     on;
            proxy_set_header          X-Forwarded-Host $http_host;
        }

        location = /favicon.ico { access_log off; log_not_found off; }
        location = /robots.txt  { access_log off; log_not_found off; }
    }
}

Dockerfile:

FROM nginx:1.19.7-alpine
ENV PORT 443
ENV NGINX_ENTRYPOINT_QUIET_LOGS 1
RUN rm -rf /etc/nginx/conf.d
COPY certs/ /etc/nginx/
COPY nginx.conf /etc/nginx/nginx.conf

My Fly app is up and running but no proxied content appears at all, each time I get this message:

400 Bad Request

The plain HTTP request was sent to HTTPS port
nginx/1.19.7

This is confusing as my Fly app listens only on SSL port and backend app I proxy to is also HTTPS-enabled Kubernetes ingress-nginx. Any ideas on why this weird error occur?

Can I provide more info to help debug somehow?

44

Fly.io apps should listen for http, not for SSL. We handle SSL for you and just make normal HTTP requests to your app (in this case, nginx). If you change your listen directive to this:

listen 80;
listen [::]:80;

And make sure the internal_port in fly.toml is set to 80, it should work.

45

Hmm, okay… Just tried it that way, still no luck Kurt :slight_smile:
Namely, I see these in Fly app logs now:

2021-03-01T20:24:44.456Z 73dbc094 lhr [info] 2021/03/01 20:24:44 [error] 526#526: *68 SSL_do_handshake() failed (SSL: error:14094410:SSL routines:ssl3_read_by
tes:sslv3 alert handshake failure:SSL alert number 40) while SSL handshaking to upstream, client: 185.40.232.114, server: checkout-whitelabel.fly.dev, request
: “GET / HTTP/1.1”, upstream: “https://[2606:4700:20::681a:f94]:443/”, host: “checkout-whitelabel.fly.dev”
2021-03-01T20:24:44.459Z 73dbc094 lhr [info] 2021/03/01 20:24:44 [warn] 526#526: *68 upstream server temporarily disabled while SSL handshaking to upstream, c
lient: 185.40.232.114, server: checkout-whitelabel.fly.dev, request: “GET / HTTP/1.1”, upstream: “https://[2606:4700:20::681a:f94]:443/”, host: “checkout-whit
elabel.fly.dev”

I’m trying to proxy the HTTPS-enabled app (Kubernetes Ingress)…

Any advice?

46

Gave up. Thanks for nothing.

47

Oh dang, sorry. I’ve been unavailable most of the week and lost this.

That original config I gave you should still work. But we also can’t really help much with k8s ingress + nginx. Those logs look like the kubernetes cluster is rejecting the connection or not serving SSL for some reason.

48

I tweaked our nginx example so it should run with minimal changes, have a look here: GitHub - fly-apps/nginx: A fly app nginx config

This should only require a two line change for most origins.