Sharing this in case it helps anyone else. I had a strange issue for the past few days where one of my sites hosted on Fly - https://til.simonwillison.net/ - was returning a 520 error for some users.
After some experimentation I identified that it was working in Safari on macOS but not in Firefox on macOS. “Copy as curl” in the Firefox DevTools helped me track it down.
This command failed with an HTTP 520 error:
curl -i 'https://til.simonwillison.net/' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br, zstd' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: cross-site' -H 'Priority: u=0, i' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'TE: trailers'
And this exact same command. but with that final -H 'TE: trailers' header removed succeeded and gave me a 200:
curl -i 'https://til.simonwillison.net/' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:138.0) Gecko/20100101 Firefox/138.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br, zstd' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' -H 'Sec-Fetch-Site: cross-site' -H 'Priority: u=0, i' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache'
The same commands run against my https://my-app.fly.dev/ URL (bypassing Cloudflare) worked fine.
I ended up turning off Cloudflare caching for that instance, because it turns out Cloudflare won’t let you strip the TE: trailers heading before passing that onto Fly.
Made a few more notes in my own issue here: 520 errors when accessed through Cloudflare · Issue #102 · simonw/til · GitHub
