Unable to deploy Elixir app, no feedback in output or logs

Oh, interesting. The fly deploy --strategy immediate --remote-only attempt created a completely new app fly-builder-restless-surf-2468 in my account. I’ve not seen this happen before. It is also “dead”, along with my downed app.

I’m really annoyed I have no recovery options here. Any one of the following options would mitigate this downtime:

  • A toggle for “maintenance mode” to display a static HTML file while the toggle is active. I could switch that on and provide a notice to our users.
  • A “failover” option when an app is “dead” or non-responsive, to either show a static HTML page or proxy to another site. Again, I could provide a notice to our users.
  • A manual rollback option. Something like flyctl deploy --rollback v28, or being able to click a Rollback to here button in the “Activity” dashboard screen next to a “green” build. Being able to revert to a previously running state without a deploy would really help when deploys aren’t working as expected.

So, the site is down hard, the option of “deploying something to fix” isn’t available, I have no other tools or channels available to me, and I’m starting to get emails about it. My only option is to repoint the DNS somewhere and hope people don’t see the error for too long.