Maintenance mode?

Questions / Help
1

Is there a way to set an app to “maintenance mode”, like one can in Heroku?

For example, define a simple HTML page that can be displayed while maintenance tasks are performed which would degrade the performance of, or even break, a site.

As part of some deploys I need to toggle a switch on another service. Switching before the deploy breaks the “old” site. Until the switch is done, the new deploy will fail. Unfortunately it isn’t something that can be automated, and I don’t think any of the current deploy strategies fit the circumstance.

2

I don’t think we have a great answer for this built in. When we’ve needed to do this with our own apps, we’ve switched the database connection user to readonly, done the changeover, then swapped that. Would something like that work for what you’re doing?

3

Unfortunately, no.

What I need to do is:

  • Enable “maintenance mode” to display a static HTML file for all requests, stopping requests to the “current” container/app
  • Toggle 3rd party switch to move from one “version” to another, which would cause any requests to the “current” app to break
  • Deploy the latest version of our code, which understands how to use the latest 3rd party version
  • Disable maintenance mode to make the new container/app accessible

Think of it as a 3rd party having 2 incompatible versions of an API, but only one can be running at a time. There isn’t really any clean way to migrate between them without going offline for a couple of minutes for the deploy, or expending a lot of time and effort writing code to handle the failover modes.

Having a maintenance mode would be useful in other scenarios too – for example, protecting an app from an active attack so the vulnerability can be patched.

4

I too use a readonly flag in my app (which lets you do interesting things like move your entire DB from one provider to another while keeping your app running but I digress).

One idea might be to implement maintenance mode inside of your app (eg maintenance_on renders the maintenance page). Obviously I don’t know your stack, but in Rails this is pretty easy with a before_filter in the application controller or in Go it would be a piece of middleware running early in the chain that does the same thing.

Or you could put a proxy (say OpenResty/nginx) in front of your app to set/detect maintenance mode and render the static page albeit at the cost of extra architectural complexity.

5

That’s not a bad idea, and might work in my case in the short term. I’d need to check, as the flipping the toggle in the 3rd party system could cause automated processes to fail, and cause a restart of the app, causing a loop and degrading experience.

One thing I did find with Heroku’s maintenance mode is that it can help the app recover when it is hit hard. It’s not ideal, but blocking all requests from hitting the container while you resize or redeploy or investigate or rollback is super useful.