Request for hackin': get Discourse running on Fly with global Postgres

Discourse is great! You’re using it right now. It’s also something that should, theoretically, work really well with our global Postgres model. So we’d like to pay you (or someone like you) to build us a Discourse demo.

This project has two parts:

  1. Get Discourse running on Fly in a single region. It requires Redis and Postgres, but installing it is somewhat contorted. Their installer is designed for a single VM and there’s no documented way to just push a container somewhere.
  2. Try with read replicas and Fly-replay. This may not actually work! Discourse might write to the DB (or, at least, to a Redis) on every pageview. But we’d like to know what’s involved.

We’ll pay $1,000 for each part. If that turns out to be way too little, we will work with you to make it right.

And, just for extra fun, if Discourse-the-company ever becomes a Fly.io customer, we’ll pay you a portion of what they pay us for the first two years. We think what we have is a pretty darn good fit for the software, so there’s a non-zero chance this works out. Small, but non-zero. :slight_smile:

3 Likes

I’ll take a look at this.

Can you clarify what you want for #1?

The problem with Discourse is that the further you stray from their installation process, the harder it is to perform an upgrade. For example, the in-admin upgrade option won’t work in a containerized setup.

That said, getting just the web server to run from a locally bootstrapped container is easy, using their tooling to build the container. If we’re looking for a completely containerized solution that would use Docker itself to configure the web instance, then there’s more hacking work involved. Others have tried this and none have caught on, as far as I can tell, as its not in Discourse’s business interest to have a containerized install that’s easy to scale :smiley:

For now, my assumption is that you want:

  1. A repeatable, documented way to deploy a discourse upgrade from a Git repository, using its bootstrap tool and pushing the resulting image
  2. Run Redis and Postgres externally, point discourse to these clusters
1 Like

OK, I got a basic version of this working.

Repo: discourse_docker/fly at master · soupedup/discourse_docker · GitHub
Demo: https://mr-discourse.souped-up.dev

A major caveat is the issue of having to precompile assets in the production VM after a deployment. This should be fixable in a next version that performs the container bootstrap in production. See the README linked for more info.

Discourse generally will not write on every page load. It even has a ‘read-only’ mode. So, probably well-suited for multi-region Postgres. I’ll work on this next as a Discourse plugin, then come back to improving the deployment process.

UPDATE: For #2. Since discourse plugins are loaded after the database connection is established, the middleware modifying the environment won’t work (and is messy).

Instead, we can handle this in the Rails initializer, where it should be OK to reestablish the connection.

1 Like

A final update on this, for those who recently read this. Here are some notes on the work done. This experiment was a bust in the end because Discourse uses Redis for more than caching, and it’s slow to constantly read/write to a geographically remote Redis.

Since then, we got a multi-master KeyDB installation working, so it might be interesting to revisit this. The caveat - code that uses blocking Redis commands like BRPOPPUSH could fail if run against multiple nodes against the same keys. Since discourse doesn’t seem to use these, it might be a good thing to try!

1 Like