Docker Compose Compatibility: The Journey Begins

Fresh Produce
1

Now that you can Use Containers with Flyctl (now with docs!), it’s time to take the next step—or rather, the first step on the next journey: Docker Compose compatibility.

Make no mistake: there are a number of things that Docker Compose can do that aren’t compatible with the Fly.io containers implementation, and vice versa. But the important thing is that the overlap between what the two can do is quite significant, and many of the differences can be mitigated.

Example motivations for this feature:

  • You have an existing application that runs with Docker Compose, and you want to deploy it to Fly.io.

  • You ran fly launch on a new application and it fails, and you want to debug it locally before trying again (divide and conquer).

  • You have an application deployed to Fly.io and want the ability to test out new features locally before deploying changes.

A quick demo to demonstrate the possibilities. Start by running:

fly version upgrade

Make sure that you are running v0.3.149 or later. Then, in an empty directory, run:

fly launch --from=https://github.com/fly-apps/rate-limiter-demo.git --ha=false

This will create an app that dumps out HTTP headers—nothing fancy.

Now edit the fly.toml and add:

[build.compose]

Then change the internal port so that traffic is routed to the nginx container:

internal_port = 8080

Run fly deploy and refresh your browser rapidly. You will soon see:

503 Service Temporarily Unavailable

All without modifying a single line of your application. All that is required is a compose.yml and nginx.conf file, and you now have a two-container application that you can build and deploy.

This is just the beginning—there is much more work to be done. I would like that work to be based on real-world use cases, so let us know what you would like to see supported!

Edit: updated the example to match the current syntax

14 Likes
2

The perfect notification doesn’t exi-

4 Likes
3

flyctl v0.3.149 linux/amd64 Commit: 2c142c1141c084c54311d00923ae1c00be4d9407 BuildDate: 2025-07-03T18:18:47Z

Error: failed to fetch an image or build from source: app does not have a Dockerfile or buildpacks configured.

1 Like
4

Can you share your compose.yml file?

5

$ cat fly.toml
app = ‘testabc’
primary_region = ‘ams’

[build]
compose = ‘compose.yml’

[[vm]]
cpu_kind = ‘shared’
cpus = 1
memory_mb = 256

$ cat compose.yml
version: “3.8”

services:
nginx:
image: nginx:latest
ports:
- “8080:8080”
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
depends_on:
echo:
condition: service_healthy

echo:
build: .
healthcheck:
test: [“CMD”, “wget”, “-qO-”, “http://localhost:80”]

$ flyctl deploy
✓ Configuration is valid
→ Verified app config
==> Building image
==> Building image
Error: failed to fetch an image or build from source: app does not have a Dockerfile or buildpacks configured. See App configuration (fly.toml) · Fly Docs

1 Like
6

OK, so it looks like your fly.toml was built by hand (“memory_mb = 256”), which suggests you might not have cloned the rate limiter demo.

My directory looks like this:

% ls    
api-config.json	compose.yml	fly.toml	README.md
cli-config.json	Dockerfile	nginx.conf	server.js

You can omit the json and README files, but you will need the rest to be in place for this demo to work.

8

I’m proposing to change the fly.toml to (in most cases):

[build.compose]

Below that, one can specify the file, but in most cases that won’t be necessary: it will pick up the first of compose.yaml, compose.yml, docker-compose.yaml, or docker-compose.yml.

But the real reason for this change is to allow more options to be specified here. Examples:

  • Replacement of postgresql and redis with fly equivalents. Run locally with your own database, run in production with a managed database.
  • Allow fly secrets to override the definition in the compose file.
  • Update /etc/hosts to mimic docker compose network isolation. Everything runs on localhost on fly.io, and port number collisions are not likely, but service names are likely used in connection strings.
  • Exploit processes: run sidekiq on a separate machine
  • Volume mapping

These are just some ideas that need exploration and validating with real world use cases. But it seems obvious to me that some room for options is needed.

Next up after that will be modifying fly launch to auto detect compose files and set up a fly.toml with reasonable defaults as a starting point.

2 Likes
9

Nice, that is a good refinement idea, overall…

I think that making [build.compose], without any lines under it, be significant would put extra load on a weak point of TOML syntax, though.

Intuitively, empty stanzas do nothing—even though that’s not strictly what the semantic rules say.

(It’s the difference between {"build": {"compose": {}}} and {"build": {}} in JSON syntax.)

Maybe an explicit auto_detect_file = true, instead?

This would be really neat, :black_cat:.

(I worry about all those single-node Postgres instances out there…)

1 Like
10

That’s actually what I was going for. In JavaScript terms, I can test for:

if (c.build?.compose)

Or in my case (in go):

if c.Build != nil && c.Build.Compose != nil {

If it actually is a problem, I would rather make file required.

11

One problem though. I am using VSCode and the built-in schema autocomplete is faaaar old compared with the latest features. What’s at stake to have support for autocomplete in IDE? Is there any help the community can provide?

12

For the sake of clarity, are those api-config.json and cli-config.json files needed when compose compatibility is in use? I’m thinking probably not?

13

Correct, you can omit the json and README files.

1 Like
14

I just migrated some of my workloads from multiple Fly Apps to using a single Fly App with multiple machine, leveraging Process Groups.

It’s working great, since I have one (1) process group that needs a bigger machine and other four (4) works great with the smallest machine.

Now, it’s simpler to manage a single fly.toml definition and I get a cleaner App dashboard.

BUT … it would be even better if I could use both approaches together, containers and machines via process groups.

In my case, it would be something like:

  • 4 process groups converted to 4 containers in 1 main small machine
  • 1 process group would keep as-is in 1 bigger machine
15

Testing this out today. I was able to get the example running with no problems, then started tweaking compose.yaml settings. First thing I tried was swapping out the nginx service with a custom build of Caddy. When I deploy it my image builds fine in Depot but when Fly tries to run it I get the following error:

Error: failed to update machine configuration for <machine_id> [app]: only one service can specify build, found 2 services with build

Removing the build: ./Dockerfile.caddy line from my compose file and using an unmodified Caddy image with image: caddy:latest fixes the error.

So, are we only allowed to have one service with a local build command in docker-compose on Fly right now? That seems like a very significant limitation.

16

Another question — Is there any solution for choosing Fly Secrets to be added to specific containers? I have a few different containers which need access to different secrets, and it would be really nice to be able to specify which secrets get imported via either fly.toml or the docker-compose. I some ideas being tossed around in this thread but it didn’t seem like anything was settled on.

[Update]: It seems to be worse than I thought. Fly appears to overwrites the environment variables based on its own rules as described here, which would be entirely fine and reasonable if I could configure which secrets get passed to each container (even just passing all of them to every container would work for my current use case). But since I can’t, I tried falling back to passing env secrets like normal via an env_file param in my docker-compose, but Fly overwrites those unless I put them in my fly.toml. Storing actual secrets in fly.toml is obviously a non-starter. The only other way I can see around is adding the secrets at build time, but as my previous comment mentioned, you can only build one container at a time right now.

I think I’m finally stuck — anybody have ideas on how to get secrets securely into multiple containers when using this new docker-compose setup?

[Update 2]: Doh I was overlooking the obvious alternative of sneaking my .env files on in a volume and then adding an init script to set them at startup. That method should be good enough to allow my experimentation here to continue, but there should really be a solution to this from Fly.

17

I’ll be back home on Monday and will help more at that point.

As far as building goes, that’s flyctl logic, and if we identify common use cases that logic can be improved.

As far as runtime goes (including secrets), ultimately what is build is a ContainerConfig, which you can pass directly ( Using Containers with Flyctl ) or (if we find a reasonable mapping) we can build a ContainerConfig for you from a compose.yml file.

1 Like
18

Hi I got this error when trying your example:
fly version
WARN WARNING the config file at ‘C:\Users\IslamZaoui\Desktop\test\fly.toml’ is not valid: json: cannot unmarshal string into Go struct field Build.build.compose of type appconfig.BuildCompose
fly.exe v0.3.171 windows/amd64 Commit: 41691d428650e68060b159d64e0d24e99fbeca6c BuildDate: 2025-08-15T06:42:25Z

19

I’ve updated the example to match the current syntax.

20

Hello,

I’ve tried multiple versions of the [build.compose] syntax but I seem to be getting a similar error:

Error: failed to update machine configuration for MACHINE_ID [app]: failed to parse compose file: yaml: unmarshal errors:
line 22: cannot unmarshal !!seq into map[string]string

fly v0.3.172 darwin/arm64 Commit: b389cead658c02081d3cc1384def330a54707d67 BuildDate: 2025-08-25T15:22:14Z

I also tried the exact copy from the commit (Convert Build.Compose from string to struct with auto-detection by rubys · Pull Request #4463 · superfly/flyctl · GitHub) and it seemed to throw that error

21

Can I get you to try demo 4 of the rate limited demo:

1 Like