Running MCPs on and with fly.io

Fresh Produce
1

Kurt recently spent 30 Minutes with MCP. If you are unfamiliar with MCPs, they basically give LLMs (AI tools like Claude, Cursor, and Copilot) access to the outside world. Kurt closed his post with:

Local MCP servers are scary. I don’t like that I’m giving a Claude instance in the cloud the ability to run a native program on my machine. I think fly logs and fly status are safe, but I’d rather know it’s safe. It would be, if I was running flyctl in an isolated environment and not on my local machine.

So I took a look into that. here are three transports defined: stdio (a.k.a. local), SSE Streamable HTTP (a.k.a. remote), and Custom (a.k.a. undefined).

Current status: there are a lot of existing servers, client tools (like Claude Desktop, Cursor, and Copilot) either only support stdio, or perhaps they support stdio and partially support a prior version of the remote protocol, and finally, stdio servers are the easiest to write.

So, I figured I would write a stdio MCP that proxies a wrapper MCP (basically a slimmed down and streamlined Streamable HTTP server), which in turn forwards requests to a stdio MCP running on a remote server.

Basically something like this:

image
image1530×188 9.78 KB

It turns out that it runs amazing well, so I merged it into flyctl and shipped it experimentally.

If anybody wants to try this out, here is a scenario that works:

In an empty directory, create a Dockerfile:

FROM node:slim
COPY --from=flyio/flyctl flyctl /usr/bin
EXPOSE 8080
VOLUME /data
CMD [ "/usr/bin/flyctl", "mcp", "wrap", "--mcp=npx", "--", \
  "-f", "@modelcontextprotocol/server-filesystem", "/data/" ]

I’m using the Node.js slim image, and copying in flyctl, defining my port and the mount point for a volume.

I’m wrapping the Filesytem MCP Server and limiting its access to the data contained on a fly volume.

Run fly launch, and accept the defaults.

Configure your favorite LLM. Here’s claude_desktop_config.json for example:

{
  "mcpServers": {
    "filesystem": {
      "command": "/Users/rubys/.fly/bin/flyctl",
      "args": [
         "mcp",
         "proxy",
         "--url=https://mcp.fly.dev/"
       ]
    }
  }
}

Obviously adjust the flyctl path and the value of the --url.

Example session:

image
image862×1690 170 KB

Verification of results:

rubys@rubym4p mcp % fly ssh console
Connecting to fdaa:0:d445:a7b:e5:9afc:714d:2... complete
root@d8d9930b293628:/# cd /data
root@d8d9930b293628:/data# ls
lost+found
root@d8d9930b293628:/data# touch killroy
root@d8d9930b293628:/data# ls
killroy  log.txt  lost+found
root@d8d9930b293628:/data# cat log.txt
Sunday, April 13, 2025root@d8d9930b293628:/data#

Now putting a HTTP server that can access a volume on the internet without any access control is foolhardy. So to lock this down, set FLY_MCP_USER and FLY_MCP_PASSWORD secrets on the app, and add --user and --password arguments on the flyctl proxy command in your LLM config file.

You can find out a bit more at fly mcp. For good measure, I also shipped Kurt’s experiment at fly mcp server - you can use it to examine the logs and status of your own applications.

This is potentially just the beginning. Some things I’m planning to explore (perhaps some will even work out):

  • If you are developing an MCP, we can perhaps make fly launch smarter and wrap it automatically for you.
  • If you are running somebody else’s MCP, they probably gave you a command starting with npx or perhaps uv run. We should be able to build a dockerfile for you. Since you don’t have source, fly launch won’t have much to work with, but perhaps we could have a fly mcp run command.
  • Running an MCP to access a volume that isn’t connected to your application is of limited use. It would be much better to run the MCP as a container; a sidecar to your application, if you will. This would enable, for example, running an MCP to access and analyze sqlite3 databases.
  • Since we have flyctl on both sides of the connection, perhaps we could eliminate exposing the MCP to the internet at all - we could make use of our existing wireguard infrastructure and tunnel our way in.
  • Kurt implemented two tools. We have a vast CLI and Machines API, so we can do more. Imagine world where you are in Cursor and tell it to deploy your application, the deployment fails and Cursor proceeds to analzye the failure and propose (or perhaps even directly implement) solutions much like how it handles unit test failures today.

Those are just some ideas. Give it a try. If you spot problems, or if any of these ideas interest you, or you have other ideas on this topic you want to share, let us know!

6 Likes