Route to specific instance (machine) of app?

[couchcode]

I am exploring using Fly for game/room servers - kinda like how one can use Durable Objects, but with more power / cheaper for long running servers.

The machine is a NodeJS server that accepts web sockets, and everyone connecting to that machine is in the same game. Like an in-memory chat room - no db or anything.

When a group of friends want to play a game, I want to spin up an instance/machine of a Fly App for them and get an https:// url for that specific machine so the browser can connect to that machine directly via websocket and the players can communicate with just each other.

I tried the FLY_PUBLIC_IP but I couldn’t connect to it - I don’t really want to pay for a static IP, because I really just want exactly what Fly already does with providing an https url for my App… except not for the whole App routing to any random-ish instance… I want to get an https url for a very specific instance.

Any way to accomplish this? It looks like I’d have to create a new App for each instance, but that doesn’t seem like what I should be doing.

[couchcode]

I found this Session Affinity (a.k.a. Sticky Sessions) · Fly Docs

Unfortunately browser websockets can’t set request headers (my first choice), so I can’t use fly-force-instance-id to immediately go to the right place.

BUT! I was able to have the wrong instance tell the browser’s websocket connection to try instead going to the correct instance by responding with fly-replay

I would still love to have some way to immediately be routed to the correct instance (maybe a special url we can opt-into, or a cookie maybe?), but this does work (even with a bit of overhead).

Anyway, here’s the code the worked for anybody else trying to use a NodeJS websocket:

 const server = http.createServer(app);
const wss = new WebSocket.Server({ noServer: true });

// Get current Fly machine ID (https://fly.io/docs/machines/runtime-environment/#environment-variables)
const FLY_INSTANCE_ID = process.env.FLY_MACHINE_ID || 'no-instance-id';


// Handle WebSocket upgrade requests for fly-replay
server.on('upgrade', (request, socket, head) => {
  // Parse query parameters
  const url = new URL(request.url, `http://${request.headers.host}`);
  const targetInstance = url.searchParams.get('instance');
  console.log('url', url.href)
  console.log('targetInstance', targetInstance)

  // If an instance is specified and it doesn't match ours, replay to the correct instance
  // https://fly.io/docs/blueprints/sticky-sessions/
  if (targetInstance && FLY_INSTANCE_ID && targetInstance !== FLY_INSTANCE_ID) {
    console.log(`Replaying WebSocket connection from ${FLY_INSTANCE_ID} to ${targetInstance}`);
    socket.write('HTTP/1.1 307 Temporary Redirect\r\n' +
                 `fly-replay: instance=${targetInstance}\r\n` +
                 '\r\n');
    socket.destroy();
    return;
  }

  console.log(`Accepting WebSocket connection on instance: ${FLY_INSTANCE_ID || 'local'}`);
  
  // Let the WebSocket server handle the upgrade
  wss.handleUpgrade(request, socket, head, (ws) => {
    wss.emit('connection', ws, request);
  });
});

[rubys]

This is the approach I’ve used for years (details differ, I started with nginx and now in go), but it works fine. It adds at most milliseconds to routing, generally unnoticeable particularly for log running web sockets.