Use case for Fly Machines?

Hello, first time Fly.io explorer here trying to figure out if my use case is right for Fly’s offerings – I’d love some feedback :pray:

Currently using Apify to run my containers, but I’d like to run it on Firecracker VMs.

From my frontend:

  • /init api endpoint
    • creates a new instance of a worker via Apify
    • returns the WS URL associated with the worker container (.../worker/{id}/)
  • .../worker/{id} endpoint (for websocket connection)
    • proxies websocket requests between the container and frontend

The worker container:

  • has a websocket server for 2-way communication

I got a test version of my “worker” app running on Fly, and I’m able to spin up multiple instances and connect via the custom url (app-name.fly.dev) pointing to the websocket port – but I’m not able to figure out how to direct messages to specific machines.

How would I do this?

It isn’t clear to me whether the worker “container” (we don’t actually run containers here) is accessible over the internet. If it is, Dynamic Request Routing · Fly Docs will help; otherwise Private Networking · Fly Docs will point you in the right direction for private networking.

1 Like

Since the api service (also running on fly) is proxying the WS messages, private networking seems like the way to go. Thank you for pointing in that direction, I’m a step closer now.

From the docs I’ve managed to verify that I can ping the private_ip from within the org, but the app-name.internal thing doesn’t seem to work. Am I missing something?
I think that’s the only way I’ll be able to connect to the machine via websocket?

What’s wrong when you try and use the .internal address? Additionally, you can avoid having to proxy the websocket connection at all by leveraging fly-replay. Check out this article from a bit ago for an example of the model: Real-Time Collaboration with Replicache and Fly-Replay · Fly

1 Like

If you have a Debian based image, and have installed inetutils-ping, then ping6 app-name.internal (assuming you substitute app-name with your real app name, of course) should work. If you have dnsutils installed, try:

dig +short -t txt _instances.internal | sed 's/" *//g; s/;/\n/g'

Oops I was confusing app name and machine name… I’ll give it another go with the right format here.

Still not getting any connections to the websocket. The hostname is good, but not sure what address to bind the ws server to.

  const httpServer = http.createServer();
  httpServer.addListener('upgrade', (_, res) => {res.end()});
  io.installHandlers(httpServer);
  httpServer.listen(port, process.env.FLY_MACHINE_ID || '0.0.0.0'); // ??? 'fly-local-6pn' doesn't seem to work.
  console.log(`Socket server listening on ${port}`);

Try:

httpServer.listen(port, '::');

I tried it out – it likely works, just no luck actually connecting to it.

Tried every variation of this with no success in finding an open host:port combo:
curl -vs http://{id}.vms.{app-name}.internal

Edit: able to successfully ping the correct host http://{id}.vms.{app-name}.internal now, but still not able to get a response from the webserver / websocket.

It is vm, not vms. And check your port number.

I have an app running in multiple regions, with nginx on port 3000, and the above works for me.

1 Like

@rubys I really appreciate your time and help.

Never had so many issues setting up websockets before. Any chance you can see what I’m doing wrong here? This all works locally without issues, so I’m expecting the issue to be with my fly setup.

import express from "express";
import { WebSocketServer } from "ws";
import http from "http";

const app = express();

app.get('/ping', (req, res) => {
  res.send('pong');
});

const wss = new WebSocketServer({
  noServer: true,
  path: "/websocket",
});

wss.on('connection', (ws) => {
  console.log('New WebSocket connection');

  ws.on('message', (message) => {
    console.log(`Received message: ${message}`);
    ws.send(`Echo: ${message}`);
  });

  ws.on('close', () => {
    console.log('Connection closed');
  });
});

// Integrate WebSocket server with the Express application
const server = http.createServer(app);
server.on('upgrade', (request, socket, head) => {
  wss.handleUpgrade(request, socket, head, (ws) => {
    wss.emit('connection', ws, request);
  });
});

const port = process.env.PORT || 8080;
server.listen(port, "::", undefined, () => {
  console.log(`Server is listening on :${port}`);
});
app = "testauther"
primary_region = "sea"

[[services]]
  protocol = "tcp"
  internal_port = 8080
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

  [[services.ports]]
    port = 80
    handlers = ["http"]

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]

  [[services.tcp_checks]]
    interval = "10s"
    timeout = "2s"
    restart_limit = 0

Verified on the machine that port 8080 is exposed / being listened on using netstat -plntu

Then on a separate app I try connecting using websocat. Tried with ws / wss / port 80 – all the same.

> ./websocat ws://e78434ef401e83.vm.testauther.internal/websocket
websocat: WebSocketError: I/O failure
websocat: error running

Nothing is obviously wrong with the code to me. I’ve created express apps with websockets before and never had to handle the upgrade myself, but if that is working locally it should be fine.

What catches my eye is the fly.toml has [[services]] instead of [http_service]. If that is intentional, that’s fine; but otherwise it may indicate that you have an old flyctl version and/or are still on apps v1.

Before I look deeper, can we try it the other way? I can provide you with a working app that uses express and websockets that you can deploy to multiple regions. In an empty directory run:

npx --yes @flydotio/node-demo@latest --express --ws
fly launch
fly deploy

More information on the demo can be found here: Vanilla with Candy Sprinkles · The Fly Blog

1 Like

I think you need to include the internal port: <id>.vm.<app-name>.internal:8080.

The [[services.ports]] section configures the routing at the fly-proxy level for external requests. The proxy receives requests and forwards them to your application, which is listening on port 8080. Requests within the private network do not pass through the proxy [unless you are using Flycast - Private Load Balancing].

1 Like

@rubys I’ve just been copying parts of examples / blogs that I’m running into! I appreciate the demo project, it definitely provides a better foundation for me.

Looks like this is what was missing! Thank you @containerops.