Problem routing to correct instance via fly-force-instance-id

, , ,
1

Hi everyone!

I have a quite a pickle in my game server routing handling. I’m creating fly machines on demand using the machines api. They are created fine, and everything works really well… If I have only one machine running. I’m using fly-replay and fly-force-instance-id to route my requests to the correct game server instance. There are basically two requests that happen:

  • A regular http get request that gets the game map for the user. This is routed to random game server as well.
  • A ws upgrade request. This is routed to random game server.

This is my gateway application, it is really simple and as I said, it works, but the problem is only that the replay goes to wrong machine :frowning: Any ideas what the problem could be?

import { Hono } from 'hono'
import { cors } from 'hono/cors'
import { csrf } from 'hono/csrf'

const app = new Hono()
app.use(
  '*',
  cors({
    origin: [
      // Origins
    ],
  })
)
app.use(
  '*',
  csrf({
    origin: [
      // Origins
    ],
  })
)
app.onError((error, c) => {
  console.error(`${error}`)
  return c.json({ error, message: error.message || 'Custom Error Message' }, 500)
})

app.get('*', (c) => {
  const region = c.req.header('region') ?? c.req.query('region')
  const instance = c.req.header('instance') ?? c.req.query('instance')
  if (!region || !instance) {
    throw new Error('Machine not found')
  }
  console.log(
    'replaying',
    `region=${region};app=floating-islands-game-server;fly-force-instance-id=${instance}`
  )
  c.header(
    'fly-replay',
    `region=${region};app=floating-islands-game-server;fly-force-instance-id=${instance}`
  )
  return c.body('OK')
})

export default {
  fetch: app.fetch,
  port: 8080,
  hostname: '0.0.0.0',
}

I’m getting seemingly ok logs, but they seem to not go to the right machine:

2024-12-06T12:26:31.547 app[48e5376b3d9128] ams [info] replaying region=ams;app=floating-islands-game-server;fly-force-instance-id=9185954bd43518
2024-12-06T12:26:32.899 app[48e5376b3d9128] ams [info] replaying region=ams;app=floating-islands-game-server;fly-force-instance-id=9185954bd43518
2

Hey @Panuchka

This doesn’t look like correct fly-replay syntax.
In order to target a specific instance with fly-replay your app needs to respond with instance=<machine id> value:

region=${region};app=floating-islands-game-server;instance=${instance}
1 Like
3

Do you mean that I need to also include the instance parameter with fly-force-instance-id? I really need to make sure that the request goes to exact machine and it is not routed to any other machine in any case.

4

No, instead of it. fly-force-instance-id is not a valid syntax for fly-replay response. instance=<machine id> is the valid syntax.

Yeah, that’s how fly-replay to specific instance works.

6

Oooh I didn’t realize they were different headers altogether, I just thought they were options in the fly-replay header…

So as I want to make absolutely sure that the correct instance is being hit, I should use the fly-force-instance-id ? Do I still need the fly-replay header?

Like:

  c.header('fly-replay', `region=${region};app=floating-islands-game-server;instance=${instance};`)
  c.header('fly-force-instance-id', instance)
7

fly-force-instance-id is a request header. The clients of your app can set it to make sure their requests are routed to a specific machine, but the machine needs to be a part of the app that received this request.

fly-replay is a response header that instructs the proxy to re-route the original request to a different app/region/instance.

So if you control the clients as well you can use fly-force-instance-id and avoid the round-trip to your app (for fly-replay). Though, keep in mind that if a client asks to route its requests to a machine that, for example, no longer exists, the request will just fail.

So I’d say it’s better to rely on fly-replay as your app can keep track of instances and re-route requests to the ones that are definitely valid.

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.