Routing internal request failed with getaddrinfo ENOTFOUND

Hello! Apologies if this is a simple request, I am quite new to working with Fly.

I have a node (remix) application running on Fly.

It has a few internal routes that initiate requests to other internal routes.

When I have tried to trigger these routes (i.e though a POST request with postman), I am getting a 500 response with the following error (note: ipv6 address truncated):

2022-07-08T21:38:35.185 app[8c90c601] ewr [info] FetchError: request to http://[fdaa:0:5832:...:...:1:3681:2]:8080/_content/update-content?_data=routes%2F_content%2Fupdate-content failed, reason: getaddrinfo ENOTFOUND [fdaa:0:5832:...:...:1:3681:2]

It seems like the address is not found. Curious if you have any thoughts on what might be happening here? This is the piece of TypeScript code where this request is dispatched:

  const body = await request.text()
  const address = `global.${getRequiredEnvVar('FLY_APP_NAME')}.internal`
  const ipv6s = await dns.promises.resolve6(address)

  const urls = ipv6s.map(ip => `http://[${ip}]:${getRequiredEnvVar('PORT')}`)

  const queryParams = new URLSearchParams()
  queryParams.set('_data', 'routes/_content/update-content')

  const fetches = urls.map(url =>
    fetch(`${url}/_content/update-content?${queryParams}`, {
      method: 'POST',
      body,
      headers: {
        auth: getRequiredEnvVar('REFRESH_TOKEN'),
        'content-type': 'application/json',
        'content-length': Buffer.byteLength(body).toString(),
      },
    }),
  )

  const response = await Promise.all(fetches)

  return json(response)

Try comparing the output of fly dig aaaa global.<appname>.internal with fly ips private -a <appname>.

This happens when the IP is not available. This could either mean the VM has shut down and you got a stale DNS response. It can take a few seconds for DNS changes to propagate. It can also happen if we haven’t removed a DNS entry for an app properly. This could take a few hours to resolve itself.

For what I think you’re doing, it’s best to capture errors. You’ll get an error like that during a deploy, or when one instance crashes and hasn’t come back up yet, or a number of other reasons.

Hi @kurt, thanks for the help.

I confirmed the output of fly dig aaa global.<appname>.internal matches the output of fly ips private -a <appname>.

I’m still seeing this same error getaddrinfo ENOTFOUND today when making internal requests.

If it’s any help, the project I am trying to build is this repo: GitHub - Girish21/speed-metal-stack: The Remix Blog Stack for deploying to Fly with MDX, SQLite, testing, linting, formatting, etc.

and this is the script that initiates the (currently failing) request: speed-metal-stack/refresh-content.ts at main · Girish21/speed-metal-stack · GitHub

Just wanted to bump this as I am still running into the above issue.

One thing I noticed is the IPv6 address shown on my dashboard, and the IPv6 address shown when doing fly ups private -a <appname> appear to be different IP’s. is this expected?:

I’m essentially trying to do this from the example privatenet docs - get the IP address of all instances, and then performing a POST to each instance. Not sure why this is failing.

I’m not sure what’s going on with your app currently, but I can help you with one of your questions:

it looks like the dashboard is configured to show your app’s public IP addresses, whereas fly ips private -a <app-name> will show the app’s 6pn address. so this is normal.

1 Like

Thanks @eli ! Glad to know that is expected, at least.

Honestly I’m a bit stumped on this. I’m primarily a frontend guy, so a lot of the setup for Fly is pretty new to me. According to the repo maintainer, this code previously worked as expected and hasn’t been changed on his side, so seems like something has changed on the fly.io side to possibly break this?

@eli sorry for the necro. I have a situation where my fly dig aaa global.<appname>.internal DOES NOT match the output of fly ips private -a <appname>

What should I do? I have tried to read the docs on private networking but it’s very sparse.