(Node) How do I connect to my API app from my WEB app in Fly?

I have two Node apps on Fly

One is my Web, and the other is my API

I’ve tried using the static IPs from fly ips list as well as the DNS urls <appname>.internal and top1.nearest.to.<appname>.internal but I always get “ECONNREFUSED” from “fetch” everytime.

What am I missing?

The API doesn’t have any public services in the fly.toml but it does have it’s internal port specified, which is what I’m using to try and connect to it.

TL;DR
How do I setup my fly.toml on the API side so that it’s an internal private API only, and then what URLs am I using to make HTTP calls to it from my Fly WEB app?

Off the top of my head … I suspect it may be because of IPv6. I suspect those private .internal domains return IPv6 but Node does not seem to like resolving those.

Do you use Alpine Linux as the base in your Dockerfile? That can definitiely cause issues with DNS (The Nodejs in IPv6 only networks problem). I’ve had that. I switched to Slim and the DNS issues I had went away e.g I now use FROM node:16.13-slim.

As a test you could swap out the API hostname for its IP address, and see what happens e.g node.js - node-fetch with ipv6 proxy - Stack Overflow If the web app can’t connect to the API using its IPv6, but can using the IPv4, that would prove it.

Else it may be a conflict between the port the app is listening on in server.js (or whatever) and the fly.toml / Dockerfile, though it sounds like you’ve checked that.

Greg’s answers here seem good; I’ll just note that connectivity between apps on Fly.io, the kind where you use .internal addresses, aren’t influenced at all by fly.toml. It’s subtle, I know, we should document this better.

ECONNREFUSED suggests to me that your DNS lookups and your IPv6 connectivity is fine, because that’s the error Linux gives you when you call connect(2), it sends a SYN to the target host, and the host responds with an RST. It’s not what you want to have happen, but it does mean there’s bidirectional connectivity.

Are you sure the ports on your API service are right? Are you mapping them in fly.toml? Your internal communications won’t follow that mapping, so if your API is listening on 8080, that’s the port you want your other app to connect to on the internal address.

1 Like

@greg thank you for the detailed response, here is what I got!

I’ve confirmed that I’m using node:17-slim

Also, I tried swapped out the hostname for the static IPV4 address as you suggested and it still fails with ECONNRESET and reason: socket hang up so it must be how my API app is configured in it’s fly.toml I’m assuming?

Here is my fly.toml

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]
  PORT = "8080"
  PRIMARY_REGION = "iad"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

I don’t want it to be accessible externally, which is why I removed the 80 and 443 service port descriptions, and I’m assuming the internal_port = 8080 should cover me for having it still be available on that port internally?

@Thomas I’ve confirmed that my API app is running on 8080 and that is the port listed in fly.toml (see above)

After testing it with the static IPV4 address and it still doesn’t work I wonder if I’m missing something in my fly config?

As I told Greg, I want it only accessible internally, any suggestion on the config I posted above are appreciated!

Hey @danalloway, here’s a dumb suggestion - can you include the port in your fetch call, i.e., fetch('http://api-app.internal:8080')

Good question, my fetch definitely had the port number!

And I tried with the IPV4 and the private IPV6 static ips, nether worked.

Thanks for clarifying :slight_smile:
Here’s another one - is your API app listening on 0.0.0.0?

EDIT: But per this, it shouldn’t be required.

If host is omitted, the server will accept connections on the unspecified IPv6 address (::) when IPv6 is available, or the unspecified IPv4 address (0.0.0.0) otherwise.

It is (I’m using Fastify)

Hi @danalloway

Good news: I’ve got it working so it will be a case of seeing what I’m doing compared to your app to see what differs.

To test it, I’ve made a web and api Fly app, with Node 17, using Fastify. The web app calls the api app using the .internal hostname and I get a response back from it :rocket:

See https://fastify-web.fly.dev/call-api

(I’ll delete that app at some point, but it’s there as of now)

The files from the two apps to compare to yours - of course it’s pretty basic!

API

package.json

{
  "name": "fastify-api",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "type": "module",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "fastify": "^3.27.1"
  }
}

server.js

'use strict'

import Fastify from 'fastify'

const fastify = Fastify({
    logger: true
})

fastify.get('/api', async function (request, reply) {
    return { hello: 'this is a response from the api' }
})

const start = async () => {
    try {
        fastify.log.info('Starting api server ...');
        await fastify.listen(8080, '::')
    } catch (err) {
        fastify.log.error(err)
        process.exit(1)
    }
}
start()

fly.toml

# fly.toml file generated for fastify-api on 2022-02-05T17:38:04Z

app = "fastify-api"


[experimental]
  private_network = true

[[services]]
  internal_port = 8080
  protocol = "tcp"

  [services.concurrency]
    hard_limit = 50
    soft_limit = 25

  [[services.tcp_checks]]
    interval = 5000
    timeout = 2000

Dockerfile

FROM node:17-slim

USER node
RUN mkdir -p /home/node/app
WORKDIR /home/node/app

COPY --chown=node:node package.json .
COPY --chown=node:node package-lock.json .

RUN npm ci --only=production

COPY --chown=node:node . .

CMD ["node","server.js"]

… and then the web app calls that api. Its files are as follows …

WEB

package.json

{
  "name": "fastify-web",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "type": "module",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "fastify": "^3.27.1",
    "node-fetch": "^3.2.0"
  }
}

server.js

'use strict'

import Fastify from 'fastify'
import fetch from 'node-fetch';

const fastify = Fastify({
    logger: true
})

fastify.get('/', async (request, reply) => {
    return { hello: 'world' }
})

fastify.get('/call-api', async function (request, reply) {
    try {
        fastify.log.info('Call the api app using fetch ...');
        const results = await fetch('http://fastify-api.internal:8080/api')
        return results.json()
    } catch (err) {
        fastify.log.error(err)
        return { success: false }
    }
})

const start = async () => {
    try {
        fastify.log.info('Starting web server ...');
        await fastify.listen(8080, '::')
    } catch (err) {
        fastify.log.error(err)
        process.exit(1)
    }
}
start()

fly.toml

# fly.toml file generated for fastify-web on 2022-02-05T17:08:46Z

app = "fastify-web"


[experimental]
  private_network = true

[[services]]
  internal_port = 8080
  protocol = "tcp"

  [services.concurrency]
    hard_limit = 50
    soft_limit = 25

  [[services.http_checks]]
    interval = 5000
    method = "get"
    path = "/"
    protocol = "http"
    timeout = 2000
    tls_skip_verify = true

  [[services.ports]]
    handlers = ["tls", "http"]
    port = "443"

  [[services.tcp_checks]]
    interval = 5000
    timeout = 2000

Dockerfile

FROM node:17-slim

USER node
RUN mkdir -p /home/node/app
WORKDIR /home/node/app

COPY --chown=node:node package.json .
COPY --chown=node:node package-lock.json .

RUN npm ci --only=production

COPY --chown=node:node . .

CMD ["node","server.js"]

So … it could be your fetch needs http as the protocol and the port? For me, it did not work without the port appended (which makes sense, as I guess it defaults to 80 otherwise).

And/or change to listen on ‘::’ rather than 0.0.0.0 (which suggests to me IPv4, not IPv6).

I’m actually not sure if the private networking in my fly.toml is even needed: that may be a legacy of when they first added it, and I’ve left it in ever since. But I have that too.

Finally, if still no luck, you can double-check the private networking is working by adding an extra route in your fastify server to check it resolves e.g

import dns from 'dns';

fastify.get('/test-the-dns', async (request, reply) => {
    let records = [];
    try {
        records = await dns.promises.resolveTxt(`_apps.internal`)
        fastify.log.info(records)
    } catch (err) {
        fastify.log.error(err)
        return { "error": err }
    }

    let appset = records[0][0]
    if (appset == "") return [];

    return appset.split(",");
})

Good luck!

2 Likes

YESSSSSS! Greg you did it!

changing the Fastify API address to listen on from 0.0.0.0 to ::
and adding the private_network = true to my fly.toml did the trick!!

and, now we have a great fly.toml reference in here for other people trying to do that in the future, thank you SO much for helping me fix this super appreciate you!

2 Likes

private_network = true is a no-op now, just for what it’s worth. :slight_smile:

What made it work was binding to the IPv6 address ::.

6 Likes

@danalloway Yay! You’re welcome.

@thomas Ah … I suspected not but like I say, left it in case. Handy to know.

:rocket:

3 Likes