6PN failing?

I’m getting this error in my there-nur which tries to connect to noor-nats service.

2021-10-08T14:29:04.478460531Z app[75a2dffa] lhr [info] Failed to connect to Nats Error: getaddrinfo ENOTFOUND noor-nats.internal

Opening via WireGaurd works:
CleanShot 2021-10-08 at 18.01.08

I tried restarting both a couple times. Could you make sure it’s all good from your side? I don’t know what else to try without knowing that. This is a critical production node :pray:

Seems like lhr region has the issue.
Update: No doesn’t relate to the region. Seems like a private networking issue
Update 2: I had to scale to 1 and run the app in iad region so it’s working. lhr and nrt have issues with private networking. lhr connects once, then after restart doesn’t. It’s really weird.

Can you give us some more details on what you were seeing? You saw:

getaddrinfo ENOTFOUND noor-nats.internal

From your app. How long had noor-nats been running, had it been deployed recently, etc?

Also what runtime are you connecting from?

It’s possible there’s an issue syncing DNS records around when VMs change, but we’ll need more details to troubleshoot.

1 Like

Yes. noor-nats has been running for months, but I did a scale count change a couple of hours earlier to this issue.

I’m connecting from Node.js to it. It’s been fine. It’s also fine in some regions, sometimes. For example now in iad it’s fine. But it was having issues in nrt, and then lhr. I kept restarting, changing scale, regions, etc, and it worked for example for 2 regions, but one was failing, etc.

Is your Node app using an alpine image by chance?

We’ve seen this error from node apps somewhat frequently, even when DNS resolution is working fine (dig from the vm works, node is unhappy). I think there’s some strange bug or conflict with node and IPv6 DNS lookups like this.

Yes I use the alpine image: mhart/alpine-node:16.0.0. What should I use?

That could be it, it’s worth trying a slim image.

Here are more details: docker - Does Alpine have known DNS issue within Kubernetes? - Stack Overflow

1 Like

Cool, will swap it and report back.

Deployed using node:16-slim. So far no issues in 3 regions. If it starts failing again, will report here. Thank you.