Cannot call outside APIs from my Bun+Elysia test app

connyrgbg · September 26, 2023, 6:07pm

I am working on a small prototype testing a few things, like loading content from a Strapi cms hosted on AWS.

An trying to fetch data from the Strapi it seems that I cannot call outside services from Fly.io.
I am using Strapi client library that I found for node/Bun that gives me this error:

2023-09-26T18:04:07.280 app[7811370f250558] ord [info] 🦊 Elysia is running at http://0.0.0.0:3000

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] 102 | if (err.code === "ENOTFOUND" || err.syscall === "getaddrinfo") {

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] 103 | error.status = err.code;

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] 104 | error.message = `The given url ${err.config.baseURL} is incorrect or invalid `;

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] 105 | error.name = err.syscall;

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] 106 | } else {

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] 107 | if (!err.response.data.error) {

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] ^

2023-09-26T18:04:14.723 app[48ed171cd17618] ord [info] TypeError: undefined is not an object (evaluating 'err.response.data')

2023-09-26T18:04:14.724 app[48ed171cd17618] ord [info] at _returnErrorHandler (/app/node_modules/@kmariappan/strapi-client-js/dist/strapi-client.js:107:11)

2023-09-26T18:04:14.724 app[48ed171cd17618] ord [info] at /app/node_modules/@kmariappan/strapi-client-js/dist/strapi-client.js:354:20

And using “fetch” like this:

const response = await fetch("http://xx.xx.xx.x:1337/api/recipes");
const html = await response.text(); // HTML string
console.log(html);

even causes the running process to fail and the VM to restart:

2023-09-26T17:55:59.495 app[7811370f250558] ord [info] 🦊 Elysia is running at http://0.0.0.0:3000

2023-09-26T17:56:16.059 proxy[7811370f250558] ord [error] could not complete HTTP request to instance: connection closed before message completed

2023-09-26T17:56:16.133 app[7811370f250558] ord [info] INFO Main child exited with signal (with signal 'SIGSEGV', core dumped? false)

2023-09-26T17:56:16.134 app[7811370f250558] ord [info] INFO Starting clean up.

2023-09-26T17:56:16.135 app[7811370f250558] ord [info] WARN hallpass exited, pid: 304, status: signal: 15 (SIGTERM)

2023-09-26T17:56:16.140 app[7811370f250558] ord [info] 2023/09/26 17:56:16 listening on [fdaa:3:2053:a7b:194:ac58:dc30:2]:22 (DNS: [fdaa::3]:53)

2023-09-26T17:56:17.135 app[7811370f250558] ord [info] [ 18.334233] reboot: Restarting system

2023-09-26T17:56:17.286 runner[7811370f250558] ord [info] machine did not have a restart policy, defaulting to restart

2023-09-26T17:56:17.574 app[7811370f250558] ord [info] [ 0.042203] PCI: Fatal: No config space access function found

2023-09-26T17:56:17.804 app[7811370f250558] ord [info] INFO Starting init (commit: 9fc6a62)...

2023-09-26T17:56:17.829 app[7811370f250558] ord [info] INFO Preparing to run: `/usr/local/bin/docker-entrypoint.sh bun src/index.tsx` as root

2023-09-26T17:56:17.839 app[7811370f250558] ord [info] INFO [fly api proxy] listening at /.fly/api

2023-09-26T17:56:17.847 app[7811370f250558] ord [info] 2023/09/26 17:56:17 listening on [fdaa:3:2053:a7b:194:ac58:dc30:2]:22 (DNS: [fdaa::3]:53)

2023-09-26T17:56:18.192 app[7811370f250558] ord [info] 🦊 Elysia is running at http://0.0.0.0:3000

Researching for a few hours now and out of ideas what that could be.
Initially I though it’s a DNS issue, that it could not convert the AWS domain to an IP, replaced the domain with IP in the API URL, but that didn’t change the issue.

What’s wrong? What can I do?

ben-io · September 26, 2023, 7:35pm

Looks like your bug is in accessing err.response.data, when either err or err.response is undefined.

connyrgbg · September 26, 2023, 9:11pm

I don’t think this is the problem, it’s just a bug in the error handling of that library.
Because locally this api call works and this is in the code path of an error.
I can surely double check this and produce a similar error locally.
And there is an error message earlier that says invalid base url, though the exact same works locally.

connyrgbg · September 27, 2023, 1:58pm

Reproduced the same error by giving the api call a random base url “asdf:1337/api”.

[0] asdf:1337/api/recipes?
[0] 102 |     if (err.code === "ENOTFOUND" || err.syscall === "getaddrinfo") {
[0] 103 |       error.status = err.code;
[0] 104 |       error.message = `The given url ${err.config.baseURL} is incorrect or invalid `;
[0] 105 |       error.name = err.syscall;
[0] 106 |     } else {
[0] 107 |       if (!err.response.data.error) {
[0]               ^
TypeError: undefined is not an object (evaluating 'err.response.data')
      at _returnErrorHandler (/Users/kguess/Development/beth/beth-fr/node_modules/@kmariappan/strapi-client-js/dist/strapi-client.js:107:11)
[0]       at /Users/kguess/Development/beth/beth-fr/node_modules/@kmariappan/strapi-client-js/dist/strapi-client.js:354:20
[0]       at processTicksAndRejections (:55:76)

The error is now locally the same and the same problem occurs with the actual error handling.
So the API wrapper has an issue with the error handling at this part, but that’s not the actual problem.
The actual problem is that a URL that works locally does not work on fly.io.
Even though the API call base url is now given with the IP, like “http://12.23.34.45:1337/api”.
So there is no DNS lookup involved, but still it fails.

What’s different in the container/VM of fly.io that this simple call would not work?
Do I need to enable outgoing calls anywhere?
Haven found anything about that in the documentation.

rubys · September 27, 2023, 3:16pm

The following works for me, does it work for you?

const source = "https://fly.io/static/images/brand/brandmark.svg";

Bun.serve({
  async fetch(req) {
    const response = await fetch(source);
    return new Response(await response.text(), {
      headers: {
        'Content-Type': response.headers.get('content-type')
      }
    })
  }
});

connyrgbg · September 27, 2023, 3:47pm

Good idea!
Both locally and from the fly.io hosted instance I get a downloaded SVG file.

connyrgbg · September 27, 2023, 4:00pm

After I replaced my ajax GET handler code with:

export async function get() {
  const source = "https://fly.io/static/images/brand/brandmark.svg";
  const response = await fetch(source);
    return new Response(await response.text(), {
      headers: {
        'Content-Type': 'image/svg' //: response.headers.get('content-type')
      }
    })
  return response;
}

I get the SVG delivered from the local instance.

And a 502 bad gateway on the Ajax request on the fly.io hosted instance, means: browser calls the above as Ajax request and the response is 502. That’s the response of the fetch call above … so I cannot reach fly.io from that point in the code. WTF, why?

connyrgbg · September 27, 2023, 4:22pm

If I refresh the page then it sometimes works and it sometimes doesnt.
Works = svg loaded, doesn’t = 502.

When refreshing very often it works most of the times to always now.

What’s going on!?

connyrgbg · September 27, 2023, 4:30pm

Now I replaced the get of fly.io svg image with the same fetch call to my Strapi api.
And the same happens: 502 on the first 2-3 request, then it suddenly works on returns the JSON string successfully.
That’s a step forward, though I am quite puzzled that the calls “sometimes” don’t work or don’t work after starting freshly.
Why is that?

Now will try that Strapi api wrapper library if that behaves the same in the end.

connyrgbg · September 27, 2023, 4:43pm

Ok, using the API library for Strapi I am back to the error described in my orignal question.
So for now: I can call the API with fetch, but not with the library.

That library:

Using Axios for the call.

  async get(): Promise<StrapiApiResponse<T>> {
    if (this.debug) {
      // eslint-disable-next-line no-console
      console.log(this.url);
    }
    return new Promise<StrapiApiResponse<T>>((resolve) => {
      if (this.isNotUserContent) {
        this.httpClient
          .get<StrapiApiResponse<T>>(this.url)
          .then((res) => {
            resolve(this.normalizeData ? this._returnDataHandler(res.data) : res.data);
          })
          .catch((err) => {
            if (err) {
              resolve(this._returnErrorHandler(err));
            }
          });
      }
      if (!this.isNotUserContent) {
        this.httpClient
          .get<T>(this.url)
          .then((res) => {
            resolve({ data: res.data, meta: undefined });
          })
          .catch((err) => {
            if (err) {
              resolve(this._returnErrorHandler(err));
            }
          });
      }
    });
  }

from here:

github.com

kmariappan/strapi-client-js/blob/main/src/lib/strapi-filter-builder.ts#L26


      
            debug: boolean,
            private isNotUserContent: boolean
          ) {
            super(url);
            this.debug = debug;
            this.url = url;
            this.httpClient = axiosInstance;
            this.normalizeData = normalizeData;
          }
          
          async get(): Promise<StrapiApiResponse<T>> {
            if (this.debug) {
              // eslint-disable-next-line no-console
              console.log(this.url);
            }
            return new Promise<StrapiApiResponse<T>>((resolve) => {
              if (this.isNotUserContent) {
                this.httpClient
                  .get<StrapiApiResponse<T>>(this.url)
                  .then((res) => {
                    resolve(this.normalizeData ? this._returnDataHandler(res.data) : res.data);

connyrgbg · September 27, 2023, 5:09pm

Looks like it could be this issue here:

github.com/nodejs/node

http.get returns "getaddrinfo ENOTFOUND" error

opened 01:57PM - 25 Feb 16 UTC

closed 01:41PM - 27 Feb 16 UTC

T94T

http dns

Hi all, I'm trying to fix following problem since five days. When I'm doing a - …http.get("http://url/", cb); or - http.get(options, cb); there's this error coming back: ``` HTTP 5021: SOCKET ERROR: getaddrinfo ENOTFOUND _host_ _host_:80 Error: getaddrinfo ENOTFOUND _url_ _url_:80 at errnoException (dns.js:26:10) at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26) Got error: getaddrinfo ENOTFOUND _host_ _host_:80 NET 5021: destroy undefined NET 5021: destroy NET 5021: close NET 5021: close handle NET 5021: destroy NET 5021: already destroyed, fire error callbacks NET 5021: emit close HTTP 5021: CLIENT socket onClose HTTP 5021: removeSocket _host_:80: destroyed: true HTTP 5021: HTTP socket close ``` When doing a https request to google.com: `Got error: connect ENETUNREACH 2a00:1450:400a:806::1014:443 - Local (:::0)` I can definitely say it's NOT a DNS or a PROXY issue. What I testet so far: - changed hosts file, so no dns lookup is required - changed resolv.conf to use other DNS servers - **curl and wget requests do work** - **ping to destination host works** - reinstalled complete system - same node.js code works on a raspbian system node.js sample code: ``` var http = require('http'); var options = { host: '_host_', port: 80, path: '/' }; http.get(options, function(res) { console.log("Got response: " + res.statusCode); res.on("data", function(chunk) { console.log("BODY: " + chunk); }); }).on('error', function(e) { console.log("Got error: " + e.message); }); ``` - **Version**: v5.5.0 - **Platform**: Linux buildroot 4.1.15 armv6l (raspberrypi)

ben-io · September 27, 2023, 7:28pm

What URL are you trying to call?

connyrgbg · September 27, 2023, 7:42pm

It’s an AWS instance where I deployed a Strapi CMS.
Same problem regardless whether I put the ip or the domain name.
With “fetch” it works, with Axios it’s failing.

const options: StrapiClientOptions = {
url: ‘http://ec2-…-…-…-…eu-central-1.compute.amazonaws.com:1337/api’,
apiToken: ‘’, // Built in API token,
normalizeData: true, // Normalize Unified response Format. default - true
headers: {}, // Custom Headers
persistSession: false, // Persist authenticated token in browser local storage. default -false
debug: true,
};

connyrgbg · September 27, 2023, 9:08pm

OMG!!
I have done a modification to the docker file of my Bun setup.
That caused an update to the Bun version used: 0.6.14 → 1.0.2

And now it simply works!

So it might have been that DNS issue I have linked more above, the Bun version 0.6 using old libraries or configs or something.

connyrgbg · September 27, 2023, 9:12pm

Did the cross check and reverted the Bun version to 0.6.14 and the problem appears again.

system · October 4, 2023, 9:13pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
can't fetch external urls from node (timeout error)	2	7547	December 2, 2022
EHOSTUNREACHED Error when reaching external services Questions / Help	2	23	November 17, 2024
random 502 bad gateway errors with Strapi endpoints - app deployed on Fly.io + postgres database on Supabase Build debugging postgres , databases	7	1167	July 8, 2023
Trouble Connecting to Non-HTTP Service on Fly.io proxy	2	30	January 3, 2025
Singapore region network errors Questions / Help	6	727	March 2, 2023

Cannot call outside APIs from my Bun+Elysia test app

Related topics