failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: 01H4XBY8K7S162TWHVZYVYWASE-nrt

Error: failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: 01H4XC187NAQZPKEF9T8TJD7VE-nrt

I am having the same issue since Friday. If I run deploy setting LOG_LEVEL, here are the logs I get on terminal.

LOG_LEVEL=debug flyctl deploy -a openindex --wait-timeout 240
DEBUG Loaded flyctl config from/Users/titocosta/.fly/config.yml
DEBUG determined hostname: "Titos-MacBook-M1-Pro.local"
DEBUG determined working directory: "/Users/titocosta/Developer/openindex/openindexnext"
DEBUG determined user home directory: "/Users/titocosta"
DEBUG determined config directory: "/Users/titocosta/.fly"
DEBUG ensured config directory exists.
DEBUG ensured config directory perms.
DEBUG cache loaded.
DEBUG config initialized.
DEBUG initialized task manager.
DEBUG skipped querying for new release
Update available 0.1.36 -> 0.1.52.
Run "flyctl version upgrade" to upgrade.
DEBUG client initialized.
DEBUG Config has metrics token
DEBUG app config loaded from /Users/titocosta/Developer/openindex/openindexnext/fly.toml
DEBUG --> POST https://api.fly.io/graphql

{
  "query": "query ($appName: String!) { appbasic:app(name: $appName) { id name platformVersion organization { id slug paidPlan } } }",
  "variables": {
    "appName": "openindex"
  }
}

DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (182.4ms)

{
  "data": {
    "appbasic": {
      "id": "openindex",
      "name": "openindex",
      "platformVersion": "machines",
      "organization": {
        "id": "aaV5JD7y9pVvoTGw31AbKZXeDbI3yX8m8",
        "slug": "personal",
        "paidPlan": false
      }
    }
  }
}
==> Verifying app config
Validating /Users/titocosta/Developer/openindex/openindexnext/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
DEBUG --> POST https://api.fly.io/graphql

{
  "query": "query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed status appUrl platformVersion organization { id slug paidPlan } postgresAppRole: role { name } imageDetails { repository version } } }",
  "variables": {
    "appName": "openindex"
  }
}

DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (121.29ms)

{
  "data": {
    "appcompact": {
      "id": "openindex",
      "name": "openindex",
      "hostname": "openindex.fly.dev",
      "deployed": true,
      "status": "deployed",
      "appUrl": "https://2a09:8280:1::1c:6d0",
      "platformVersion": "machines",
      "organization": {
        "id": "aaV5JD7y9pVvoTGw31AbKZXeDbI3yX8m8",
        "slug": "personal",
        "paidPlan": false
      },
      "postgresAppRole": null,
      "imageDetails": {
        "repository": "openindex",
        "version": null
      }
    }
  }
}
==> Building image
DEBUG trying remote docker daemon
DEBUG --> POST https://api.fly.io/graphql

{
  "query": "mutation($input: EnsureMachineRemoteBuilderInput!) { ensureMachineRemoteBuilder(input: $input) { machine { id state ips { nodes { family kind ip } } }, app { name organization { id slug } } } }",
  "variables": {
    "input": {
      "appName": "openindex",
      "organizationId": null
    }
  }
}

DEBUG {}
DEBUG <-- 500 https://api.fly.io/graphql (373.45ms)

{
  "errors": [
    {
      "message": "You hit a Fly API error with request ID: 01H4XF38AJKRQ0SNFJ2QP3E2B2-yyz",
      "extensions": {
        "code": "SERVER_ERROR",
        "fly_request_id": "01H4XF38AJKRQ0SNFJ2QP3E2B2-yyz"
      }
    }
  ],
  "data": {}
}
WARN Failed to start remote builder heartbeat: You hit a Fly API error with request ID: 01H4XF38AJKRQ0SNFJ2QP3E2B2-yyz
DEBUG --> POST https://api.fly.io/graphql

{
  "query": "\nmutation ResolverCreateBuild ($input: CreateBuildInput!) {\n\tcreateBuild(input: $input) {\n\t\tid\n\t\tstatus\n\t}\n}\n",
  "variables": {
    "input": {
      "appName": "openindex",
      "builderType": "remote",
      "clientMutationId": "",
      "imageOpts": {
        "buildArgs": {
          "NEXT_PUBLIC_...": "...",
          "NODE_VERSION": "18.14.2"
        },
        "buildPacks": null,
        "builder": "",
        "builtIn": "",
        "builtInSettings": null,
        "dockerfilePath": "",
        "extraBuildArgs": null,
        "imageLabel": "",
        "imageRef": "",
        "noCache": false,
        "publish": true,
        "tag": "registry.fly.io/openindex:deployment-01H4XF38MN4RS3TFM4CBTFN8FR",
        "target": ""
      },
      "machineId": "",
      "strategiesAvailable": [
        "Buildpacks",
        "Dockerfile",
        "Builtin"
      ]
    }
  },
  "operationName": "ResolverCreateBuild"
}
DEBUG {0x140009dc780}
DEBUG <-- 200 https://api.fly.io/graphql (95.14ms)

{
  "data": {
    "createBuild": {
      "id": "2578225",
      "status": "started"
    }
  }
}
DEBUG Trying 'Buildpacks' strategy
DEBUG no buildpack builder configured, skipping
DEBUG result image:<nil> error:<nil>
DEBUG Trying 'Dockerfile' strategy
DEBUG --> POST https://api.fly.io/graphql

{
  "query": "mutation($input: EnsureMachineRemoteBuilderInput!) { ensureMachineRemoteBuilder(input: $input) { machine { id state ips { nodes { family kind ip } } }, app { name organization { id slug } } } }",
  "variables": {
    "input": {
      "appName": "openindex",
      "organizationId": null
    }
  }
}

DEBUG {}
DEBUG <-- 500 https://api.fly.io/graphql (396.86ms)

{
  "errors": [
    {
      "message": "You hit a Fly API error with request ID: 01H4XF38S8TZQJ1W86JXDFB4JM-yyz",
      "extensions": {
        "code": "SERVER_ERROR",
        "fly_request_id": "01H4XF38S8TZQJ1W86JXDFB4JM-yyz"
      }
    }
  ],
  "data": {}
}
DEBUG result image:<nil> error:error connecting to docker: You hit a Fly API error with request ID: 01H4XF38S8TZQJ1W86JXDFB4JM-yyz
DEBUG --> POST https://api.fly.io/graphql

{
  "query": "\nmutation ResolverFinishBuild ($input: FinishBuildInput!) {\n\tfinishBuild(input: $input) {\n\t\tid\n\t\tstatus\n\t\twallclockTimeMs\n\t}\n}\n",
  "variables": {
    "input": {
      "appName": "openindex",
      "buildId": "2578225",
      "builderMeta": {
        "builderType": "",
        "buildkitEnabled": false,
        "dockerVersion": "",
        "platform": "",
        "remoteAppName": "",
        "remoteMachineId": ""
      },
      "clientMutationId": "",
      "finalImage": {
        "id": "",
        "sizeBytes": 0,
        "tag": ""
      },
      "logs": "error connecting to docker: You hit a Fly API error with request ID: 01H4XF38S8TZQJ1W86JXDFB4JM-yyz",
      "machineId": "",
      "status": "failed",
      "strategiesAttempted": [
        {
          "error": "",
          "note": "no buildpack builder configured, skipping",
          "result": "failed",
          "strategy": "Buildpacks"
        },
        {
          "error": "error connecting to docker: You hit a Fly API error with request ID: 01H4XF38S8TZQJ1W86JXDFB4JM-yyz",
          "note": "",
          "result": "failed",
          "strategy": "Dockerfile"
        }
      ],
      "timings": {
        "buildAndPushMs": 398,
        "buildMs": 398,
        "builderInitMs": 398,
        "contextBuildMs": -1,
        "imageBuildMs": -1,
        "pushMs": -1
      }
    }
  },
  "operationName": "ResolverFinishBuild"
}
DEBUG {0x1400090dc20}
DEBUG <-- 200 https://api.fly.io/graphql (94.12ms)

{
  "data": {
    "finishBuild": {
      "id": "2578225",
      "status": "failed",
      "wallclockTimeMs": 487
    }
  }
}
Error: failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: 01H4XF38S8TZQJ1W86JXDFB4JM-yyz

Hi both,

First try running fly doctor just to see if there are any issues at your end. It’s also worth making sure you have the latest CLI with fly version update.

Assuming neither are the issue, I’d then try doing a local build (to bypass the remote builder). At least for now, for debugging. Assuming you have Docker running locally, run fly deploy --local-only

See if that works. If so, it may be a temporary fault with the remote builder app.

I upgraded to version 0.1.53 and destroyed the builder, but I am still getting the same error on deploy

==> Verifying app config
Validating /Users/titocosta/Developer/openindex/openindexnext/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
WARN Failed to start remote builder heartbeat: You hit a Fly API error with request ID: 01H502N66PGYA7ZH6JTA72DNDS-yyz

Error: failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: 01H502N6M34BJDJSYKW9ZHPPF8-yyz

1fa6968a30c5040b09dde30d8dc43515
I am still getting the same error on deploy

Hi,

I don’t know but it appears like you may be having separate issues.

@titocosta You are getting a Fly API error. No idea why that would be I’m afraid. That will need someone from Fly to investigate.

@KIYO Your error message is a bit different: it says no space left, rather than an API error. I’d assume your builder app’s local disk is full. I had thought they auto-scale, but perhaps not. So the quick fix/hack for that is to destroy the builder app. That’s fine because next time you deploy, Fly will make you a new one, with an empty disk. Which should solve that particular issue. Try running fly apps destroy your-builder-name-here. Then deploy again.

You should be able to fix your deploys by deleting the builder app and redeploying. (I believe the host your builder is on ended up overprovisioned)

Hey! (I am from Rails)

I am also having this problem since yesterday. I commented over here:

I have already deleted the apps and images many times. I deleted the apps from the website dashboard.

After I do that, I do fly launch (using the same Dockerfile and fly.toml file… but the results are the same.

@greg , Here is me doing exactly what you said:

fly apps list
NAME                       	OWNER   	STATUS 	PLATFORM	LATEST DEPLOY
fly-builder-divine-sky-4238	personal	pending	machines	             	
spg                        	personal	pending	machines	             	

fly apps destroy spg
Destroying an app is not reversible.
? Destroy app spg? Yes
Destroyed app spg
fly deploy
==> Verifying app config
Validating /Users/dilraj/Documents/SPG/spg2/fly.toml
WARNING: Failed to fetch platform version: Could not find App "spg"
WARN the http_service section is ignored for Nomad apps
✓ Configuration is valid
--> Verified app config
Error: Could not find App "spg"

fly launch
Creating app in /Users/dilraj/Documents/SPG/spg2
An existing fly.toml file was found for app spg
? Would you like to copy its configuration to the new app? Yes
Scanning source code
Detected a Rails app
? Choose an app name (leaving blank will default to 'spg')
automatically selected personal organization: Dilraj Singh
App will use 'yyz' region as primary

Created app 'spg' in organization 'personal'
Admin URL: https://fly.io/apps/spg
Hostname: spg.fly.dev
Set secrets on spg: RAILS_MASTER_KEY
? Would you like to set up a Postgresql database now? No
? Would you like to set up an Upstash Redis database now? No
Wrote config file fly.toml
Validating /Users/dilraj/Documents/SPG/spg2/fly.toml
Platform: machines
✓ Configuration is valid

Your Rails app is prepared for deployment.

Before proceeding, please review the posted Rails FAQ:
https://fly.io/docs/rails/getting-started/dockerfiles/.

Once ready: run 'fly deploy' to deploy your Rails app.

fly deploy
==> Verifying app config
Validating /Users/dilraj/Documents/SPG/spg2/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
WARN Failed to start remote builder heartbeat: You hit a Fly API error with request ID: 01H52VKGKFFTBZXFW4Y7GQNQXM-yyz

Error: failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: 01H52VKH30HGGBXWMY9PBKQ3MQ-yyz

@dilraj Ah. You appear to be having the same issue that one of the guys above was having (“You hit a Fly API error”). That I don’t know how to fix. It would need someone from Fly to investigate to figure out why. As @allison also mentions: it’s worth trying deleting the builder with fly apps destroy fly-builder-its-name-here just in case there is a fault with that particular one/host. When you next deploy, you will get a new builder. Of course if the issue is elsewhere (like with the Fly API or region level) that will make no difference. But it’s worth trying while you wait.

Hey!

Re-doing fly apps destroy fly-builder-its-name-here ended up working. I was then able to do fly launch and fly deploy. :smile:

I am positive I ran this before! LOL, just happy to have it operating again! Thanks for the help :slight_smile:

1 Like