DEBUG result image:<nil> error:error connecting to docker: Couldn't allocate volume, not enough compute capacity left in ord
How am I supposed to deploy? I’m on an M1 chip so --local-only isn’t an option either…
Deleting the builder and trying again doesn’t work.
Deleting the app and relaunching in a different region also doesn’t work.
Hi,
It does seem to be a current issue.
Unfortunately Fly will try and create a remote builder in a region close to you, so I assume that is ORD. Using a different region for the app indeed does not fix that, since it is based on your location rather than its.
If you can’t build locally then I guess it will either be a case of waiting for more capacity to be added (may be minutes/hours for them to add?) or using a VPN to create a builder somewhere else:
I see. I waited an hour or so here and now noticing 500s. Something definitely appears to be going wrong in the ORD region:
Error: failed to fetch an image or build from source: error connecting to docker: server returned a non-200 status code: 500
@pepioppe:
Could you please rerun your deploy with
LOG_LEVEL=debug fly deploy | tee deploy.log
This will store a copy of the detailed log in deploy.log - it’s likely it’ll error out anyway but this way you have a capture of the logs, I’d love if you could share that with us.
If it blows up as it’s likely to happen, the fastest way to get around this is to identify your builder app in
fly apps list
it’ll be named fly-builder-something. Then, destroy it unceremoniously:
fly app destroy fly-builder-somethingsomething
at this point you can retry your deploy and a new app will be created for that, and it will hopefully work this time.
Regards,
Destroying the builder and unfortunately results in the same behavior. I am however getting 500s now.
deploy.log:
DEBUG Loaded flyctl config from/Users/peter.crist/.fly/config.yml
DEBUG determined hostname: "Rebeccas-MacBook-Air.local"
DEBUG determined working directory: "/Users/peter.crist/go/src/github.com/peter-crist/kipbot"
DEBUG determined user home directory: "/Users/peter.crist"
DEBUG determined config directory: "/Users/peter.crist/.fly"
DEBUG ensured config directory exists.
DEBUG ensured config directory perms.
DEBUG cache loaded.
DEBUG config initialized.
DEBUG skipped querying for new release
DEBUG client initialized.
DEBUG app config loaded from /Users/peter.crist/go/src/github.com/peter-crist/kipbot/fly.toml
DEBUG --> POST https://api.fly.io/graphql
DEBUG {
"query": "query ($appName: String!) { appbasic:app(name: $appName) { id name platformVersion organization { id slug paidPlan } } }",
"variables": {
"appName": "kipbot"
}
}
DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (520.69ms)
DEBUG {
"data": {
"appbasic": {
"id": "kipbot",
"name": "kipbot",
"platformVersion": "machines",
"organization": {
"id": "zQpLKVXNGapwgH61jXMLyk5zbLc3jeX2K",
"slug": "personal",
"paidPlan": false
}
}
}
}
==> Verifying app config
DEBUG Starting task manager
DEBUG Config has metrics token
--> Verified app config
Validating /Users/peter.crist/go/src/github.com/peter-crist/kipbot/fly.toml
Platform: machines
✓ Configuration is valid
DEBUG --> POST https://api.fly.io/graphql
DEBUG {
"query": "query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed status appUrl platformVersion organization { id slug paidPlan } postgresAppRole: role { name } imageDetails { repository version } } }",
"variables": {
"appName": "kipbot"
}
}
DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (187.49ms)
DEBUG {
"data": {
"appcompact": {
"id": "kipbot",
"name": "kipbot",
"hostname": "kipbot.fly.dev",
"deployed": false,
"status": "pending",
"appUrl": null,
"platformVersion": "machines",
"organization": {
"id": "zQpLKVXNGapwgH61jXMLyk5zbLc3jeX2K",
"slug": "personal",
"paidPlan": false
},
"postgresAppRole": null,
"imageDetails": {
"repository": "unknown",
"version": "unknown"
}
}
}
}
==> Building image
DEBUG trying remote docker daemon
DEBUG --> POST https://api.fly.io/graphql
DEBUG {
"query": "mutation($input: EnsureMachineRemoteBuilderInput!) { ensureMachineRemoteBuilder(input: $input) { machine { id state ips { nodes { family kind ip } } }, app { name organization { id slug } } } }",
"variables": {
"input": {
"appName": "kipbot",
"organizationId": null
}
}
}
DEBUG {}
DEBUG <-- 500 https://api.fly.io/graphql (12.43s)
DEBUG {
"errors": [
{
"message": "You hit a Fly API error with request ID: 01HCJCK4TAVNDGZAP8ZJ60SNDW-ord",
"extensions": {
"code": "SERVER_ERROR",
"fly_request_id": "01HCJCK4TAVNDGZAP8ZJ60SNDW-ord"
}
}
],
"data": {}
}
WARN Failed to start remote builder heartbeat: server returned a non-200 status code: 500
DEBUG --> POST https://api.fly.io/graphql
DEBUG {
"query": "\n# @genqlient\nmutation ResolverCreateBuild ($input: CreateBuildInput!) {\n\tcreateBuild(input: $input) {\n\t\tid\n\t\tstatus\n\t}\n}\n",
"variables": {
"input": {
"appName": "kipbot",
"builderType": "remote",
"clientMutationId": "",
"imageOpts": {
"buildArgs": {},
"buildPacks": null,
"builder": "",
"builtIn": "",
"builtInSettings": null,
"dockerfilePath": "",
"extraBuildArgs": null,
"imageLabel": "",
"imageRef": "",
"noCache": false,
"publish": true,
"tag": "registry.fly.io/kipbot:deployment-01HCJCKGY9B4M8MEWW39NGAQ22",
"target": ""
},
"machineId": "",
"strategiesAvailable": [
"Buildpacks",
"Dockerfile",
"Builtin"
]
}
},
"operationName": "ResolverCreateBuild"
}
DEBUG {0x14000833b00}
DEBUG <-- 200 https://api.fly.io/graphql (115.76ms)
DEBUG {
"data": {
"createBuild": {
"id": "3849361",
"status": "started"
}
}
}
DEBUG Trying 'Buildpacks' strategy
DEBUG no buildpack builder configured, skipping
DEBUG result image:<nil> error:<nil>
DEBUG Trying 'Dockerfile' strategy
DEBUG --> POST https://api.fly.io/graphql
DEBUG {
"query": "mutation($input: EnsureMachineRemoteBuilderInput!) { ensureMachineRemoteBuilder(input: $input) { machine { id state ips { nodes { family kind ip } } }, app { name organization { id slug } } } }",
"variables": {
"input": {
"appName": "kipbot",
"organizationId": null
}
}
}
DEBUG {}
DEBUG <-- 500 https://api.fly.io/graphql (2.52s)
DEBUG {
"errors": [
{
"message": "You hit a Fly API error with request ID: 01HCJCKH2NR96B2WF0REMCJCY2-ord",
"extensions": {
"code": "SERVER_ERROR",
"fly_request_id": "01HCJCKH2NR96B2WF0REMCJCY2-ord"
}
}
],
"data": {}
}
DEBUG result image:<nil> error:error connecting to docker: server returned a non-200 status code: 500
DEBUG --> POST https://api.fly.io/graphql
DEBUG {
"query": "\n# @genqlient\nmutation ResolverFinishBuild ($input: FinishBuildInput!) {\n\tfinishBuild(input: $input) {\n\t\tid\n\t\tstatus\n\t\twallclockTimeMs\n\t}\n}\n",
"variables": {
"input": {
"appName": "kipbot",
"buildId": "3849361",
"builderMeta": {
"builderType": "",
"buildkitEnabled": false,
"dockerVersion": "",
"platform": "",
"remoteAppName": "",
"remoteMachineId": ""
},
"clientMutationId": "",
"finalImage": {
"id": "",
"sizeBytes": 0,
"tag": ""
},
"logs": "error connecting to docker: server returned a non-200 status code: 500",
"machineId": "",
"status": "failed",
"strategiesAttempted": [
{
"error": "",
"note": "no buildpack builder configured, skipping",
"result": "failed",
"strategy": "Buildpacks"
},
{
"error": "error connecting to docker: server returned a non-200 status code: 500",
"note": "",
"result": "failed",
"strategy": "Dockerfile"
}
],
"timings": {
"buildAndPushMs": 2528,
"buildMs": 2528,
"builderInitMs": 2528,
"contextBuildMs": -1,
"imageBuildMs": -1,
"pushMs": -1
}
}
},
"operationName": "ResolverFinishBuild"
}
DEBUG {0x14000d339b0}
DEBUG <-- 200 https://api.fly.io/graphql (105.11ms)
DEBUG {
"data": {
"finishBuild": {
"id": "3849361",
"status": "failed",
"wallclockTimeMs": 2634
}
}
}
DEBUG Task manager done
Error: failed to fetch an image or build from source: error connecting to docker: server returned a non-200 status code: 500
thanks @pepiopee! With the request ID I was able to trace this to an error on our side - we’re on it, it’s reported on the status page https://status.flyio.net/ in case you want to follow along.
Regards,
Not sure if it helps but tried again a minute or so ago and got the following:
Error: failed to fetch an image or build from source: error connecting to docker: Could not find Volume
I am getting the same error.
Thanks for confirming - can you try nuking your builder again and redeploying? Let me know if it still gives you this “could not find volume” error.
EDIT: nuking the builder won’t help unfortunately 
We can repro this on our side, we’ll get it fixed!
Working for me now!
Works for me now as well, thanks.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.