Attaching unmanaged PG database results in "Error: http: read on closed response body"

Hi, today I can’t attach my pg db to app.

fly pg attach dbAppName -a appName
Checking for existing attachments
Error: http: read on closed response body

1 Like

Thats odd. Can you run that same command with LOG_LEVEL=debug command and check which of the API calls failed?

Would you try fly --verbose pg ... to see if that gives any more detail?

What region is the database in? How many nodes do you have in your cluster?

Edit: Ah, crossed messages with @lubien :flexed_biceps::relieved_face:

1 Like

Added –verbose --debug

Error: http: read on closed response body
Stacktrace:
goroutine 1 [running]:
runtime/debug.Stack()
        runtime/debug/stack.go:26 +0x64
github.com/superfly/flyctl/internal/cli.printError(0x1400015a000, 0x14000767bfd, 0x140005e6f08, {0x106db2038, 0x1082063c0})
        github.com/superfly/flyctl/internal/cli/cli.go:185 +0x42c
github.com/superfly/flyctl/internal/cli.Run({0x106dd9780?, 0x14000325380?}, 0x1400015a000, {0x1400004e090, 0x7, 0x7})
        github.com/superfly/flyctl/internal/cli/cli.go:118 +0x958
main.run()
        github.com/superfly/flyctl/main.go:47 +0x160
main.main()
        github.com/superfly/flyctl/main.go:26 +0x20
DEBUG determined hostname: “MacBook-Pro-Yura.local”
DEBUG determined working directory: “/Users/yurashiryaev/Work/OTRUTA/pflegehirte-directus”
DEBUG determined config directory: “/Users/yurashiryaev/.fly”
DEBUG ensured config directory exists.
DEBUG ensured config directory perms.
DEBUG cache loaded.
DEBUG config initialized.
DEBUG skipped querying for new release
DEBUG checking for updates…
DEBUG started querying for statuspage incidents
DEBUG client initialized.
DEBUG app config loaded from /Users/yurashiryaev/Work/OTRUTA/pflegehirte-directus/fly.toml
DEBUG started querying for host issues
DEBUG → POST 


DEBUG → POST 


DEBUG {
“query”: “query($admin: Boolean!) { organizations(admin: $admin) { nodes { id slug rawSlug name type paidPlan billable viewerRole internalNumericId } } }”,
“variables”: {
“admin”: false
}
}

DEBUG {
“query”: “query($appName: String!) { apphostissues:app(name: $appName) { hostIssues { nodes { internalId message createdAt updatedAt } } } }”,
“variables”: {
“appName”: “pflege-dir”
}
}

DEBUG {}
DEBUG {}
DEBUG ← 200 
 (816.42ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“apphostissues”:{“hostIssues”:{“nodes”:
}}}}
DEBUG ← 200 
 (822.76ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“organizations”:{“nodes”:[{“id”:“ZRJPoxp8kwJ0ZuKLVmgJ2ZQ1bJHvGjNaz”,“slug”:“personal”,“rawSlug”:“julien-167”,“name”:“Julien”,“type”:“PERSONAL”,“paidPlan”:true,“billable”:true,“viewerRole”:“admin”,“internalNumericId”:“1026174”}]}}}
DEBUG app config loaded from /Users/yurashiryaev/Work/OTRUTA/pflegehirte-directus/fly.toml
DEBUG Starting task manager
DEBUG monitoring tokens at /Users/yurashiryaev/.fly/config.yml
DEBUG → POST 


DEBUG {
“query”: “query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed network status appUrl platformVersion organization { id internalNumericId slug paidPlan } postgresAppRole: role { name } } }”,
“variables”: {
“appName”: “pflegehirte-db”
}
}

DEBUG {}
DEBUG ← 200 
 (426.71ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“appcompact”:{“id”:“pflegehirte-db”,“name”:“pflegehirte-db”,“hostname”:“pflegehirte-db.fly.dev”,“deployed”:true,“network”:“”,“status”:“deployed”,“appUrl”:“https://fdaa:14:3591:0:1::6”,“platformVersion”:“machines”,“postgresAppRole”:{“name”:“postgres_cluster”},“organization”:{“id”:“ZRJPoxp8kwJ0ZuKLVmgJ2ZQ1bJHvGjNaz”,“internalNumericId”:“1026174”,“slug”:“personal”,“paidPlan”:true}}}}
DEBUG → POST 


DEBUG {
“query”: “query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed network status appUrl platformVersion organization { id internalNumericId slug paidPlan } postgresAppRole: role { name } } }”,
“variables”: {
“appName”: “pflege-dir”
}
}

DEBUG {}
DEBUG ← 200 
 (409.31ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“appcompact”:{“id”:“pflege-dir”,“name”:“pflege-dir”,“hostname”:“pflege-dir.fly.dev”,“deployed”:false,“network”:“”,“status”:“pending”,“appUrl”:null,“platformVersion”:“machines”,“postgresAppRole”:null,“organization”:{“id”:“ZRJPoxp8kwJ0ZuKLVmgJ2ZQ1bJHvGjNaz”,“internalNumericId”:“1026174”,“slug”:“personal”,“paidPlan”:true}}}}
DEBUG → POST 


DEBUG {
“query”: “mutation($input: ValidateWireGuardPeersInput!) { validateWireGuardPeers(input: $input) { invalidPeerIps } }”,
“variables”: {
“input”: {
“peerIps”: [
“fdaa:14:3591:a7b:17ec:0:a:102”
]
}
}
}

DEBUG {}
DEBUG ← 200 
 (372.9ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“validateWireGuardPeers”:{“invalidPeerIps”:
}}}
DEBUG → POST 


DEBUG {
“query”: “query ($appName: String!) { app(name: $appName) { ipAddresses { nodes { id address type region createdAt network { name organization { slug } } serviceName } } sharedIpAddress } }”,
“variables”: {
“appName”: “pflegehirte-db”
}
}

DEBUG {}
DEBUG ← 200 
 (402.31ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“app”:{“sharedIpAddress”:null,“ipAddresses”:{“nodes”:[{“id”:“ip_nq8m9ljl0mq1r0yp”,“address”:“2a09:8280:1::73:5fef:0”,“type”:“v6”,“region”:“global”,“createdAt”:“2025-05-02T22:18:38Z”,“network”:null,“serviceName”:null},{“id”:“ip_zxp31er85ee2kr7g”,“address”:“fdaa:14:3591:0:1::6”,“type”:“private_v6”,“region”:“global”,“createdAt”:“2025-05-02T22:17:52Z”,“serviceName”:null,“network”:{“name”:“”,“organization”:{“slug”:“personal”}}}]}}}}
DEBUG → GET https://api.machines.dev/v1/apps/pflegehirte-db/machines

DEBUG ← 200 https://api.machines.dev/v1/apps/pflegehirte-db/machines (1s)

DEBUG   ← https://api.machines.dev/v1/apps/pflegehirte-db/machines: [{“id”:“328766d4f52e18”,“name”:“old-surf-3351”,“state”:“started”,“region”:“fra”,“instance_id”:“01K3JEYRPN0ZVBDFA043F95TPV”,“private_ip”:“fdaa:14:3591:a7b:3ff:9bd5:4df1:2”,“config”:{“env”:{“FLY_PROCESS_GROUP”:“app”,“PRIMARY_REGION”:“fra”},“init”:{},“guest”:{“cpu_kind”:“shared”,“cpus”:1,“memory_mb”:512},“metadata”:{“fly-managed-postgres”:“true”,“fly_flyctl_version”:“0.3.114”,“fly_platform_version”:“v2”,“fly_process_group”:“app”,“fly_release_id”:“yaZe4zjKK0pZnTKXNVbLjO80G”,“fly_release_version”:“2”,“managed-by
DEBUG   ← https://api.machines.dev/v1/apps/pflegehirte-db/machines: -fly-deploy”:“true”},“mounts”:[{“encrypted”:true,“path”:“/data”,“size_gb”:1,“volume”:“vol_4m88z1ml0y0lx9gr”,“name”:“pg_data”}],“services”:[{“protocol”:“tcp”,“internal_port”:5432,“autostart”:true,“ports”:[{“port”:5432,“handlers”:[“pg_tls”]}],“concurrency”:{“type”:“connections”,“hard_limit”:1000,“soft_limit”:1000},“force_instance_key”:null},{“protocol”:“tcp”,“internal_port”:5433,“autostart”:true,“ports”:[{“port”:5433,“handlers”:[“pg_tls”]}],“concurrency”:{“type”:“connections”,“hard_limit”:1000,“soft_limit”:1000},“force_instance_key”:null}],“metrics”:{“port”:9187,“path”:“/metrics”},“che
DEBUG   ← https://api.machines.dev/v1/apps/pflegehirte-db/machines: cks”:{“pg”:{“port”:5500,“type”:“http”,“interval”:“15s”,“timeout”:“10s”,“path”:“/flycheck/pg”},“role”:{“port”:5500,“type”:“http”,“interval”:“15s”,“timeout”:“10s”,“path”:“/flycheck/role”},“vm”:{“port”:5500,“type”:“http”,“interval”:“15s”,“timeout”:“10s”,“path”:“/flycheck/vm”}},“image”:“docker-hub-mirror.fly.io/flyio/postgres-flex:17",“restart”:{“policy”:“on-failure”,“max_retries”:10}},“incomplete_config”:null,“image_ref”:{“registry”:“docker-hub-mirror.fly.io”,“repository”:“flyio/postgres-flex”,“tag”:“17”,“digest”:“sha256:f4301ae20d193ab3c3539eb9df9a8f8d3736dd331aeec1bfb2e34b539dc353c5”,“labels”:{“fly.app_role”:“postgres_cluster”,“fly.pg-manager”:“repmgr”,“fly.pg-version”:“17.2”,“fly.version”:“v0.0.66”,“org.opencontainers.image.ref.name”:“ubuntu”,“org.opencontainers.image.version”:“24.04”}},“created_at”:“2025-08-26T05:52:30Z”,“updated_at”:“2025-08-26T05:52:32Z”,“events”:[{“id”:“01K3JEYTXSF7MKWD20NYKDCAJK”,“type”:“start”,“status”:“started”,“request”:{},“source”:“flyd”,“timestamp”:1756187552697},{“id”:“01K3JEYRTYHVSPQ1NFPBDPPA1K”,“type”:“launch”,“status”:“created”,“source”:“flyd”,“timestamp”:1756187550558}],“checks”:[{“name”:“pg”,“status”:“passing”,“output”:"[✓] connections: 12 used, 3 reserved, 300 max (11.02ms)\n[✓] cluster-locks: No active locks detected (16.01µs)\n[✓] disk-capacity: 16.8% - readonly mode will be enabled at 90.0% (18.92µs)”,“updated_at”:“2025-08-26T05:52:46.456Z”},{“name”:“vm”,“status”:“passing”,“output”:“[✓] checkDisk: 809.61 MB (83.1%) free space on /data/ (52.66µs)\n[✓] checkLoad: load averages: 0.14 0.13 0.02 (85.02µs)\n[✓] memory: system spent 0s of the last 60s waiting on memory (54.7µs)\n[✓] cpu: system spent 2.54s of the last 60s waiting on cpu (44.75µs)\n[✓] io: system spent 0s of the last 60s waiting on io (39.16µs)”,“updated_at”:“2025-10-22T16:35:36.049Z”},{“name”:“role”,“status”:“passing”,“output”:“primary”,“updated_at”:“2025-08-26T05:52:36.101Z”}],“host_status”:“ok”}]

DEBUG → POST 


DEBUG {
“query”: “query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed network status appUrl platformVersion organization { id internalNumericId slug paidPlan } postgresAppRole: role { name } } }”,
“variables”: {
“appName”: “pflege-dir”
}
}

DEBUG {}
DEBUG ← 200 
 (438.76ms)

DEBUG   ← https://api.fly.io/graphql: {“data”:{“appcompact”:{“id”:“pflege-dir”,“name”:“pflege-dir”,“hostname”:“pflege-dir.fly.dev”,“deployed”:false,“network”:“”,“status”:“pending”,“appUrl”:null,“platformVersion”:“machines”,“postgresAppRole”:null,“organization”:{“id”:“ZRJPoxp8kwJ0ZuKLVmgJ2ZQ1bJHvGjNaz”,“internalNumericId”:“1026174”,“slug”:“personal”,“paidPlan”:true}}}}
DEBUG flypg will connect to: http://fdaa:14:3591:a7b:3ff:9bd5:4df1:2:5500

Checking for existing attachments
DEBUG → GET https://api.machines.dev/v1/apps/pflege-dir/secrets

DEBUG ← 200 https://api.machines.dev/v1/apps/pflege-dir/secrets (1.12s)

DEBUG   ← https://api.machines.dev/v1/apps/pflege-dir/secrets: {“secrets”:[{“name”:“AWS_ACCESS_KEY_ID”,“digest”:“6f13b0e5a00a044e”,“created_at”:“2025-10-22T14:34:16Z”,“updated_at”:“2025-10-22T14:34:16Z”},{“name”:“AWS_ENDPOINT_URL_S3”,“digest”:“85e8ac62d7de0c23”,“created_at”:“2025-10-22T14:34:16Z”,“updated_at”:“2025-10-22T14:34:16Z”},{“name”:“AWS_REGION”,“digest”:“274e16452b90854d”,“created_at”:“2025-10-22T14:34:16Z”,“updated_at”:“2025-10-22T14:34:16Z”},{“name”:“AWS_SECRET_ACCESS_KEY”,“digest”:“4dd977a46526bfc6”,“created_at”:“2025-10-22T14:34:16Z”,“updated_at”:“2025-10-
DEBUG   ← https://api.machines.dev/v1/apps/pflege-dir/secrets: 22T14:34:16Z”},{“name”:“BUCKET_NAME”,“digest”:“cb901a092cafa7cb”,“created_at”:“2025-10-22T14:34:16Z”,“updated_at”:“2025-10-22T14:34:16Z”}]}

DEBUG → GET http://fdaa:14:3591:a7b:3ff:9bd5:4df1:2:5500/commands/databases/pflege_dir

DEBUG ← 200 http://fdaa:14:3591:a7b:3ff:9bd5:4df1:2:5500/commands/databases/pflege_dir (868.34ms)

DEBUG Task manager done
DEBUG done monitoring tokens
DEBUG querying for host issues resulted to 

DEBUG querying for statuspage incidents resulted to &{
}
Error: http: read on closed response body


This is a internal fly DNS IP (fdaa:, possibly a flycast) and the command you used is trying to check you unmanaged postgres status to continue with the attachment but it seems your database could be misbehaving.

My suggestions are:

  • Look into your DB app for logs that could indicate bad states.
  • Try fly wireguard websockets enable or fly wireguard websockets disabled to try wireguard with and without websockets in case your local internet connection to fly.io is flaky
  • Try fly pg restart on your unmanaged PG to see if it self heals.

Here’s some docs on how to check your unmanaged postgres: Check Provisioned Resources · Fly Docs

Hi there! :waving_hand:

Unfortunately, the suggested steps didn’t help. Here’s what I’ve tried:

  • There are no logs in the consoles of either application — nothing suspicious appears.

  • Turning WebSockets on and off (fly wireguard websockets enable/disable) didn’t make a difference.

  • Restarting with fly pg restart didn’t help either — the database is still unavailable.

Any further suggestions would be appreciated!

If your pg machine is started I’d suggest sshing into it via fly ssh console to debug whats happening there. If there’s no logs it probably means your DB would be in a bad shape

I’ll put in my usual encouragement to move to a managed database service, on Fly or elsewhere. Unless one is actually a DBA, it’s probably better to pay a little to not have a problem, rather than getting stuck on lengthy (and costly) maintenance/fix work.

Hi again! :waving_hand:

I can connect to the machine via fly ssh console, but I’m not sure what I should be looking for once connected. The database I’m trying to attach is new — I’ve recreated it, even spun up a brand‑new one, and also tried attaching the old DB, but I keep getting the same error in every case.

  • Able to SSH into the PG machine, but unsure what to inspect.

  • Target database is newly created (and recreated) and even tried an older one.

  • Same attachment error persists across all attempts.

Any advice on what to check next would be greatly appreciated!

Hi! :waving_hand:

I didn’t have this problem before, it has only started happening recently. Switching to a managed database isn’t a good fit for me because of the price — I’d prefer to keep using an unmanaged setup if possible.

Do you have any suggestions on how to resolve the attachment issue without moving to managed?

What size is your database? If it’s under 0.5G, you can have a Supabase one free of charge, permanently. Someone posted elsewhere in this forum that you can have one a Digital Ocean, at 10G of storage, for 15USD/month.

I appreciate the machine costs at Fly are indeed low, but it’s worth having at least two nodes in your cluster, otherwise you’re risking (unfortunately quite frequent) total node failure. Plus you should set up an external backup, as another home-made-solution user else here recently discovered (a month of work was lost). So, once all that is summed, I am not entirely sure much cost saving has been realised.

Hi! :waving_hand:

I’m here on the Fly.io forum for help because this command used to work without any issues. Talking about switching to another provider (like Supabase) doesn’t make sense when I’m trying to troubleshoot this specific problem on Fly.io.

I’d appreciate guidance on fixing the current issue rather than migrating away.

Hi! Error: http: read on closed response body is a known bug on flyctl v0.3.198 and 0.3.200.

You can either downgrade to 0.3.195 or wait for 0.3.201 to come out (should be in the next few hours) - those versions don’t (or won’t, for 0.3.201!) have this issue.

  • Daniel
1 Like

I understand that perspective, but this may be an XY problem; it is common enough for folks to ask how to achieve a solution, rather than how to fix the original problem.

I certainly don’t insist my ideas fit all cases, but the advice on not spending hours fixing unmanaged dbs probably is still sound.

Hi

I downgraded to flyctl version 0.3.195 as you suggested and it indeed resolved the issue. Thank you so much for the prompt help — really appreciate it! :folded_hands:

2 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.