Postgres app unresponsive, timing out

Last week I had some issues with an unresponsive postgres app Postgres restart timing out

Today I’m having a similar issue:

❯ fly pg restart -a "envuelos-db--prod"                                          
Error: failed to obtain lease: failed to get lease on VM e784e965b23783: Post "http://[fc01:a7b:146::]:3593/flyd.v1.LeaseService/Acquire": dial tcp [fc01:a7b:146::]:3593: i/o timeout

I’ve also stopped getting any metrics as of a few hours ago

The machine seems to be running:

envuelos-db--prod
ID            	NAME               	STATE  	REGION	IMAGE	IP ADDRESS	VOLUME	CREATED             	LAST UPDATED        	APP PLATFORM	PROCESS GROUP	SIZE
e784e965b23783	frosty-feather-5628	started	      	:    	          	      	1970-01-01T00:00:00Z	0001-01-01T00:00:00Z

Why is this happening? and what can I do to fix this? This is a low traffic app, but any suggestions on what could be done to improve its reliability would be very helpful.

Still no luck, trying to restart or do anything like update the machine:

❯ fly pg restart -a "envuelos-db--prod"
Error: failed to obtain lease: failed to get lease on VM e784e965b23783: dial tcp [fc01:a7b:146::]:3593: i/o timeout

❯ fly machine update e784e965b23783 -s shared-cpu-2x                                        ✘ 1
Error: could not get machine e784e965b23783: failed to get VM e784e965b23783: unavailable: dial tcp [fc01:a7b:146::]:3593: i/o timeout

I’m unable to restore to a new postgres instance from a snapshot either:

❯ fly postgres create --snapshot-id vs_On55QzQ2M4PekfnNma9
? Choose an app name (leave blank to generate one): envuelos-db-prod-2
? Select Organization: Bruno B (personal)
Some regions require a paid plan (fra, maa).
See https://fly.io/plans to set up a plan.

? Select region: Ezeiza, Argentina (eze)
? Select configuration: Development - Single node, 1x shared CPU, 256MB RAM, 1GB disk
? Scale single node pg to zero after one hour? Yes
Creating postgres cluster in organization personal
Creating app...
Setting secrets on app envuelos-db-prod-2...
Error: You hit a Fly API error with request ID: 01H28FWGMAZDWJAQJH9JQE6MS2-mia

❯ fly postgres create --snapshot-id vs_On55QzQ2M4PekfnNma9                                  ✘ 1
? Choose an app name (leave blank to generate one): envuelos-db-prod-3
? Select Organization: Bruno B (personal)
Some regions require a paid plan (fra, maa).
See https://fly.io/plans to set up a plan.

? Select region: Ezeiza, Argentina (eze)
? Select configuration: Development - Single node, 1x shared CPU, 256MB RAM, 1GB disk
? Scale single node pg to zero after one hour? No
Creating postgres cluster in organization personal
Creating app...
Setting secrets on app envuelos-db-prod-3...
Error: You hit a Fly API error with request ID: 01H28FXYW1GM4SR59G4ZPNKBMH-mia

❯ fly postgres create --snapshot-id vs_On55QzQ2M4PekfnNma9                                  ✘ 1
? Choose an app name (leave blank to generate one): envuelos-db-prod-4
? Select Organization: Bruno B (personal)
Some regions require a paid plan (fra, maa).
See https://fly.io/plans to set up a plan.

? Select region: Santiago, Chile (scl)
? Select configuration: Development - Single node, 1x shared CPU, 256MB RAM, 1GB disk
? Scale single node pg to zero after one hour? Yes
Creating postgres cluster in organization personal
Creating app...
Setting secrets on app envuelos-db-prod-4...
Error: You hit a Fly API error with request ID: 01H28FYFYCCT7VNN9BHACH37KS-mia

I tried an earlier snapshot and no luck either:

❯ fly postgres create --snapshot-id vs_x766PqPznGaVKfp7kPp
? Choose an app name (leave blank to generate one): envuelos-db-prod-5
? Select Organization: Bruno B (personal)
Some regions require a paid plan (fra, maa).
See https://fly.io/plans to set up a plan.

? Select region: Ezeiza, Argentina (eze)
? Select configuration: Development - Single node, 1x shared CPU, 256MB RAM, 1GB disk
? Scale single node pg to zero after one hour? No
Creating postgres cluster in organization personal
Creating app...
Setting secrets on app envuelos-db-prod-5...
Error: You hit a Fly API error with request ID: 01H28G0R2D28ED36BCXRPZ3327-mia

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.