postgres instance is down

1

Suddenly, since 20~30min ago, I started to get a ton of errors on Sentry about being unable to connect to the database.
checking the postgres app, I see this:

Health check for your postgres vm has failed. Your instance has hit resource limits. Upgrading your instance / volume size or reducing your usage might help.
HTTP GET http://172.19.134.146:5500/flycheck/vm: 500 Internal Server Error Output: [✗] checkDisk: 90.24 MB (9.1%!)(MISSING) free space on /data/ (32.68µs)

and

Health check for your postgres database has failed. Your database is malfunctioning.

I’m not sure if the missing free space is correct, this app barely has any traffic and generates no data.

this is how the health checks look like

image
image2356×478 49.3 KB

The status page says everything is ok.

I wonder if there is something going on or is it on my side

2

Update: ended up scaling the volume up to 2Gb and now it is working but the thing is that my DB has just 9mb, doesn’t make much sense

update #2: same error again

3

update #3: database is still down, the primary shows this:

500 Internal Server Error
[✗] checkDisk: 145.12 MB (7.3%!)(MISSING) free space on /data/ (41.15µs)
[✓] checkLoad: load averages: 0.00 0.00 0.00 (83.29µs)
[✓] memory: system spent 0s of the last 60s waiting on memory (40.35µs)
[✓] cpu: system spent 312ms of the last 60s waiting on cpu (19.08µs)
[✓] io: system spent 300ms of the last 60s waiting on io (16.26µs)

while the replicas are showing something different:

[✓] checkDisk: 858.27 MB (87.1%) free space on /data/ (100.07µs)
[✓] checkLoad: load averages: 0.00 0.00 0.00 (101.73µs)
[✓] memory: system spent 12ms of the last 60s waiting on memory (35.97µs)
[✓] cpu: system spent 600ms of the last 60s waiting on cpu (29.35µs)
[✓] io: system spent 540ms of the last 60s waiting on io (24.26µs)

the volume was 1Gb, already scaled to 2Gb as mentioned above, what is going on?

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.