My Postgres database recently went down. When I try to restart this, this is what the logs show.
2023-10-18T01:55:45.944 app[e286771ae36786] sjc [info] checking stolon status
2023-10-18T01:55:45.955 app[e286771ae36786] sjc [info] keeper | 2023-10-18T01:55:45.955Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {“error”: “dial unix /tmp/.s.PGSQL.5433: connect: no such file or directory”}
2023-10-18T01:55:46.943 app[e286771ae36786] sjc [info] checking stolon status
2023-10-18T01:55:47.574 app[e286771ae36786] sjc [info] exporter | INFO[0159] Established new database connection to “fdaa:0:94de:a7b:2295:993e:7c28:2:5433”. source=“postgres_exporter.go:970”
2023-10-18T01:55:47.637 app[e286771ae36786] sjc [info] sentinel | 2023-10-18T01:55:47.636Z WARN cmd/sentinel.go:276 no keeper info available {“db”: “adcc4ade”, “keeper”: “16c802a85612”}
2023-10-18T01:55:47.639 app[e286771ae36786] sjc [info] sentinel | 2023-10-18T01:55:47.638Z ERROR cmd/sentinel.go:1018 no eligible masters
2023-10-18T01:55:47.739 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.739 UTC [1400] LOG: starting PostgreSQL 14.6 (Debian 14.6-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-10-18T01:55:47.739 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.739 UTC [1400] LOG: listening on IPv6 address “fdaa:0:94de:a7b:2295:993e:7c28:2”, port 5433
2023-10-18T01:55:47.740 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.740 UTC [1400] LOG: listening on Unix socket “/tmp/.s.PGSQL.5433”
2023-10-18T01:55:47.747 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.742 UTC [1401] LOG: database system shutdown was interrupted; last known up at 2023-10-18 01:55:42 UTC
2023-10-18T01:55:47.869 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.869 UTC [1401] LOG: database system was not properly shut down; automatic recovery in progress
2023-10-18T01:55:47.871 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.871 UTC [1401] LOG: redo starts at 55/4A000028
2023-10-18T01:55:47.871 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.871 UTC [1401] LOG: redo done at 55/4B0014A0 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2023-10-18T01:55:47.876 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.875 UTC [1401] PANIC: could not write to file “pg_wal/xlogtemp.1401”: No space left on device
2023-10-18T01:55:47.878 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.876 UTC [1400] LOG: startup process (PID 1401) was terminated by signal 6: Aborted
2023-10-18T01:55:47.878 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.876 UTC [1400] LOG: aborting startup due to startup process failure
2023-10-18T01:55:47.878 app[e286771ae36786] sjc [info] keeper | 2023-10-18 01:55:47.877 UTC [1400] LOG: database system is shut down
2023-10-18T01:55:47.928 app[e286771ae36786] sjc [info] keeper | 2023-10-18T01:55:47.928Z ERROR cmd/keeper.go:1526 failed to start postgres {“error”: “postgres exited unexpectedly”}
2023-10-18T01:55:47.944 app[e286771ae36786] sjc [info] checking stolon status
2023-10-18T01:55:48.457 app[e286771ae36786] sjc [info] keeper | 2023-10-18T01:55:48.456Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {“error”: “dial unix /tmp/.s.PGSQL.5433: connect: no such file or directory”}
2023-10-18T01:55:48.575 app[e286771ae36786] sjc [info] exporter | ERRO[0160] Error opening connection to database (postgresql://flypgadmin:PASSWORD_REMOVED@[fdaa:0:94de:a7b:2295:993e:7c28:2]:5433/postgres?sslmode=disable): dial tcp [fdaa:0:94de:a7b:2295:993e:7c28:2]:5433: connect: connection refused source=“postgres_exporter.go:1658”
It just repeats this ad nauseaum.
Is this something that I did, or is there something that I need to do to correct this?