postgres failing - pre-existing shared memory block

Getting

pre-existing shared memory block

2022-07-26T14:46:19.907 app[223737dd] fra [info] keeper | 2022-07-26T14:46:19.906Z INFO cmd/keeper.go:1504 our db requested role is master

2022-07-26T14:46:19.924 app[223737dd] fra [info] keeper | 2022-07-26T14:46:19.924Z INFO postgresql/postgresql.go:319 starting database

2022-07-26T14:46:19.936 app[223737dd] fra [info] keeper | 2022-07-26 14:46:19.936 UTC [31298] FATAL: pre-existing shared memory block (key 8193, ID 1) is still in use

2022-07-26T14:46:19.936 app[223737dd] fra [info] keeper | 2022-07-26 14:46:19.936 UTC [31298] HINT: Terminate any old server processes associated with data directory "/data/postgres".

2022-07-26T14:46:20.126 app[223737dd] fra [info] keeper | 2022-07-26T14:46:20.125Z ERROR cmd/keeper.go:1526 failed to start postgres {"error": "postgres exited unexpectedly"}

2022-07-26T14:46:21.619 app[223737dd] fra [info] keeper | 2022-07-26T14:46:21.619Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:22.082 app[223737dd] fra [info] exporter | INFO[592445] Established new database connection to "fdaa:0:5914:a7b:23c4:1:3878:2:5433". source="postgres_exporter.go:970"

2022-07-26T14:46:23.082 app[223737dd] fra [info] exporter | ERRO[592446] Error opening connection to database (postgresql://flypgadmin:PASSWORD_REMOVED@[fdaa:0:5914:a7b:23c4:1:3878:2]:5433/postgres?sslmode=disable): dial tcp [fdaa:0:5914:a7b:23c4:1:3878:2]:5433: connect: connection refused source="postgres_exporter.go:1658"

2022-07-26T14:46:24.120 app[223737dd] fra [info] keeper | 2022-07-26T14:46:24.120Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:24.442 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:24.441Z INFO cmd/sentinel.go:995 master db is failed {"db": "99c3097e", "keeper": "23c4138782"}

2022-07-26T14:46:24.442 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:24.442Z INFO cmd/sentinel.go:1006 trying to find a new master to replace failed master

2022-07-26T14:46:24.442 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:24.442Z ERROR cmd/sentinel.go:1009 no eligible masters

2022-07-26T14:46:25.160 app[223737dd] fra [info] keeper | 2022-07-26T14:46:25.160Z INFO cmd/keeper.go:1504 our db requested role is master

2022-07-26T14:46:25.172 app[223737dd] fra [info] keeper | 2022-07-26T14:46:25.172Z INFO postgresql/postgresql.go:319 starting database

2022-07-26T14:46:25.182 app[223737dd] fra [info] keeper | 2022-07-26 14:46:25.181 UTC [31306] FATAL: pre-existing shared memory block (key 8193, ID 1) is still in use

2022-07-26T14:46:25.182 app[223737dd] fra [info] keeper | 2022-07-26 14:46:25.181 UTC [31306] HINT: Terminate any old server processes associated with data directory "/data/postgres".

2022-07-26T14:46:25.374 app[223737dd] fra [info] keeper | 2022-07-26T14:46:25.372Z ERROR cmd/keeper.go:1526 failed to start postgres {"error": "postgres exited unexpectedly"}

2022-07-26T14:46:26.624 app[223737dd] fra [info] keeper | 2022-07-26T14:46:26.623Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:29.125 app[223737dd] fra [info] keeper | 2022-07-26T14:46:29.124Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:30.489 app[223737dd] fra [info] keeper | 2022-07-26T14:46:30.488Z INFO cmd/keeper.go:1504 our db requested role is master

2022-07-26T14:46:30.502 app[223737dd] fra [info] keeper | 2022-07-26T14:46:30.501Z INFO postgresql/postgresql.go:319 starting database

2022-07-26T14:46:30.514 app[223737dd] fra [info] keeper | 2022-07-26 14:46:30.513 UTC [31314] FATAL: pre-existing shared memory block (key 8193, ID 1) is still in use

2022-07-26T14:46:30.514 app[223737dd] fra [info] keeper | 2022-07-26 14:46:30.513 UTC [31314] HINT: Terminate any old server processes associated with data directory "/data/postgres".

2022-07-26T14:46:30.703 app[223737dd] fra [info] keeper | 2022-07-26T14:46:30.702Z ERROR cmd/keeper.go:1526 failed to start postgres {"error": "postgres exited unexpectedly"}

2022-07-26T14:46:31.625 app[223737dd] fra [info] keeper | 2022-07-26T14:46:31.625Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:31.779 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:31.779Z INFO cmd/sentinel.go:1006 trying to find a new master to replace failed master 138782"}

2022-07-26T14:46:31.779 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:31.779Z INFO cmd/sentinel.go:1006 trying to find a new master to replace failed master

2022-07-26T14:46:31.779 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:31.779Z ERROR cmd/sentinel.go:1009 no eligible masters

2022-07-26T14:46:34.126 app[223737dd] fra [info] keeper | 2022-07-26T14:46:34.126Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:36.628 app[223737dd] fra [info] keeper | 2022-07-26T14:46:36.627Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:36.702 app[223737dd] fra [info] keeper | 2022-07-26T14:46:36.697Z INFO cmd/keeper.go:1504 our db requested role is master

2022-07-26T14:46:36.727 app[223737dd] fra [info] keeper | 2022-07-26T14:46:36.727Z INFO postgresql/postgresql.go:319 starting database

2022-07-26T14:46:36.751 app[223737dd] fra [info] keeper | 2022-07-26 14:46:36.749 UTC [31322] FATAL: pre-existing shared memory block (key 8193, ID 1) is still in use

2022-07-26T14:46:36.751 app[223737dd] fra [info] keeper | 2022-07-26 14:46:36.749 UTC [31322] HINT: Terminate any old server processes associated with data directory "/data/postgres".

2022-07-26T14:46:36.931 app[223737dd] fra [info] keeper | 2022-07-26T14:46:36.929Z ERROR cmd/keeper.go:1526 failed to start postgres {"error": "postgres exited unexpectedly"}

2022-07-26T14:46:36.979 app[223737dd] fra [info] exporter | INFO[592460] Established new database connection to "fdaa:0:5914:a7b:23c4:1:3878:2:5433". source="postgres_exporter.go:970"

2022-07-26T14:46:37.979 app[223737dd] fra [info] exporter | ERRO[592461] Error opening connection to database (postgresql://flypgadmin:PASSWORD_REMOVED@[fdaa:0:5914:a7b:23c4:1:3878:2]:5433/postgres?sslmode=disable): dial tcp [fdaa:0:5914:a7b:23c4:1:3878:2]:5433: connect: connection refused source="postgres_exporter.go:1658"

2022-07-26T14:46:38.935 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:38.935Z INFO cmd/sentinel.go:995 master db is failed {"db": "99c3097e", "keeper": "23c4138782"}

2022-07-26T14:46:38.935 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:38.935Z INFO cmd/sentinel.go:1006 trying to find a new master to replace failed master

2022-07-26T14:46:38.935 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:38.935Z ERROR cmd/sentinel.go:1009 no eligible masters

2022-07-26T14:46:39.129 app[223737dd] fra [info] keeper | 2022-07-26T14:46:39.128Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:41.629 app[223737dd] fra [info] keeper | 2022-07-26T14:46:41.629Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:41.957 app[223737dd] fra [info] keeper | 2022-07-26T14:46:41.957Z INFO cmd/keeper.go:1504 our db requested role is master

2022-07-26T14:46:41.967 app[223737dd] fra [info] keeper | 2022-07-26T14:46:41.966Z INFO postgresql/postgresql.go:319 starting database

2022-07-26T14:46:41.974 app[223737dd] fra [info] keeper | 2022-07-26 14:46:41.974 UTC [31329] FATAL: pre-existing shared memory block (key 8193, ID 1) is still in use

2022-07-26T14:46:41.974 app[223737dd] fra [info] keeper | 2022-07-26 14:46:41.974 UTC [31329] HINT: Terminate any old server processes associated with data directory "/data/postgres".

2022-07-26T14:46:42.168 app[223737dd] fra [info] keeper | 2022-07-26T14:46:42.167Z ERROR cmd/keeper.go:1526 failed to start postgres {"error": "postgres exited unexpectedly"}

2022-07-26T14:46:44.130 app[223737dd] fra [info] keeper | 2022-07-26T14:46:44.130Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:46.632 app[223737dd] fra [info] keeper | 2022-07-26T14:46:46.631Z ERROR cmd/keeper.go:719 cannot get configured pg parameters {"error": "dial unix /tmp/.s.PGSQL.5433: connect: connection refused"}

2022-07-26T14:46:46.774 app[223737dd] fra [info] sentinel | 2022-07-26T14:46:46.773Z INFO cmd/sentinel.go:995 master db is failed {"db": "99c3097e", "keeper": "23c4138782"}

@blasrodri That’s strange.

  1. What version of Postgres are you running?
fly image show --app <app-name>
  1. Have you tried restarting the VM?

Oh man, it worked. Thanks so much