Intermittent 'Read-only Database' Errors with LiteFS in Multi-region EPIC-Stack (Remix) App

1

I have an epic-stack (remix) app using litefs to manage my sqlite db in multiple regions. Everything appears fine, but about every other POST request is erroring with “attempt to write a readonly database”. I’ve scoured the internet, proxy docs (Built-in HTTP Proxy · Fly Docs), and still can’t understand what’s wrong with my setup. Again, every other request succeeds, but the others throw this error. Logs show the same POST request failing the first time and then succeeding the second time. What’s going on?

2024-05-31T05:36:55.901 app[d8dd9eeae39498] ord [info] level=INFO msg="fuse: create(): cannot create journal: read only replica"

2024-05-31T05:36:55.902 app[d8dd9eeae39498] ord [info] prisma:error

2024-05-31T05:36:55.902 app[d8dd9eeae39498] ord [info] Invalid `prisma.group.update()` invocation:

2024-05-31T05:36:55.902 app[d8dd9eeae39498] ord [info] Error occurred during query execution:

2024-05-31T05:36:55.902 app[d8dd9eeae39498] ord [info] ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(SqliteError { extended_code: 1544, message: Some("attempt to write a readonly database") }), transient: false })

2024-05-31T05:36:55.905 app[d8dd9eeae39498] ord [info] POST /app/groups/clwpwocse0001yvdd23s9cz5i?_data=routes/app+/groups+/$id/route 500 - - 12.445 ms

2024-05-31T05:36:56.616 app[d8dd9e1ce90258] ord [info] GET /app/groups/clwpwocse0001yvdd23s9cz5i?_data=root 200 - - 10.639 ms

2024-05-31T05:36:57.841 app[d8dd9e1ce90258] ord [info] HEAD / 200 - - 6.461 ms

2024-05-31T05:36:58.557 app[d8dd9eeae39498] ord [info] HEAD / 200 - - 6.171 ms

2024-05-31T05:36:59.935 app[d8dd9e1ce90258] ord [info] GET / 302 - - 7.950 ms

2024-05-31T05:37:00.158 app[d8dd9eeae39498] ord [info] GET /app 200 - - 49.616 ms

2024-05-31T05:37:04.298 app[d8dd9eeae39498] ord [info] GET /app/groups?_data=routes/app+/groups+/_index 200 - - 9.115 ms

2024-05-31T05:37:05.626 app[d8dd9e1ce90258] ord [info] GET /app/groups/clwpwocse0001yvdd23s9cz5i?_data=routes/app+/groups+/$id/route 200 - - 32.090 ms

2024-05-31T05:37:06.639 app[d8dd9e1ce90258] ord [info] POST /app/groups/clwpwocse0001yvdd23s9cz5i?_data=routes/app+/groups+/$id/route 204 - - 29.891 ms

2024-05-31T05:37:06.912 app[d8dd9eeae39498] ord [info] GET /app/groups?_data=routes/app+/route 200 - - 2.532 ms

2024-05-31T05:37:06.932 app[d8dd9e1ce90258] ord [info] GET /app/groups?_data=root 200 - - 21.985 ms

2024-05-31T05:37:06.933 app[d8dd9e1ce90258] ord [info] GET /app/groups?_data=routes/app+/groups+/_index 200 - - 17.755 ms
2

Fixed it! Remove the entire [http_service] section from your fly.toml. It doesn’t play well with the litefs proxy (something may have changed recently. See this comment for the solution/context: Proxy not forwarding POST requests to primary · Issue #386 · superfly/litefs · GitHub.

1 Like
3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.