harold
September 5, 2022, 4:41pm
1
Hi,
I have managed to transfer my Rails app over from Heroku, but the performance of it is weird. Sometimes it works perfectly, then other times it takes a long time to load.
I looked at the logs and found that the Fly instance keeps restarting, so when you make a request on the site you have to wait for the app to restart and then it’ll process the request. Everytime the app restarts (which is every 30 seconds or so), I get this error in the logs:
2022-09-05T16:31:10Z app[25d2e559] lhr [info]rails aborted!
2022-09-05T16:31:10Z app[25d2e559] lhr [info]Command failed with status (1): [bin/rails server...]
Then Puma boots up again:
2022-09-05T16:34:09Z app[c5ad1943] lhr [info]Starting clean up.
2022-09-05T16:34:15Z runner[c5ad1943] lhr [info]Starting instance
2022-09-05T16:34:15Z runner[c5ad1943] lhr [info]Configuring virtual machine
2022-09-05T16:34:15Z runner[c5ad1943] lhr [info]Pulling container image
2022-09-05T16:34:16Z runner[c5ad1943] lhr [info]Unpacking image
2022-09-05T16:34:16Z runner[c5ad1943] lhr [info]Preparing kernel init
2022-09-05T16:34:16Z runner[c5ad1943] lhr [info]Configuring firecracker
2022-09-05T16:34:16Z runner[c5ad1943] lhr [info]Starting virtual machine
2022-09-05T16:34:17Z app[c5ad1943] lhr [info]Starting init (commit: 249766e)...
2022-09-05T16:34:17Z app[c5ad1943] lhr [info]Preparing to run: `/bin/bash -c ${SERVER_COMMAND}` as root
2022-09-05T16:34:17Z app[c5ad1943] lhr [info]2022/09/05 16:34:17 listening on [fdaa:0:9579:a7b:aa9c:c5ad:1943:2]:22 (DNS: [fdaa::3]:53)
2022-09-05T16:34:19Z app[c5ad1943] lhr [info]DEPRECATION WARNING: Including LoggerSilence is deprecated and will be removed in Rails 6.1. Please use `ActiveSupport::LoggerSilence` instead (called from <main> at /app/config/application.rb:9)
2022-09-05T16:34:19Z app[c5ad1943] lhr [info]bin/rails server
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]DEPRECATION WARNING: Including LoggerSilence is deprecated and will be removed in Rails 6.1. Please use `ActiveSupport::LoggerSilence` instead (called from <main> at /app/config/application.rb:9)
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]=> Booting Puma
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]=> Rails 6.0.4 application starting in production
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]=> Run `rails server --help` for more startup options
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/will_paginate-3.1.7/lib/will_paginate/deprecation.rb:35: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/will_paginate-3.1.7/lib/will_paginate/deprecation.rb:35: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
2022-09-05T16:34:20Z app[c5ad1943] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/stripe_event-2.3.0/lib/stripe_event.rb:21: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
2022-09-05T16:34:21Z app[c5ad1943] lhr [info]Puma starting in single mode...
2022-09-05T16:34:21Z app[c5ad1943] lhr [info]* Version 3.12.4 (ruby 2.7.0-p0), codename: Llamas in Pajamas
2022-09-05T16:34:21Z app[c5ad1943] lhr [info]* Min threads: 5, max threads: 5
2022-09-05T16:34:21Z app[c5ad1943] lhr [info]* Environment: production
2022-09-05T16:34:21Z app[c5ad1943] lhr [info]* Listening on tcp://0.0.0.0:8080
2022-09-05T16:34:21Z app[c5ad1943] lhr [info]Use Ctrl-C to stop
This is the output of fly status --all:
Instances
ID PROCESS VERSION REGION DESIRED STATUS HEALTH CHECKS RESTARTS CREATED
b7096346 app 31 ⇡ lhr run pending 1 total, 1 passing 1 2m41s ago
c5ad1943 app 31 ⇡ lhr stop failed 1 total, 1 passing 2 7m10s ago
25d2e559 app 31 ⇡ lhr stop failed 1 total, 1 passing 2 11m47s ago
d607d714 app 31 ⇡ lhr stop failed 1 total, 1 passing 2 16m2s ago
cb1eab76 app 30 lhr stop complete 0 15m40s ago
I really don’t understand why the app keeps restarting, so if anyone can help it would be massively appreciated.
rubys
September 5, 2022, 6:28pm
2
Not much to go on here, so all I can do is guess.
Restarting periodically generally means something external is timing out - for example redis. But the problem with that theory is that normally you will see a stack traceback.
Not seeing a stack traceback could mean that you are out of memory, after all some memory is allocated while producing the traceback messages.
Try executing fly dashboard, then clicking on the metrics item on the left, scrolling to the bottom and see what your memory usage is. If this applies to you, take a look at the grey box starting with the third paragraph here: Getting Started · Fly Docs
1 Like
kurt
September 5, 2022, 6:52pm
3
harold:
c5ad1943
fly vm logs -i c5ad1943 should show you any errors that were logged when the VM stopped previously.
harold
September 5, 2022, 7:00pm
4
I think you might be right with Redis timing out, this is the traceback before the app restarted again:
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]#<Thread:0x00007f0699ce4130 /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:151 run> terminated with exception (report_on_exception is true):
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:267:in `rescue in io': Connection lost (ECONNRESET) (Redis::ConnectionError)
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:259:in `io'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:271:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:141:in `block (3 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `block (2 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:241:in `block (2 levels) in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:384:in `ensure_connected'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:231:in `block in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:319:in `logging'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:84:in `block in listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:58:in `block in with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:57:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:64:in `without_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:81:in `listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:155:in `block in ensure_listener_running'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:82:in `rescue in _read_from_socket': Connection reset by peer (Errno::ECONNRESET)
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:62:in `_read_from_socket'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:56:in `gets'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:395:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:272:in `block in read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:260:in `io'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:271:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:141:in `block (3 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `block (2 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:241:in `block (2 levels) in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:384:in `ensure_connected'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:231:in `block in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:319:in `logging'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:84:in `block in listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:58:in `block in with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:57:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:64:in `without_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:81:in `listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:155:in `block in ensure_listener_running'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/socket.rb:452:in `__read_nonblock': end of file reached (EOFError)
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/socket.rb:452:in `read_nonblock'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:65:in `_read_from_socket'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:56:in `gets'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:395:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:272:in `block in read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:260:in `io'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:271:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:141:in `block (3 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `block (2 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:241:in `block (2 levels) in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:384:in `ensure_connected'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:231:in `block in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:319:in `logging'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:84:in `block in listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:58:in `block in with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:57:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:64:in `without_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:81:in `listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:155:in `block in ensure_listener_running'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]Exiting
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:267:in `rescue in io': Connection lost (ECONNRESET) (Redis::ConnectionError)
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:259:in `io'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:271:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:141:in `block (3 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `block (2 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:241:in `block (2 levels) in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:384:in `ensure_connected'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:231:in `block in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:319:in `logging'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:84:in `block in listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:58:in `block in with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:57:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:64:in `without_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:81:in `listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:155:in `block in ensure_listener_running'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:82:in `rescue in _read_from_socket': Connection reset by peer (Errno::ECONNRESET)
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:62:in `_read_from_socket'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:56:in `gets'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/connection/ruby.rb:395:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:272:in `block in read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]:271:in `read'
from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:271:in `read'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:141:in `block (3 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:140:in `block (2 levels) in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:241:in `block (2 levels) in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:384:in `ensure_connected'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:231:in `block in process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:319:in `logging'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:84:in `block in listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:58:in `block in with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:57:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:64:in `without_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:81:in `listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /usr/lib/fullstaq-ruby/versions/2.7.0-jemalloc/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/actioncable-6.0.4/lib/action_cable/subscription_adapter/redis.rb:84:in `block in listen'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info] from /app/vendor/bundle/ruby/2.7.0/gems/redis-4.1.3/lib/redis.rb:58:in `block in with_reconnect'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.4.6/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.4.6/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/bootsnap-1.4.6/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/activesupport-6.0.4/lib/active_support/dependencies.rb:324:in `block in require'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/activesupport-6.0.4/lib/active_support/dependencies.rb:291:in `load_dependency'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]/app/vendor/bundle/ruby/2.7.0/gems/activesupport-6.0.4/lib/active_support/dependencies.rb:324:in `require'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]bin/rails:4:in `<main>'
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info]Tasks: TOP => fly:server
2022-09-05T18:45:02Z app[0a10b7c7] lhr [info](See full trace by running task with --trace)
2022-09-05T18:45:03Z app[0a10b7c7] lhr [info]Starting clean up.
2022-09-05T18:45:10Z runner[0a10b7c7] lhr [info]Starting instance
2022-09-05T18:45:14Z runner[0a10b7c7] lhr [info]Configuring virtual machine
2022-09-05T18:45:14Z runner[0a10b7c7] lhr [info]Pulling container image
2022-09-05T18:45:15Z runner[0a10b7c7] lhr [info]Unpacking image
2022-09-05T18:45:15Z runner[0a10b7c7] lhr [info]Preparing kernel init
2022-09-05T18:45:16Z runner[0a10b7c7] lhr [info]Configuring firecracker
2022-09-05T18:45:16Z runner[0a10b7c7] lhr [info]Starting virtual machine
2022-09-05T18:45:16Z app[0a10b7c7] lhr [info]Starting init (commit: 249766e)...
2022-09-05T18:45:16Z app[0a10b7c7] lhr [info]Preparing to run: `/bin/bash -c ${SERVER_COMMAND}` as root
2022-09-05T18:45:16Z app[0a10b7c7] lhr [info]2022/09/05 18:45:16 listening on [fdaa:0:9579:a7b:bada:a10:b7c7:2]:22 (DNS: [fdaa::3]:53)
2022-09-05T18:45:21Z app[0a10b7c7] lhr [info]=> Booting Puma
2022-09-05T18:45:21Z app[0a10b7c7] lhr [info]=> Rails 6.0.4 application starting in production
2022-09-05T18:45:21Z app[0a10b7c7] lhr [info]=> Run `rails server --help` for more startup options
2022-09-05T18:45:22Z app[0a10b7c7] lhr [info]Puma starting in single mode...
2022-09-05T18:45:22Z app[0a10b7c7] lhr [info]* Version 3.12.4 (ruby 2.7.0-p0), codename: Llamas in Pajamas
2022-09-05T18:45:22Z app[0a10b7c7] lhr [info]* Min threads: 5, max threads: 5
2022-09-05T18:45:22Z app[0a10b7c7] lhr [info]* Environment: production
2022-09-05T18:45:22Z app[0a10b7c7] lhr [info]* Listening on tcp://0.0.0.0:8080
2022-09-05T18:45:22Z app[0a10b7c7] lhr [info]Use Ctrl-C to stop
2022-09-05T18:45:26Z app[0a10b7c7] lhr [info][ded7131c-5853-405b-9112-2d753a973e48] Started GET "/cable" for 168.220.93.55 at 2022-09-05 18:45:26 +0000
2022-09-05T18:45:26Z app[0a10b7c7] lhr [info][ded7131c-5853-405b-9112-2d753a973e48] Started GET "/cable/" [WebSocket] for 205.234.240.70 at 2022-09-05 18:45:26 +0000
2022-09-05T18:45:26Z app[0a10b7c7] lhr [info][ded7131c-5853-405b-9112-2d753a973e48] Successfully upgraded to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: Upgrade, HTTP_UPGRADE: websocket
I have this in my cable.yml file:
production:
adapter: redis
url: redis://default:06d79c*********************55@fly-bishbashbooked-redis.upstash.io
channel_prefix: bishbashbooked-redis
I also have REDIS_URL set in my application secrets. Do you have any idea why it is closing the connection to Redis?
Thanks for the help.
harold
September 5, 2022, 7:01pm
5
Thanks Kurt, it’s a Redis problem by the looks of it.
rubys
September 5, 2022, 7:29pm
6
In the getting started guide there is a patch you might try: Patching Action Cable to Handle Redis Timeouts . This doesn’t avoid the timeouts, it just recovers better from them.
Phusion Passenger · Fly Docs describes an alternate approach where redis is run in the same VM. This requires slightly more setup, but might work better for you.
harold
September 5, 2022, 7:34pm
7
Thanks for this, I’ll have a look.
harold
September 5, 2022, 10:16pm
8
The patch seems to have fixed my issue! Thanks again for the help.
The problem with that patch is that it is not optimal to restart the ActionCable server all the time. Clients lose connection and have to re-subscribe again.
In my case, my application is restarting very often, every 1 or 2 minutes maximum.
Upstash Redis seems to timeout the connection every few seconds.
What can be done? @kurt , could you help us with this?
I have opened this thread: Upstash Redis - Connection lost (ECONNRESET) (Redis::ConnectionError - Questions / Help - Fly.io
Thanks!