Possible error in docs regarding log output using supervisor

I have been experimenting with running multiple processes in-app and have got it working using supervisord.

I then started looking where to send logs (rather than to local files).

I notice your example conf on Running Multiple Processes Inside A Fly.io App has …

[supervisord]
logfile=/dev/stdout
...

… however in my experiments, that does not work. It crashes supervisor.

Using /dev/stdout for a process log output works fine. But not for supervisor itself. So it may be worth changing that example to use /dev/null as you say underneath that you don’t want it to generate a log file. Else a file path like /var/log/supervisor.log if you do.

Unless you have got it to work using /dev/stdout. Maybe it’s down to the OS. I’m playing with Alpine linux. And it seems confirmed by others on stackoverflow e.g

:thinking: that should not crash. Are there any errors in the logs?

Yes, it’s not happy. Even as root. If I switch to using /dev/stdout

[supervisord]
user=root
nodaemon=true
loglevel=info
;logfile=/dev/null
logfile=/dev/stdout
pidfile=/var/run/supervisord.pid

… the image still builds and deploys, naturally. But when Fly runs the entrypoint (which is supervisor), it crashes:

2022-03-18T20:08:26Z   [info]Preparing to run: `supervisord -c /etc/supervisord.conf` as root
2022-03-18T20:08:26Z   [info]2022/03/18 20:08:26 listening on [fdaa:0:5d9:a7b:23c2:c09d:8057:2]:22 (DNS: [fdaa::3]:53)
2022-03-18T20:08:26Z   [info]2022-03-18 20:08:26,651 INFO Set uid to user 0 succeeded
2022-03-18T20:08:26Z   [info]2022-03-18 20:08:26,651 INFO Set uid to user 0 succeeded
2022-03-18T20:08:26Z   [info]Traceback (most recent call last):
2022-03-18T20:08:26Z   [info]  File "/usr/bin/supervisord", line 33, in <module>
2022-03-18T20:08:26Z   [info]    sys.exit(load_entry_point('supervisor==4.2.2', 'console_scripts', 'supervisord')())
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/supervisor/supervisord.py", line 359, in main
2022-03-18T20:08:26Z   [info]    go(options)
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/supervisor/supervisord.py", line 369, in go
2022-03-18T20:08:26Z   [info]    d.main()
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/supervisor/supervisord.py", line 72, in main
2022-03-18T20:08:26Z   [info]    self.options.make_logger()
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/supervisor/options.py", line 1507, in make_logger
2022-03-18T20:08:26Z   [info]    self.logger.info(msg)
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/supervisor/loggers.py", line 327, in info
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/
upervisor/loggers.py", line 345, in log
2022-03-18T20:08:26Z   [info]    handler.emit(record)
2022-03-18T20:08:26Z   [info]  File "/usr/lib/python3.10/site-packages/supervisor/loggers.py", line 227, in emit
2022-03-18T20:08:26Z   [info]  File "/usr/lib/pyth
2022-03-18T20:08:26Z   [info]  File "/usr/lib/pyth
n3.10/site-packages/supervisor/loggers.py", line 264, in doRollover
2022-03-18T20:08:26Z   [info]    if not (self.stream.tell() >= self.maxBytes):
2022-03-18T20:08:26Z   [info]OSError: [Errno 29] Invalid seek
2022-03-18T20:08:27Z   [info]Main child exited normally with code: 1
2022-03-18T20:08:27Z   [info]Starting clean up.
--> v39 failed - Failed due to unhealthy allocations - rolling back to job version 38 and deploying as v40 

--> Troubleshooting guide at https://fly.io/docs/getting-started/troubleshooting/
Error abort

Seems from that it’s because it’s trying to seek: Invalid seek. So that’s ok with a real file, but not with stdout.

Interesting! Check this out: Invalid seek when using logfile = /dev/stdout · Issue #1078 · Supervisor/supervisor · GitHub

1 Like

Ah … yes! That’s it. Adding that line then does let it seek and it doesn’t crash.

All good then. Thanks.

1 Like

Sounds like we need to update our docs. Can you show us your supervisord config?

Ah, well no, the weird thing is I’ve checked your conf in your docs and it does have that line in there :thinking:

Well that’s weird! Maybe I was hacking around with the lines in there (like I’ve just checked the stackoverflow answer also has that logfile_maxbytes=0 and despite that it also says /dev/null) to get it to work. So maybe I removed it.

But with that line and /dev/stdout, it is now working.

My fault it seems.

1 Like