May I suggest you use metadata?
Updating envs could trigger machine restart but metadata can be dynamically set from within (machines already can do that, auth is already set) and from outside the machine and you can retrieve those as easily as well.
If I may also offer a more near-real-time solution we ship logs via NATS. That means if your machine can write logs such as Task 123 Progress: 10% or even raw json like {"type": "task_progress", ...} and you listen to logs from another machine or even an outside from fly server you can get ordered message updates.
- Announcement: Fly Logs over NATS
- Example in golang: GitHub - superfly/fly-log-shipper: Ship logs from fly to other providers
- Example in elixir: fly_nats_ex/lib/fly_nats_ex/watcher.ex at main · lubien/fly_nats_ex · GitHub ( I plan on making this a hex.pm library)
- https://nats.io/ might have more libraries