To answer my own Qs:
Error: failed to connect to fly machine: Supposedly started, and not stopped, but: connection refused by remote server
Happens when the proxy is trying to connect to a machine VM that has stopped listening on service ports but is yet to exit. In case of our app, this was a result of Node talking its own merry time in closing out other resources long after it had stopped listening for incoming connections.
Ideally, once health-checks are up and working for Machines, this scenario should auto-recover as the Machine should be auto-restarted once health-checks start failing (which they will if process isn’t listening on service ports): How do I change the restart policy for Machines? - #8 by ignoramous
Error: while TCP-proxying data to/from app: failed to copy
Most probably the client / server half-closed the connection early.
Error: an error occured while trying to wake up a fly machine
Not sure, but this auto-recovers (as in, the Machine wakes up the next time, just fine).