Could you clarify the Machines exec contract for flyctl 0.4.123 with fly-go v0.11.6?
On October 28, a 2,365-byte Python command succeeded. A 79,067-byte command containing a 73,160-byte ASCII argument returned empty stdout and 175 bytes of stderr. The CLI exited 0, but its output omitted the Machine exit code. A later entry-marker check also returned no valid marker. Execution and termination remain unknown, and we have stopped probing.
Please clarify:
Maximum request-body, command, individual-argument and stdin sizes.
Legacy cmd parsing rules and supported command-array/stdin usage.
Whether timeout or client disconnection terminates the command and its descendants, and how termination is confirmed.
How to retrieve explicit exit-code and signal information without client-side omission.
Whether a supported read-only method can determine an outstanding exec request’s outcome without rerunning it.
This is a documentation clarification request. Please do not execute commands, retry requests or change resources. No spending or paid support is authorized.
It appears the 79,067-byte request was rejected before execution. The 175-byte stderr output should contain details on the rejection, likely a PayloadTooLarge error. exec request bodies are currently limited to 16 KiB. Right now this rejection is returned as HTTP 200 with the error in stderr and exit_code: 0, this is something I’ll look into fixing so the exit code/http status is clearer.
The legacy cmd string is split into arguments using shell-style quoting. For exact arguments, send command (an array) in a direct API call. stdin is ignored on Machines without containers.
A timeout returns 408 deadline_exceeded but does not stop the command, and a client disconnect doesn’t either. There’s no endpoint that returns an earlier exec’s result; GET /v1/apps/{app}/machines/{id}/ps shows whether a process is still running.
The API returns exit_code and exit_signal. fly-go v0.11.6 drops exit_signal, and flyctl only prints the exit code when it isn’t 0. You can call the API directly if you need both.
For large payloads, you can work around the request body size limit by writing the payload into the Machine (e.g. with fly ssh sftp) and exec’ing a short command that reads it.
Thank you for explaining the exec limit and timeout behavior. Before using the suggested SSH/SFTP staging route, we reviewed flyctl 0.4.123 (revision c6de331cc0f40847c29604ee4b3bbe226d9ec594) and have two follow-up questions:
The agent startup failure path appears able to attach a log excerpt to a Sentry report, and the analytics opt-out does not appear to disable that path. Is there a supported option that prevents private log contents from being exported during SSH/SFTP startup failures?
The reviewed SSH path uses InsecureIgnoreHostKey. Is there a supported way to verify the selected Machine’s SSH host identity, such as a trusted host key or host CA, for both staging and subsequent execution?
Which supported route would you recommend that provides both controls, and where are its failure and completion semantics documented? We are looking for a supported method, not a workaround that bypasses verification. We have not performed the proposed upload or SSH execution.
1. Sentry log excerpt. There’s currently no supported option to stop flyctl from sending agent startup failures to Sentry, including the log excerpt. The analytics setting only controls metrics, not error reporting.
2. SSH host identity. There’s currently no supported way to verify a Machine’s SSH host identity. The host key is generated each time the Machine starts, and no host key or host CA is published.
3. Given the above, perhaps avoid using SSH/SFTP for this. One alternative is to call the Machines API directly rather than through flyctl:
- Stage the payload with the Machine config files field (guest_path plus base64 raw_value), documented at https://fly.io/docs/machines/api/machines-resource/. Updating the config restarts the Machine. Use GET /v1/apps/{app}/machines/{id}/wait?state=started to confirm it’s running again.
- Run it with an exec command array that reads the staged file.
Out of curiosity, why do you need to send such a large payload to exec?
there is no point in verifying the SSH host identity, as the Machine identity is verified by you connecting to it via 6PN. it’s not possible to MITM a Machine SSH connection between you and Fly, nor by any other Fly Machine. what is your threat model with this?
if you are concerned about agent startup failure logs being able to be sent to Sentry (these logs do not contain any command stdout or other information about your SSH connection), you can use fly ssh issue and connect to the Machine via a regular OpenSSH client.
let me know if the above helps ^