The root disk on a Fly VM is ephemeral: the root filesystem is rebuilt from the app’s container image every time that the VM is started. As a result, any files that you write (including software that you install) through the SSH console will be gone after the VM restarts, as you’ve seen.
Perhaps you can add Git to your container image, and then use environment variables and secrets as necessary to configure your environment?