How many secrets do your builds need?
You can run --build-arg ENV_VAR1=${ENV_VAR1} to set build time environment variables. If you’re doing deploys from GitHub Actions or another CI, it should be reasonably straightforward to script something up that passes a whole bunch all at once.
We designed the secrets this way intentionally. Admittedly, we didn’t expect Heroku to drive away their users, or we’d have done some work to make this part smooth. 