I followed the guidance in this post, and by utilizing the fly.toml and Dockerfile shared by @edmbn, I successfully resolved my problem. If anyone else is facing a similar issue, you might find the mentioned post helpful.
P.S. I still need to remove .env in .dockerignore to make it work. I recon there is a better way to handle this?