The workaround @mayailurus posted here Trouble Connecting to Postgres Cluster - #4 by mayailurus did work for me.
fly m list -a <db-app-name> # consult the IP ADDRESS column
fly proxy 5432 fdaa:<hex-digits-from-above>:2
But there is certainly something that need to be fixed to make the fly proxy command work.