I finally made it through the whole migration.
fly launch created an app server with pg_dump version 11. Setting the database URL secretes and running the migration command on the database app server (which gets version 14 from fly launch) instead of the main app server worked fine with my version 13 database on Heroku (though it did time out and fail once in my testing).
I’m going to call this solved. Hope my experience was helpful in clarifying the docs.