My Fly subdomain is getting crawled

Questions / Help
1

I have a Fly hosted website: https://ridly-rubbish-removal.fly.dev/.

The web domain I have purchased and pointed towards the fly.io site is https://ridlyrubbishremoval.com.au/.

I have a robots.txt file in my code that allows google to crawl my site when I am in production. My issue is that the robots are crawling my fly provided domain (https://ridly-rubbish-removal.fly.dev/) and my custom domain (https://ridlyrubbishremoval.com.au/).

How do I stop the fly.dev address appearing in Google and encourage only my own domain (ridlyrubbishremoval.com.au) to be used?

Do I need to build some custom logic in to the robots.txt or is there a more accepted or standardised way to do this? Thanks in advance!

2

I have set up a redirect now at the root layer of my app via middleware, but the question above still stands. Can I use the fly.toml to achieve the same effect at a server level?

3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.