I think I’ve misconfigured my app for serving Prometheus metrics. How can I verify whether my apps are getting scraped by Prometheus in the same way that I would check the status page on an instance I was hosting myself? In this case I suspect I forgot to remove the password protecting the metrics endpoint, but I don’t know for certain. Additionally there’s no way to tell the built-in Prometheus instance that it should use a password.
Fly doesn’t support authenticated metrics endpoints. For verifying, you might want to see if your metrics collector has a verbose logging mode or something similar. What’s your runtime?
The easiest way to verify scrapes is to setup grafana and try and query one of your metrics.
You can also SSH into your VM and run something like:
curl http://127.0.0.1:9000/metrics
That curl will need to work for our collector to work.
Hi Joshua.
Is there a way to check if the request to /metrics is coming from fly.io and not from the outside world?
We don’t want to expose our metrics on a publicly accessible URL.
Thanks!
The only ports exposed to the internet are those listed in [services] in fly.toml. And you could verify this by checking your metrics logs to see the request source IP.
1 Like