After reading the blog post and the docs page, I’m intrigued by the availability of metrics in api.fly.io! I’d love to use those, but there’s only one problem: I’m running my own prometheus & alerting infrastructure and I’d like to have the metrics exposed by my fly.io apps be imported there.
Is there any API endpoint that I’m not finding, which I could point my own prometheus at? I’d be happy with a single “exporter” kind of endpoint, but a federation endpoint would work just as well, I think.
Unfortunately we don’t provide that.
I think we could provide a way to export data in JSON format or CSV, and then you could scrape it and re-export it by parsing the JSON into prometheus’ text exposition format.
Would that suffice?
We don’t have such a endpoint right now, but we could make one. We’d need to make sure it wasn’t a big stress on our cluster.
Depending on what metrics you’re after, you could also setup a Wireguard peer on your scraper and use DNS lookups to discover and scrape your internal app metrics.
So! That’s a good point you raise, kurt: I am interested only in the metrics that fly.io adds to those exposed by my service (I can easily scrape them via wireguard, and do): What I’d really care about though is to know what error rates / response codes / durations fly.io is cognizant of.
So - I guess my desire is less, “give me a way to federate my prometheus with yours” and more, “can I have a prometheus-compatible HTTP endpoint where I can retrieve fly’s metrics?”
I think scraping might be the wrong way to “sync” metrics for this case. We might be able publish metrics through the same NATs service we use for logs, though. It wouldn’t take much go to subscribe and make your own exporter.
This sounds great actually. Is this something that is on you roadmap on the short term?