I should think the Fly Grafana stack would help you here. Can you find a way to send metrics to it e.g. via InfluxDB or Prometheus? I would have thought there would be a ready-made agent for it.
It is worth noting though that Grafana is a fairly big ecosystem, and not very well organised or documented. I am wading through it at present.
But if you can get it working, then adding alerts in the GUI should be quite straightforward.