This has been brought up in the past but in the post it was suggested that the metrics should automatically resurface after a while. It’s been over 2 weeks now and they are still not back though.
Can you take a look for us? <3
This has been brought up in the past but in the post it was suggested that the metrics should automatically resurface after a while. It’s been over 2 weeks now and they are still not back though.
Can you take a look for us? <3
@kurt Hey. The metrics are still not shown for us so I suspect this is something different as you mentioned in the other post that they should re-appear after a short while. It’s been a month now though so I doubt that’s still happening?
Can you check please? It’s a bit worrying to not be able to monitor our db and just “hope that everything will be fine”
Thanks for reminding us! This definitely would seem like a different issue if metrics have been offline for 2 weeks. We can certainly dig into this more.
Would you mind sharing which organization is having the problem, and confirm which specific apps are affected?
Thanks @eli. The organization is avantgarde-finance
. Within that org, all postgres apps are affected (because we moved both of them).
Awesome, thank you. We took a look at your apps, and the metrics themselves are working, but our logging pipeline wasn’t sending them to the right place.
We’ve applied a fix for this, and you should now be able to get metrics working again by restarting the apps’ instances-- please let us know if that doesn’t do the trick!
Hey @eli. Thanks for the update.
I’ve restarted our db apps but it seems they are still stuck trying to finish the restart for about an hour now. I simply ran fly apps restart <app>
.
Anything else I need to do?
I am not seeing the metrics yet.
@eli I have re-created one of our dbs under a different name now and metrics are back for that one. But for the other one (the temporal-db instance), they are not.
They are back for both instances now. Thanks!
Thank you for keeping us updated with that extra info, and confirming that things are back up!
An aside: before the third most recent post, we thought that the app you’d recreated elsewhere (not temporal-db, but the other, original one) needed fixing as its scale count was set to 0. We bumped that up as part of fixing it.
In retrospect it sounds like that this could have been intentional. If you’re not planning on keeping it around, you may want to either reset its scale count or destroy it (since it still also has volumes up)
Yeah I scaled that down to 0 after recreating the db with a new name and migrating the data over from the old one. Wanted to keep it around to see if everything’s fine with the new db first. Will delete it soon.