Downscaled apps keep restarting semi-regularly

[paulrudy]

Thank you @Zane_Milakovic and @andie!

@Zane_Milakovic

As I understand is, scaling is not about sending all traffic to one app, until the soft limit is hit, but instead also sends traffic to closer nodes, in favor of having a app that has reduced latency.

Thanks, that was my misunderstanding. I had intended for one instance to just work as a backup in case of the primary region going down, or the unlikely chance of too much traffic. Is there a (relatively simple) method to set things up that way?

@Zane_Milakovic

What port are you on?

In my experience, anything on a common port, gets scanned by the internet at large. I see traffic days after putting a app on fly.io, where someone is looking for WP Admin or other common exploited routes.

The typesense app is on 443, so that also makes sense that it would be triggered occasionally by bots, etc.

The other app is Umami (website analytics), and it’s on 3000. As an analytics app, it receives a hit (with an authentication header) any time the website it’s tracking is hit. The website Umami tracks is hosted as a serverless function on Vercel’s “hobby” plan, and its primary location (San Francisco) is close to the primary locations of all of my Fly apps (Los Angeles). I couldn’t find anything about Vercel using a fallback region on my plan level, so it does seem a bit strange that an Umami instance that’s father away (Toronto) would be triggered by hits on the website.

Edit: actually, Umami has a frontend https web UI for management, so that’s likely to be getting hit by bots from various locations.

@Zane_Milakovic

Can you log incoming requests?

I’ll look into logging requests if I can learn of an easy to implement way, but at this point it’s kind of a matter of being curious and a bit over-obsessive about it rather than a cost issue.

@andie

This behaviour is where load balancing intersects with the auto start and stop feature. (Hoping to update docs for this soon!)

Thanks. It would be interesting to learn the ins and outs of that intersection!

@andie

But if this Machine is restarting on a very regular schedule that doesn’t match with incoming requests, then something else might be going on.

Definitely not very regular, just semi

@andie to your thought from my previous thread:

@andie

If you ever want to write up a doc on running typesense on Fly.io, let me know!

Aside from this particular issue, I pretty much have everything working with my cheapo typesense app: A primary typesense and a secondary backup, each running as single-node typesense instances, and both updated by another Fly app that is triggered by 1) typesense instance machine start and 2) github action on website deploy and cron. Don’t know if that would be at all useful to anyone but me, but I’ll try doing a write-up!