PUT Request on `/logger` waking up suspended machines every few minutes

Questions / Help
,
1

Hi there!

My app is configured to suspend machines automatically. So after sitting idle for a few minutes, machines suspend to not infer cost while it’s not doing anything.

It’s also not HTTP, there’s no ports 80 or 443, only some random high-order TCP port.

Now, after it sits suspended for 1 or 2 minutes, the machine supposedly receives a strange API request.

The API server received a Put request on "/logger" with body "{\"log_path\":\"logs.fifo\",\"level\":\"info\"}".

(remember, my app isn’t even HTTP, i.e. there is no /logger route or anything.)

What could be triggering these requests that wake up my machine even though it’s not serving actual requests?

2

Could you explain more about this? Does your app not offer an HTTP listener? Why do you say this request is not HTTP?

Is your listener listening on a specific port number?

3

No, I don’t even have an http_services part in my fly.toml.
See my actual services config:

[[services]]
  protocol = 'tcp'
  internal_port = 28000
  auto_stop_machines = 'suspend'
  auto_start_machines = true
  min_machines_running = 0

  [[services.ports]]
    port = 28000

I’m not saying the request isn’t HTTP, it obviously is HTTP, it’s a PUT request. The thing is, I don’t have any ports open that even accept HTTP requests. The HTTP request goes into the flyd proxy though and triggers the server to wake up.

4

I can’t add much value, but this was noticed in September: Suspended machines woken up by PUT from fc_api every few mins

… and fc_api apparently is firecracker. See: Docker without Docker · The Fly Blog

5

Righto, thanks for the detail. I wonder if I’d ask some questions about what you want to achieve. If you don’t offer a listener, then protocol, internal_port, and port don’t make sense, in my view. I assume that whatever is connecting to them is something from Fly itself (maybe to configure loggers, based on the content of the request).

So after sitting idle for a few minutes

How are you defining ‘idle’? Is this intended to be based on CPU level or similar?

7

I actually read that thread too, but sadly there was no solution :frowning:

I do offer a listener, it’s just not HTTP. I have a Java process (specifically, Minecraft) running on that high-order port. It doesn’t accept HTTP requests, it’s a Minecraft server. That’s just why I’m baffled to why my machine even gets strange HTTP requests if I don’t even have a web server running or anything :face_with_spiral_eyes:

I’m just using the auto_stop_machines = 'suspend' setting in my fly.toml.
See the reference here.

tl;dr: flyd checks every few minutes if there are active connections to that machine and if not, it suspends it.

1 Like
8

Firecracker is what we use to start and stop your VM. It has a REST API.

I don’t know what is causing the problem you are seeing, but it is not because your application is getting HTTP requests.

9

Okay yeah, that’s what I was thinking.
Thank you!

Now… do you think I could stop this from happening somehow? :joy:

Or maybe there is an option to have the autostart handler only accept requests on some specific port?

10

Let’s just recap where we are.

Your app is configured to autostart when it receives a request (on some “random” port that you have chosen).

In the process of autostarting, some component internal to fly.io is being overly chatty, giving the false impression that that was the cause of the restart, rather than being a part of the process of autostarting. So let’s ignore that for now.

That leaves the question: why is your app receiving periodic requests? I can’t answer that without more data.

As an example, my app is using sentry which has uptime monitoring. It does that by sending periodic requests. This surprised me, but being able to examine the HTTP headers made it easy to identify why this was occurring.

11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.