sprites not shutting off

Just playing around with sprites. I have one that I just created and it doesn’t seem to be shutting off. I don’t have any open consoles, or execs to it and have been checking it intermittently with sprites ls. How long before they shut off? Under what conditions?

Hm… The (preliminary) Sprites docs seem to be partially down at the moment, but the copy in their GitHub repository currently states it as follows:

Sprites automatically hibernate when they’re no longer doing anything useful. The default timeout is 30 seconds, and it’s not configurable yet.

A Sprite is considered active when any of the following are true:

  • A command is executing (via exec or the console)
  • Data is being written to stdin
  • There’s an active TCP connection to the Sprite’s URL
  • A detachable session is running

If none of those are happening, the Sprite is considered idle. The inactivity timer starts at 30 seconds and resets whenever new activity is detected.

I’m guessing the 30 seconds elapsed long ago, :stopwatch:, though… Maybe it’s the third condition that’s the culprit?

Yes, I’ve definitely left it alone longer than 30 seconds.

# sprite ls
Sprites in organization <org>:

┌────────────────┬───────┬───────┐
│      NAME      │STATUS │CREATED│
├────────────────┼───────┼───────┤
│hardstuck-sprite│running│ 6h ago│
└────────────────┴───────┴───────┘

Total: 1 sprite(s)```

I may have had a tmux session going on before but I’ve since used a checkpoint to to back to v0 and pretty much have nothing going on anymore and it still seems to be stuck running.

# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
sprite         1  0.0  0.0   1012   524 ?        Ss   01:42   0:00 /.pilot/tini -- tail -f /dev/null
sprite         2  0.0  0.0   5796  1972 ?        S    01:42   0:00 tail -f /dev/null
sprite        51  0.3  0.0  10484  5892 pts/0    Ss   02:45   0:00 zsh --login
sprite        58  0.0  0.0  10192  4044 pts/0    R+   02:45   0:00 ps aux

I’ll probably just destroy it soon.

I’m assuming that in sprite ls the status will change from running to something else? Or is there some other indication of if the sprite is hibernating?

I just spun up a brand new sprite and that one is also not switching out of the running state when I check with sprite ls

The hibernating state is reported as cold, I think:

Assuming that it’s just repeating values from the REST API, there is also an intermediate warm. (Possibly that one means that the underlying machine is online, in a minimal sense, but not currently doing anything particularly productive.)

It might be worth checking from the REST API, since that’s the part that has publicly visible documentation at present, :thinking:

I have also run into this problem. I started a sprite last night and it’s still running this morning. No consoles were left open. I am running tail scale as a long running service though. Do those prevent the sprite from sleeping? Tailscale reports no connection to the sprite.

I also ran into this problem. I thought I might be able to shut it down myself by killing PID 1, but now it’s in a weird state where it says it’s still running but I can’t connect to it at all. So don’t do that!

2 Likes

For services, as opposed to detachable sessions, the draft docs on GitHub say that they “hibernate automatically”, although there’s not much detail that I’ve found beyond that (apart from the TCP quote above that probably you already saw).

The official docs still seem to be mostly offline, so it’s hard to say for sure. It might be worth checking for the detachable sessions. I think that’d be sprite exec, with no further arguments, :thinking:.

Hi all,

There are some issue with Sprites not suspending at the moment that we’re actively working on. No ETA on that right now, so the only reliable way to stop a Sprite that seems stuck is to destroy it.

Either cold or warm, cold being stopped and warm being suspended.

No, services won’t prevent the Sprite from sleeping. When functioning correctly, they will sleep after 30 seconds of inactivity. “Activity” here means an attached session, a command run through exec that is producing logs, or network requests to proxied ports or the public url.

This is fun to do as long as you made a checkpoint first! :slight_smile:


I (or someone) will post here when we have an update re: the problems with suspending.

3 Likes

Thanks for the clarifications… The latter is like (or exactly is) Firecracker’s suspend, then?

I have this problem too. I created a sprite yesterday and there are no active shells. It’s also not configured to accept any HTTP traffic, I am not using tailscale or anything. I used it, exited it, and assumed it would shut down. But it’s still up, and has been up for 21 hours now. I don’t really want to destroy it, because I spent quite a bit of time getting it set up!

Exactly is! That state reflects the state of the underlying Fly Machine for that Sprite.

1 Like

I’m having the same issue. Created a bunch of sprites and all of them stay in the running state except one is at warm.

I cannot tell what was keeping the instance up. The instance has very few processes:

⋊> ~ sprite x -s pool-1768397344747-1 sudo ps aux                                                                                                                             13:31:57
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
sprite       1  0.0  0.0   1012   524 ?        Ss   18:29   0:00 /.pilot/tini -- tail -f /dev/null
sprite       2  0.0  0.0   5796  1900 ?        S    18:29   0:00 tail -f /dev/null
root      1041  0.0  0.0  17180  6164 ?        Ss   18:32   0:00 sudo ps aux
root      1042  0.0  0.0   9800  4012 ?        R    18:32   0:00 ps aux

If there is a way to see our account usage/billing it’d be great. I’m a bit concerned how much I’m being billed.

I’m pretty excited about sprites. This issue is why i’m not using them yet. Excited for when the transition to “cool” becomes user-controllable, be that directly via an interaction with the API/CLI, or indirectly thanks to some known list of factors that I can govern and that deterministically result in a cooldown after some known amount of time.

Same issue, I want my free credits back :laughing: /s

(but definitely cannot use with real money yet)

=== Creating sprite: sleep-test-1768674717 ===
    cold

=== Touching sprite (quick exec) ===

=== Monitoring status (expecting 'warm' after ~30s) ===
    Waiting 45s...
    [45s] status: running
    Waiting 15s more...
    [60s] status: running
    Waiting 60s more...
    [120s] status: running
    BUG: sprite still 'running' after 120s idle

=== Cleanup ===
    Deleted sleep-test-1768674717

bash script gist
python script gist

Here reporting the same issue.

Basically my sprites is been running since I started, and it is not shutting off.

I think they fixed this issue a few days ago. Old sprites created before the issue might still have the issue. But new ones transit to warm/cold as expected.

1 Like

Oh wow all of my sprites have always only ever been running. I didn’t realize that might be absorbing my credits!

edit: if we need to create new sprites, a way to move get my disk contents onto a new sprite would be great. I’ve been just adding files willy-nilly, but if I need to worry about my sprites “going bad” then that won’t work so well.

Nice work! Just tried this on my end, the bug persists.

Hm… Was that testing with a new sprite, or an existing one?