# Error: no space left on device - Can not SSH, resize or backup data on volume

**URL:** <https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127>\
**Category:** Questions / Help\
**Created:** [May 12, 2022, 5:16pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127 "2022-05-12T17:16:55Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![sxstem](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/sxstem/32/3399_2.png) [@sxstem](https://community.fly.io/u/sxstem)\
**Post date:** [May 12, 2022, 5:16pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/1 "2022-05-12T17:16:55Z")

</div>

- can not SSH to app id `d7e4617c` via `fly ssh console`
- don’t know how to check size of the volume
- can not resize the volume
- can not backup the volume (which is most important before do something else)

Getting this:

```auto
events.js:377
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] throw er; // Unhandled 'error' event
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] ^
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info]Error: ENOSPC: no space left on device, write
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info]Emitted 'error' event on Logger instance at:
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] at WriteStream.onStreamError (/var/lib/ghost/versions/4.38.0/node_modules/bunyan/lib/bunyan.js:627:18)
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] at WriteStream.emit (events.js:400:28)
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] at emitErrorNT (internal/streams/destroy.js:106:8)
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] at emitErrorCloseNT (internal/streams/destroy.js:74:3)
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] at processTicksAndRejections (internal/process/task_queues.js:82:21) {
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] errno: -28,
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] code: 'ENOSPC',
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info] syscall: 'write'
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info]}
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info]Main child exited normally with code: 1
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info]Starting clean up.
2022-05-12T17:02:20Z app[9e1b7c2f] fra [info]Umounting /dev/vdc from /var/lib/ghost/content

```

Any options?

Thanks

---

<div class="post-metadata">

**Author:** ![sxstem](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/sxstem/32/3399_2.png) [@sxstem](https://community.fly.io/u/sxstem)\
**Post date:** [May 13, 2022, 11:08am UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/2 "2022-05-13T11:08:47Z")

</div>

Can someone please advise me how can I resize volume or backup the data on unaccessible instance? I can create new one with bigger size, but I can not mount two volumes to instance, so HOW can I backup the original volume or transfer files to the new bigger one?

Thank you

---

<div class="post-metadata">

**Author:** ![greg](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/greg/32/2119_2.png) [@greg](https://community.fly.io/u/greg)\
**Post date:** [May 13, 2022, 12:03pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/3 "2022-05-13T12:03:30Z")

</div>

As far as I know volumes can not be resized. In which case it will be a case of creating one with a bigger size and then copying the data to it.

Looking through [Volumes](https://fly.io/docs/reference/volumes/) it doesn’t mention snapshots however I recall Fly saying they take snapshots (aka backups) of volumes 🤔.

```auto
flyctl volumes list

```

… and then armed with its ID …

```auto
flyctl volumes snapshots list volume_id_goes_here

```

Which should show the snapshots Fly have taken. That doesn’t help you with getting the data _off_ the volume though - that may be one for someone from Fly when they see this.

I would wait for their go-ahead before making any changes as they may be able to sort it at their end.

---

<div class="post-metadata">

**Author:** ![sxstem](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/sxstem/32/3399_2.png) [@sxstem](https://community.fly.io/u/sxstem)\
**Post date:** [May 13, 2022, 12:16pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/4 "2022-05-13T12:16:36Z")

</div>

Thanks for the reply,

yeah I read something about snapshots, but there is no mention of it in the documentation.

I really don’t want to somehow damage the data, so I am looking for recommended way how to solve this… I think it can happen to anyone at any time, because there is no clear way to check or resize volumes, and even small increase in traffic can cause the disk to fill up because of logs etc.

At least if you could mount more than one volume to the instance and then `ssh` and `cp` data.

---

<div class="post-metadata">

**Author:** ![greg](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/greg/32/2119_2.png) [@greg](https://community.fly.io/u/greg)\
**Post date:** [May 13, 2022, 12:52pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/5 "2022-05-13T12:52:56Z")

</div>

True.

There _may_ be a way (not documented so don’t know) for them to make a new volume from a snapshot. In which case what _may_ be possible is to create a new volume of a bigger size based on that snapshot. No idea given the file system. Would save you copying the data manually.

There may be metrics available for volumes via their API. But even so, you’d still need a way to _act_ on that to resize a volume. Auto-resizing volumes would be even better.

---

<div class="post-metadata">

**Author:** ![sxstem](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/sxstem/32/3399_2.png) [@sxstem](https://community.fly.io/u/sxstem)\
**Post date:** [May 14, 2022, 11:10am UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/6 "2022-05-14T11:10:56Z")

</div>

Whole website is down for third day… Is there any support from Fly team or this is just for hobby projects?  
Hard to imagin host here some serious project…

Can someone from Fly please advice or help me to fix website (resize volume) or at least backup the data from the volume?

Thank you

---

<div class="post-metadata">

**Author:** ![jerome](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/jerome/32/17_2.png) [@jerome](https://community.fly.io/u/jerome)\
**Post date:** [May 14, 2022, 11:34am UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/7 "2022-05-14T11:34:34Z")

</div>

I’m taking a look.

Can you tell me the app’s name?

---

<div class="post-metadata">

**Author:** ![qqwy](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/qqwy/32/7475_2.png) [@qqwy](https://community.fly.io/u/qqwy)\
**Post date:** [May 14, 2022, 11:51am UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/9 "2022-05-14T11:51:32Z")

</div>

Fly takes snapshots of volumes, but IIRC currently they can only be restored for VMs running Fly’s special Postgres image.

---

<div class="post-metadata">

**Author:** ![jerome](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/jerome/32/17_2.png) [@jerome](https://community.fly.io/u/jerome)\
**Post date:** [May 14, 2022, 12:09pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/10 "2022-05-14T12:09:23Z")

</div>

I manually resized your volume to 6GB. There is no automated process for this.

Your app is now back up.

While it was full, I managed to bring your VM back up by overriding it’s `cmd` with `["sleep", "1000"]`. In your fly.toml, that looks like:

```nohighlight
[experimental]
cmd = ["sleep", "1000"]

```

This would’ve allowed you to SSH in and perform maintenance of some kind.

I’m (a little bit) surprised ghost refuses to boot entirely if the disk is full.

---

<div class="post-metadata">

**Author:** ![sxstem](https://sea1.discourse-cdn.com/fly1/user_avatar/community.fly.io/sxstem/32/3399_2.png) [@sxstem](https://community.fly.io/u/sxstem)\
**Post date:** [May 14, 2022, 12:26pm UTC](https://community.fly.io/t/error-no-space-left-on-device-can-not-ssh-resize-or-backup-data-on-volume/5127/11 "2022-05-14T12:26:43Z")

</div>

Great. Thank you!

Good to know about the “sleep” workaround.
