Timeout on CreatePostgresCluster.app

I’m trying to launch a launch a Phoenix umbrella application and should have all the right configurations in place. I run flyctl launch in my directory and it’s asking me if I want to setup a Postgresql database now. I do, but when it’s then trying to create the app for the database I get the following error failed creating the Postgres cluster purple-butterfly-5852-db: Timeout on CreatePostgresCluster.app. When I go to my dashboard it looks like the database app have been created correctly and is running, but my app isn’t and when I try to then subsequently run flyctl deploy I get the following error: failed to fetch an image or build from source: Mounts source volume "vol_" does not exist.

I’m all new to Fly.io and have never used it before, so I’m not sure if these errors are related. Maybe someone can tell me if I’m doing something wrong or I’m just having a really bad experience?

Thanks in advance, and please feel free to ask questions if you feel like there is vital information missing.

Is this in the UK (lhr) region, by any chance?

If so, I’m getting some issues there too. Requests not hitting it, timeouts etc.

Seems like Fly are on the case if so. There is a status update there:

If not, take a look at fly volumes list and see if that reveals anything about the volume I assume it failed to make.

The other option with database apps is to deploy them without a database (so, say ‘N’ when it asks). And then after/if it deploys, then attach the database to it. Which sets the DATABASE_URL secret within the app. You shouldn’t need to two-step it, however it can separate out issues into app and database related ones, making them (possibly) easier to debug.

But here it sounds like that may not help anyway.

Hi @greg,

No this was in the Amsterdam (ams) region. I’m going to give it another try today and maybe do what you said of first deploying the application and then the database separately.

Thanks!

Hi @greg,

I’ve just had another go and this time only trying to deploy the application. I’m still getting the same error:

==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
Error error connecting to docker: Mounts source volume "vol_" does not exist

Furthermore I did try and list the volumes and could see that there wasn’t any created, so I tried to create one with the following command: flyctl volumes create vol_ --app ancient-wave-7706. This resulted in this error:

? Select region: Amsterdam, Netherlands (ams)
Error failed creating volume: Timeout on CreateVolume.app

Do you think my whole account is broken somehow? Can and should I reach out to someone more officially you think?

Ah. Given it’s failing without the database step, that does indeed help with debugging as the issue must then be app-related.

It seems to be expecting a mounted volume … Does your fly.toml file reference one that actually does not exist? It’s this part:

If it does and that volume actually does not exist, it would explain why it’s not happy. Try removing that mount block and see if the deploy proceeds.

If not … then maybe that mentioned “vol_” is used internally by Fly, by its innards (either by the app, or the builder). In which case it sounds like it’s failing to be made (timeout, like when you tried to make one) and so the app is not deployed.

You could try a different region to see if that helps (if there is any current issue at ams with volumes).

Else someone from Fly will respond here at some point, likely soon, to diagnose it.

I haven’t tried to create a database today, so I’m not sure if that also still fails with the timeout error. But my guess it that it does, since it does the same thing when I’m trying to create a volume. Just a different .app failing.

When I do flyctl launch I do not yet have a fly.toml. I just checked the one that was created by my attempt from before and that doesn’t even contain a mount block.

I can give it a try with a different region, let me just clean everything up and try that right away.

I just tried Frankfurt (fra) and I get the same problem. I believe this is something universally wrong with my account. That or I’m cursed. Not completely sure what would be the issue here. I simple run flyctl launch from a Phoenix application folder.

Ok. I wasn’t sure since some people have a fly.toml file already, pre-deploy, either copied from another app of theirs, or from a Fly demo (e.g hello-rails/fly.toml at main · fly-apps/hello-rails · GitHub).

Try a different region is always worth a shot to rule out ams as the issue. Update: ah, you have. Not that then!

Yep if volumes generally are failing, a database would too. Since a db also needs a volume. In which case this will be one for Fly :frowning:

Thank you for taking the time @greg. I will leave this topic open and hopefully someone from fly.io will have time to look at it.

1 Like

I’m experiencing the same error when trying to launch an app with remote docker builder. I’ve tried personal & org, multiple regions, the error is always the same:

Error failed to fetch an image or build from source: error connecting to docker: Mounts source volume "vol_" does not exist

I will try again later today when I get home, but is also suspecting to still be getting the same error @teamon. I’m hoping that someone will have some time to take a look at this issue.

@teamon I actually just now saw that there was an official response in this thread: Creating new volumes timing out where they say that they are working on it. They say that creating the app in another region should work, but I tried that earlier last week where I saw the same error. Maybe worth a try now?

Hi Kasper, the Mounts source volume "vol_" does not exist error is showing up because the app you’re trying to deploy doesn’t have volumes provisioned.

Can you try running fly volumes create <volume name> --region ... --size ... and then add [mount] in your fly.toml. This doc has an example of what you should do.

Hi @rahmatjunaid,

Thank you for getting in touch! Shouldn’t a mount be automatically created if I’m doing flyctl launch in a Phoenix project? Not in any of the guides that I have read from you guys does it tell me to create a volume first?

I’m experiencing the same problems trying to remote-build my app. My guess is the ams region has trouble creating volumes, which causes the remote builder to be spawned without the ‘vol_’ volume, causing the error.

The issue seems to be specifically related to the ams region, as I am able to create a volume in lhr, but not in ams:

% fly deploy --remote-only
==> Verifying app config
--> Verified app config
==> Building image
Error failed to fetch an image or build from source: error connecting to docker: Mounts source volume "vol_" does not exist


% fly volumes create myapp_data --region ams --size 40

Error failed creating volume: Timeout on CreateVolume.app


% fly volumes create myapp_data --region lhr --size 40

        ID: vol_5podq4qg136vg8w1
      Name: myapp_data
       App: ***
    Region: lhr
      Zone: ad0e
   Size GB: 40
 Encrypted: true
Created at: 19 Apr 22 18:14 UTC

Is there any way to force a remote builder being created in a specific region? Not sure whether those are located in the closest region by default, which would be ams for me.

Note: as I am currently working on a M1 Mac, building remotely is the only option for me. Which means I am currently unable to build my app.

@tomgeene As regards your question:

Is there any way to force a remote builder being created in a specific region? Not sure whether those are located in the closest region by default, which would be ams for me.

… in my experience: no. When I create a new app (with no builders), even if I specify a far-away region (like fly launch --region iad) the remote builder app that gets created is still in lhr (the closest region to me, hence why you get one in ams). Which got me thinking … can I move it manually to another region? Like iad. I tried fly regions set iad --app fly-builder-name-here … and … no, doesn’t work. It complains: Error App 'fly-builder-name-here' uses volumes to control regions. Add or remove volumes to change region placement. Which makes sense. Builders make and use a volume. And so that volume pins the builder to the region. Confirmed with fly volumes list --app fly-builder-name-here. Now … could that volume be moved or a new one made to then pin to a different region? Hmm. Maybe. I wonder what would happen if I made and attached a volume in iad, and deleted the one in lhr (the one the builder made) … I haven’t tried doing that, but you could have a try if you want. But hopefully someone from Fly can respond before. Since if ams gets sorted, this wouldn’t be needed. As it’s a total hack.

I see the same timeout issue in FRA.

Futhermore I wasn’t able to deploy a new application because of the Mounts source volume "vol_" does not exist error. Hint from @rahmatjunaid wasn’t helpful unfortunately. My app has no volumes. Nevertheless I tried but was not able to create one as volume creation timed out in FRA.