I think this is now related to this “bigger” issue right now. For me it seems that almost every instance is down.
that seems the case, my dev instance is down and im still hitting fly api error. Tried all options mentioned above and still persists.
We are running to the same problem, tried everything above but the issue persists.
Can’t launch no matter what region I try:
Error You hit a Fly API error with request ID: 01GGF3QM4EH3073GRDVPGKKQJQ-ams
I’ve been getting the same error for the last three hours:
WARN Failed to start remote builder heartbeat: You hit a Fly API error with request ID: 01GGF4E1NRZ4D2SQ81YJKQ7BYM-maa
Error failed to fetch an image or build from source: You hit a Fly API error with request ID: 01GGF4EFJSJEVSZYX6NM73EE8R-maa
Yesterday i was waiting for my deploy finished but i saw here in community that was some issues about deployment process. Then, now my application is PENDING status and my users cannot access my services in PRODUCTION.
Please, there’s any solution about this case? This status “PENDING” not changing by hours.
Thanks in advance
Yeah unfortunately there is an ongoing incident, but we’re deep into the weeds with it. Keep an eye on our status page for updates.
There’s any way to redirect all traffic that hit my fly.dev application to another url ? My application remains offline. I need help!
I am having the same issue. It’s frustrating not to be able to deploy hotfix. >8h downtime is insanely long.
It’s alive!
I already switched back to Heroku. This kind of downtime is extremely damaging to my business.
hi @eli ,
I got the error Error You hit a Fly API error with request ID: 01GMCPS24Y8RQDBR50E5BBZ9YW-sin
when allocating new ipv4 flyctl ips allocate-v4 -r yul -a spread-s1
thanks
Having the same issue right now. I’ve destroy/recreate app and still getting the same thing.
Also just hit this error both in CI and locally
Here is my local request ID in case its helpful to anyone:
WARN Failed to start remote builder heartbeat: You hit a Fly API error with request ID: 01GRERYN8HZCFVS5RM6JNFKK6E-lga
Error failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: 01GRERYNVW86MJAC4N785NYVX4-lga
Sorry if this is causing weekend alarms :hugops:
I have been getting this issue since the last time I deployed.
I was having wierd issues, where my API would sometimes have the “OLD” code, instead of the new. This is unacceptable.
When I tried to redeploy, I see:
WARN Failed to start remote builder heartbeat: You hit a Fly API error with request ID: ABCDEFGHIJKLMNOPQRSTUVWXYZ-yyz
Error: failed to fetch an image or build from source: error connecting to docker: You hit a Fly API error with request ID: ABCDEFGHIJKLMNOPQRSTUVWXYZ-yyz
I then tried to destroy
the machine and do another fly launch
, but it gives back the same error! Now… MY APP IS COMPLETELY USELESS.
What should I do?!
Here is my fly.toml and my Dockerfile
Just a follow-up! I began look into Docker as a possible root-cause. When I was running:
docker build -t spg2 .
I started seeing an error:
Step 5/12 : RUN gem install bundler && bundle config set path 'vendor/bundle' && bundle config set deployment 'true' && bundle install --jobs=4 --without development test
---> Running in 110e7fa1fca8
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the /usr/local/bundle directory.
/usr/local/lib/ruby/3.2.0/rubygems/installer.rb:712:in `verify_gem_home'
/usr/local/lib/ruby/3.2.0/rubygems/installer.rb:902:in `pre_install_checks'
/usr/local/lib/ruby/3.2.0/rubygems/installer.rb:302:in `install'
/usr/local/lib/ruby/3.2.0/rubygems/resolver/specification.rb:104:in `install'
/usr/local/lib/ruby/3.2.0/rubygems/request_set.rb:194:in `block in install'
/usr/local/lib/ruby/3.2.0/rubygems/request_set.rb:182:in `each'
/usr/local/lib/ruby/3.2.0/rubygems/request_set.rb:182:in `install'
/usr/local/lib/ruby/3.2.0/rubygems/commands/install_command.rb:214:in `install_gem'
/usr/local/lib/ruby/3.2.0/rubygems/commands/install_command.rb:230:in `block in install_gems'
/usr/local/lib/ruby/3.2.0/rubygems/commands/install_command.rb:223:in `each'
/usr/local/lib/ruby/3.2.0/rubygems/commands/install_command.rb:223:in `install_gems'
/usr/local/lib/ruby/3.2.0/rubygems/commands/install_command.rb:169:in `execute'
/usr/local/lib/ruby/3.2.0/rubygems/command.rb:323:in `invoke_with_build_args'
/usr/local/lib/ruby/3.2.0/rubygems/command_manager.rb:251:in `invoke_command'
/usr/local/lib/ruby/3.2.0/rubygems/command_manager.rb:191:in `process_args'
/usr/local/lib/ruby/3.2.0/rubygems/command_manager.rb:149:in `run'
/usr/local/lib/ruby/3.2.0/rubygems/gem_runner.rb:51:in `run'
/usr/local/bin/gem:10:in `<main>'
The command '/bin/sh -c gem install bundler && bundle config set path 'vendor/bundle' && bundle config set deployment 'true' && bundle install --jobs=4 --without development test' returned a non-zero code: 1
I never seen this before; afterall, I’ve deployed and built the Docker container before.
Updating Docker
After debugging for hours, I tried ugprading Docker as suggested on Stackoverflow here.
For whatever reason, you can’t go to Docker Desktop > Check For Updates. That does NOT work.
Instead, I had to manually reinstall it from the Docker website. Before, I was on 19.x
, now I am on Docker version 24.0.2, build cb74dfc
Running Docker Build
Now, I can run: docker build -t spg2 .
and do docker run -p 3000:80 -e RAILS_ENV=development -e PORT=3000 spg2
I thought this would fix the:
Error: failed to fetch an image or build from source: error connecting to docker:
You hit a Fly API error with request ID: XYZ
But it DID not fix anything.
fly doctor
Everything checks out and looks good here:
fly doctor
Testing authentication token... PASSED
Testing flyctl agent... PASSED
Testing local Docker instance... Nope
Pinging WireGuard gateway (give us a sec)... PASSED
App specific checks for spg:
Checking that app has ip addresses allocated... Nope
No ip addresses assigned to this app. If the app is not intended to receive traffic, this is fine.
Otherwise, it likely means that the services configuration is not correctly setup to receive http, tls, tcp, or udp traffic.
https://fly.io/docs/reference/configuration/#the-services-sections
No ipv4 or ipv6 ip addresses allocated to app spg
Build checks for spg:
Checking docker context size (this may take little bit)... PASSED (76 MB)
Checking for .dockerignore... PASSED
Final Thoughts
Nothing worked.
I am pretty sure it is something on Fly’s side from what I can see from other conversations:
- Failed to start remote builder heartbeat - #6 by rolznz
- Dashboard error: You hit a Fly API error with request ID - #8 by rugwiro
Going to email billing@fly.io
or get help on Twitter.