Error failed to fetch an image or build from source...Unknown desc = quay.io/evl.ms/fullstaq-ruby:3.0.1-jemalloc-bullseye-slim: not found

First time attempt to migrate basic Rails app (with postgres) from Heroku. fly launch succeeded. fly deploy fails. Could I have help to resolve please. Thanks Daniel

=> ERROR [internal] load metadata for Quay 0.3s


[internal] load metadata for Quay


Error failed to fetch an image or build from source: error building: failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = Quay not found

Hey Simbed. Sharing fly.toml may help spot the problem.

At first I’d say the tag 3.0.1-jemalloc-bullseye-slim for repository quay.io/evl.ms/fullstaq-ruby doesn’t exist. I couldn’t find it at Quay

# fly.toml file generated for bridge-scores on 2022-11-23T16:37:28Z

app = "bridge-scores"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[build]
  [build.args]
    BUILD_COMMAND = "bin/rails fly:build"
    SERVER_COMMAND = "bin/rails fly:server"

[deploy]
  release_command = "bin/rails fly:release"

[env]
  PORT = "8080"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

[[statics]]
  guest_path = "/app/public"
  url_prefix = "/"

A friendly suggestion, use “preformatted text” to share code snippets in the forum, it eases readers’ life. TIP: You can edit a posted message, no need to add another reply.

Were you able to check whether the Quay image exists?

Yes sorry, I would love to, but it wasn’t obvious to me how to format text in my original post. In this reply I now see the options to do so.

ag 3.0.1-jemalloc-bullseye-slim for repository `Quay

means mothing to me. It was autogenerated as part of the fly launch yet seems to cause the fly deploy to fail?

I will check with the team about the autogenerated part, for now, try changing that to 3.0.1-jemalloc-slim and deploy

After second thoughts, 3.1.2-jemalloc-bullseye-slim seems better.

Pick your poison from quay.io/repository/evl.ms/fullstaq-ruby tags.

The ruby version needs to match the one specified in the following locations: “Gemfile.lock”, “Gemfile”, “.ruby_version”. See: flyctl/rails.go at ae87986faaed792d1ea6153529306a0b2972bc52 · superfly/flyctl · GitHub

This was a recent regression introduced in flyctl: Update base Rails image to point to Debian 11 (from Debian 10) · superfly/flyctl@c3b42b6 · GitHub . See the pull request for more background information: Update base Rails image to point to Debian 11 (from Debian 10) by bradgessler · Pull Request #1462 · superfly/flyctl · GitHub . I was the one to merge the pull request, but likely will be reverting it while looking for a better solution.

It looks like the latest debian image for each ruby version is as follows:

{"2.6.3"=>"2.6.3-jemalloc-stretch-slim",
 "2.6.4"=>"2.6.4-jemalloc-stretch-slim",
 "2.6.5"=>"2.6.5-jemalloc-buster-slim",
 "2.6.6"=>"2.6.6-jemalloc-buster-slim",
 "2.6.7"=>"2.6.7-jemalloc-buster-slim",
 "2.6.8"=>"2.6.8-jemalloc-bullseye-slim",
 "2.6.9"=>"2.6.9-jemalloc-bullseye-slim",
 "2.7.0"=>"2.7.0-jemalloc-buster-slim",
 "2.7.1"=>"2.7.1-jemalloc-buster-slim",
 "2.7.2"=>"2.7.2-jemalloc-buster-slim",
 "2.7.3"=>"2.7.3-jemalloc-buster-slim",
 "2.7.4"=>"2.7.4-jemalloc-bullseye-slim",
 "2.7.5"=>"2.7.5-jemalloc-bullseye-slim",
 "2.7.6"=>"2.7.6-jemalloc-bullseye-slim",
 "3.0.0"=>"3.0.0-jemalloc-buster-slim",
 "3.0.1"=>"3.0.1-jemalloc-buster-slim",
 "3.0.2"=>"3.0.2-jemalloc-bullseye-slim",
 "3.0.3"=>"3.0.3-jemalloc-bullseye-slim",
 "3.0.4"=>"3.0.4-jemalloc-bullseye-slim",
 "3.1.0"=>"3.1.0-jemalloc-bullseye-slim",
 "3.1.1"=>"3.1.1-jemalloc-bullseye-slim",
 "3.1.2"=>"3.1.2-jemalloc-bullseye-slim"}

I produced the above list with the following code:

require 'net/https'
require 'json'
tags = []

debian_releases = %w(stretch buster bullseye bookworm) 
         
Net::HTTP.start('quay.io', 443, use_ssl: true) do |http|
  (1..).each do |page|
    request = Net::HTTP::Get.new "/api/v1/repository/evl.ms/fullstaq-ruby/tag/?page=#{page}&limit=100"
    response = http.request request
    body = JSON.parse(response.body)
    tags += body['tags'].map {|tag| tag['name']}.grep /jemalloc-\w+-slim/
    break unless body['has_additional']
  end
end 
    
ruby_releases = tags.group_by {|tag| tag.split('-').first}.
  map do |release, tags|
    [release, tags.max_by {|tag| debian_releases.find_index(tag[/jemalloc-(\w+)-slim/, 1]) || -1}]
  end.sort.to_h
1 Like

TIL. thanks for pointing that Sam.

I think we can avoid the mapping by pointing to ${RUBY_VERSION}-jemalloc-slim and ignore the debian distro name and any custom mapping that needs future version to also match.

@rubys wdyt about Use -jemalloc-slim as ruby stack variant that always point to existen… by dangra · Pull Request #1487 · superfly/flyctl · GitHub ?

Thanks both. I changed the Docker file to

ARG RUBY_VERSION=3.0.1
ARG VARIANT=jemalloc-buster-slim

and that got me past the error. The deploy then progressed pretty far but again failed, but I think for an unrelated issue so I’ll ask that question separately. Thanks for your help. Daniel

1 Like

It looks like your Dockerfile is pointing at an image that doesn’t exist in the FROM directive.

Here’s the listing of available images: Quay

Try changing the lines at the top of your Dockerfile to:

ARG RUBY_VERSION=3.0.4
ARG VARIANT=jemalloc-slim
FROM quay.io/evl.ms/fullstaq-ruby:${RUBY_VERSION}-${VARIANT} as base

Then test your app to make sure it runs on Ruby 3.0.4 (it should)

Well, I’m posting here as this post seems similar. Or it did when I started debugging a deploy of a standard ruby on rails 7 app, using ruby 3.0.1

Here’s the notes: A first deploy of a ruby 3.0.1, rails 7 app to fly. ($2466137) · Snippets · Snippets · GitLab

However, I’m now getting this error:
v1 failed - Failed due to unhealthy allocations - no stable job version to auto revert to and deploying as v2

This looks related from the FAQ: Troubleshooting your Deployment · Fly Docs
But for a fix, should be looking at Fly’s Docker for the wrong port settings, or elsewhere? Rails production config port? Confused. Putting up a reference, so I have a point to come back to, but maybe on Monday I can get this deployed :slight_smile:

Send kittens haha.
( https://twitter.com/alterisian/status/1598668915420565504 )

Short answer: remove -bullseye from your Dockerfile, update flyctl, and try again.

Longer answer: In order to address a different problem, flyctl was updated to add -bullseye into the Dockerfile, but that caused other failures (like the one you are seeing) and was reverted.

The error message that you are seeing is basically saying that the deploy failed and that there is no previous successful deploy to revert to.

The suggestion to look at the port settings is a common reason for deploys to fail, it just doesn’t happen to be the reason your deploy failed.

At this point you should run flyctl version update, and then either rerun flyctl launch and let it overwrite the Dockerfile, or simply remove -bullseye yourself.

1 Like

Hi, I updated today…maybe 6 hours ago.

oceanician@River:~/code/ConnectingPeopleAndPlaces$ flyctl version
flyctl v0.0.437 linux/amd64 Commit: 7073cc94 BuildDate: 2022-12-02T01:00:29Z
oceanician@River:~/code/ConnectingPeopleAndPlaces$ flyctl version update
Error no available update

Just checked the Dockerfile, and there’s no mention of bullseye, sadly.

Puzzling. Your linked page has the following:

[internal] load metadata for quay.io/evl.ms/fullstaq-ruby:3.0.1-jemalloc-bullseye-slim

It clearly is getting -bullseye from somewhere. Here’s the change to flyctl which removed that 9 days ago:

I take this to mean that fly.io is getting ruby is coming from this site. So my attempt to update to 3.2.0 is premature since that version isn’t listed yet (2022.12.27).

I got here because I got the rpc error: code = Unknown desc = quay.io/evl.ms/fullstaq-ruby:3.2.0-jemalloc-bullseye-slim: not found

I tried with and without -bulleye. Whatever it means.

Correct. If you would like to get up and running now, try ruby:3.2.0-slim-bullseye. This won’t be as memory efficient as a version of ruby compiled with jemalloc, but you can get the same effect by adding a few lines to your Dockerfile: Preparations for Rails 7.1 - #13 by wjordan

I’m sure that a 3.2 version with jemalloc included will be posted shortly, and you can choose to change back at that time.

Thanks for responding, but not clear to me what field is what and how all this works.

ARG RUBY_VERSION=3.2.0
ARG VARIANT=jemalloc-slim
FROM Quay as base

Sorry for being so terse. The following will get you up and running:

ARG RUBY_VERSION=3.2.0
ARG VARIANT=slim
FROM ruby:${RUBY_VERSION}-${VARIANT} as base

Even longer answer. A Dockerfile starts with a base image and then proceeds to install your application and its dependencies. fly launch for Rails applications currently defaults to picking a base image from here: Quay, but you can also use base images from here: Docker. A description of the differences: https://fullstaqruby.org/

Specifying bullseye is optional. That’s the name of the Debian release, where Debian is the Linux Distribution contained in the base images linked above.