No such image error

rob-gordon · November 28, 2021, 5:37pm

I don’t want to hijack this thread if my error is unrelated, but when I try to deploy anything that’s large it takes a long time to push and then I get the following error:

8341556c45a5: Pushed
07dae7fd3cc3: Pushed
1c9ad7726ad3: Layer already exists
8d62e105cfcd: Layer already exists
a738bd8f9700: Layer already exists
e0de7f7f121a: Layer already exists
f939aee58e13: Layer already exists
ca44cfdf2cf5: Layer already exists
77b323d4ec74: Layer already exists
deployment-1638120244: digest: sha256:d844008c0eca7a07f266778076d98e65b9c265131e25cae8ba48a1c0500cdd81 size: 2214
--> Pushing image done
Error count not find built image: Error: No such image: sha256:e70ec2edf67e866ff0eb9c9b78c3f6adbe80b7fd8e8025a897967b1a9fe1aee4

I tried fly deploy --remote-only but that shows me:

==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
Error error connecting to docker: unhandled upstream service error: could not fulfill resource requirements: not enough available dedicated CPUs to fulfill request of 4 cpus

sudhir.j · November 29, 2021, 8:45am

@rob-gordon Do you still get the No such image error? The error you saw when trying to do --remote-only is a capacity problem that we can address, but the image error needs a bit more debugging.

jsierles · November 29, 2021, 6:28pm

Related: would you be able to post your Dockerfile and/or describe your app setup? It would be useful to find ways to optimize the Docker build to reduce the image size.

rob-gordon · November 29, 2021, 9:59pm

@sudhir.j I just tried fly deploy again and had the same error. Here is the full output:

❯ fly deploy
Deploying testing-the-hellofly-app
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8080
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.8 linux x86_64
Step 1/8 : FROM python:3.8-slim-buster
 ---> 5be55fb2aad1
Step 2/8 : ENV PORT 8080
 ---> Using cache
 ---> c26a14b108ba
Step 3/8 : RUN mkdir /app
 ---> Using cache
 ---> fef637d42462
Step 4/8 : RUN set -ex && 	apt-get update && 	apt-get install -y --no-install-recommends wget && 	wget -O /usr/bin/hivemind.gz https://github.com/DarthSim/hivemind/releases/download/v1.0.6/hivemind-v1.0.6-linux-amd64.gz &&     gzip -d /usr/bin/hivemind.gz &&     chmod +x /usr/bin/hivemind
 ---> Using cache
 ---> 491e3b746380
Step 5/8 : COPY . /app
 ---> 92a077e3cabc
Step 6/8 : WORKDIR /app
 ---> Running in 188a7956de0a
 ---> 5450a2f12b70
Step 7/8 : RUN pip install -r requirements.txt
 ---> Running in be9497cbdf24
Collecting autopep8==1.6.0
  Downloading autopep8-1.6.0-py2.py3-none-any.whl (45 kB)
Collecting click==7.1.2
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting Flask==1.1.2
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting gunicorn==20.0.4
  Downloading gunicorn-20.0.4-py2.py3-none-any.whl (77 kB)
Collecting itsdangerous==1.1.0
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2==2.11.2
  Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting joblib==1.1.0
  Downloading joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting MarkupSafe==1.1.1
  Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux2010_x86_64.whl (32 kB)
Collecting numpy==1.21.4
  Downloading numpy-1.21.4-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting pandas==1.3.4
  Downloading pandas-1.3.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
Collecting pycodestyle==2.8.0
  Downloading pycodestyle-2.8.0-py2.py3-none-any.whl (42 kB)
Collecting python-dateutil==2.8.2
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz==2021.3
  Downloading pytz-2021.3-py2.py3-none-any.whl (503 kB)
Collecting scikit-learn==1.0.1
  Downloading scikit_learn-1.0.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (25.9 MB)
Collecting scipy==1.7.3
  Downloading scipy-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.3 MB)
Collecting six==1.16.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting sklearn==0.0
  Downloading sklearn-0.0.tar.gz (1.1 kB)
Collecting threadpoolctl==3.0.0
  Downloading threadpoolctl-3.0.0-py3-none-any.whl (14 kB)
Collecting toml==0.10.2
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting Werkzeug==1.0.1
  Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Requirement already satisfied: setuptools>=3.0 in /usr/local/lib/python3.8/site-packages (from gunicorn==20.0.4->-r requirements.txt (line 4)) (57.5.0)
Building wheels for collected packages: sklearn
  Building wheel for sklearn (setup.py): started
  Building wheel for sklearn (setup.py): finished with status 'done'
  Created wheel for sklearn: filename=sklearn-0.0-py2.py3-none-any.whl size=1309 sha256=60b9c8582137ade4733791d170153326d913fbbba215332cd573d10910040df5
  Stored in directory: /root/.cache/pip/wheels/22/0b/40/fd3f795caaa1fb4c6cb738bc1f56100be1e57da95849bfc897
Successfully built sklearn
Installing collected packages: numpy, threadpoolctl, six, scipy, MarkupSafe, joblib, Werkzeug, toml, scikit-learn, pytz, python-dateutil, pycodestyle, Jinja2, itsdangerous, click, sklearn, pandas, gunicorn, Flask, autopep8
Successfully installed Flask-1.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 Werkzeug-1.0.1 autopep8-1.6.0 click-7.1.2 gunicorn-20.0.4 itsdangerous-1.1.0 joblib-1.1.0 numpy-1.21.4 pandas-1.3.4 pycodestyle-2.8.0 python-dateutil-2.8.2 pytz-2021.3 scikit-learn-1.0.1 scipy-1.7.3 six-1.16.0 sklearn-0.0 threadpoolctl-3.0.0 toml-0.10.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
 ---> 3dd9ed83d045
Step 8/8 : CMD ["/usr/bin/hivemind", "/app/Procfile"]
 ---> Running in efdb648ef4c3
 ---> e0e0a3def96c
Successfully built e0e0a3def96c
Successfully tagged registry.fly.io/testing-the-hellofly-app:deployment-1638222307
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/testing-the-hellofly-app]
9134d63623d9: Pushed
c6f807c14eb8: Pushed
1c9ad7726ad3: Layer already exists
8d62e105cfcd: Layer already exists
a738bd8f9700: Layer already exists
e0de7f7f121a: Layer already exists
f939aee58e13: Layer already exists
ca44cfdf2cf5: Layer already exists
77b323d4ec74: Layer already exists
deployment-1638222307: digest: sha256:b2d00859573bb0cbb0372b60a5777389f0a8989001cf75011040b8dd9f8c952d size: 2214
--> Pushing image done
Error count not find built image: Error: No such image: sha256:e0e0a3def96c14ab0f4cee6759e9f4ff2896387c5182458a28a29c5504b4f5fe

@jsierles The repo takes the hellofly python example and adds scikit learn in order to serve a small model for prediction. Scikit learn is pretty big, which makes the image size large. Just to be sure it wasn’t on my end I tried deploying the same code on Heroku and it worked without any changes. Not sure what I may be doing wrong. GitHub - rob-gordon/hellofly-python-flask

sudhir.j · November 30, 2021, 9:27am

@rob-gordon I’ve moved this error out of the old topic, since it doesn’t seem to be related to image size or registry reliability.

kurt · November 30, 2021, 11:35pm

Will you try building and pushing the image manually to see if you have the same issue? You can do something like this:

docker build . -t registry.fly.io/testing-the-hellofly-app:testing
flyctl docker auth
docker push registry.fly.io/testing-the-hellofly-app:testing
flyctl deploy -i registry.fly.io/testing-the-hellofly-app:testing

rob-gordon · December 2, 2021, 3:15pm

@kurt Gave this a try today but got hung up with the following-

❯ flyctl docker auth
Error: unknown command "docker" for "flyctl"
Run 'flyctl --help' for usage.
Error unknown command "docker" for "flyctl"

(for reference)

❯ flyctl version
flyctl v0.0.256 darwin/amd64 Commit: f39ea62 BuildDate: 2021-11-21T19:59:38Z

kurt · December 2, 2021, 3:20pm

Doh, I meant fly auth docker.

rob-gordon · December 2, 2021, 5:48pm

Had no problems pushing that way

I couldn’t get it to deploy, but I’m sure it’s because my fly.toml is misconfigured. The server runs on port 5000 but the fly health checks don’t seem to find it. Do you know what the problem might be?

Dockerfile

FROM python:3.8-slim-buster
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
ENV FLASK_APP hellofly
EXPOSE 5000
EXPOSE 80
CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0"]

fly.toml

# fly.toml file generated for testing-the-hellofly-app on 2021-11-28T11:49:38-05:00

app = "testing-the-hellofly-app"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]

[build]
  builtin = "python"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["http"]
    port = 5000

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

Deploy

2021-12-02T17:27:38.000 [info] 2021/12/02 17:27:38 listening on [fdaa:0:2129:a7b:aa4:20ba:74f1:2]:22 (DNS: [fdaa::3]:53)
2021-12-02T17:27:38.000 [info]  * Serving Flask app "hellofly"
2021-12-02T17:27:38.000 [info]  * Environment: production
2021-12-02T17:27:38.000 [info]    WARNING: This is a development server. Do not use it in a production deployment.
2021-12-02T17:27:38.000 [info]  * Debug mode: off↩︎ WSGI server instead.
2021-12-02T17:27:38.000 [info]  * Debug mode: off
2021-12-02T17:27:38.000 [info] OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
2021-12-02T17:27:39.000 [info]  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
2021-12-02T17:28:00.000 [warn] Health check status changed 'passing' => 'warning'
2021-12-02T17:28:09.000 [error] Health check status changed 'warning' => 'critical'

michael · December 2, 2021, 6:04pm

I noticed two issues with your config

services.internal_port should be 5000 to match your app.
you don’t need the services.ports block for 5000 unless you want to expose that port externally, such as yourapp.com:5000

rob-gordon · December 2, 2021, 6:46pm

That did the trick! Thanks @michael

Topic		Replies	Views
Deploy –remote-only failing and kept retrying Questions / Help	5	318	September 7, 2023
Error You hit a Fly API error with request ID: 01GFN2V0NT9MZW6CNQWSXDAXDW-cdg	5	283	October 18, 2022
Error An unknown error occured.During deploy.	13	1262	May 5, 2022
Cannot deploy application	11	981	November 4, 2023
Can't deploy the app Build debugging help-me-help-you	1	390	August 8, 2023

No such image error

Dockerfile

fly.toml

Deploy

Related Topics