Deploy timeout on gitlab ci

1

I get the error below when deploying from GitLab ci, but if I deploy from local first and then deploy again from GitLab ci it succeeds with only 2 minutes.

in the GitLab ci i use image: node:18-alpine3.15

I’m confused about how to debug the issue

What actually happens?

fly deploy --remote-only
==> Verifying app config
--> Verified app config
==> Building image
Waiting for remote builder fly-builder-red-haze-6583...
Remote builder fly-builder-red-haze-6583 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
#1 [internal] load remote build context
#1 DONE 0.0s
#2 copy /context /
#2 DONE 0.1s
#3 [internal] load metadata for docker.io/library/node:16-bullseye-slim
#3 DONE 0.2s
#4 [base 1/2] FROM docker.io/library/node:16-bullseye-slim@sha256:xxxxxxx
#4 resolve docker.io/library/node:16-bullseye-slim@sha256:xxxxxxxxxx done
#4 DONE 0.0s
#5 [base 2/2] RUN apt-get update && apt-get install -y openssl
#5 CACHED
#8 [deps 1/4] RUN mkdir /app
#8 CACHED
#9 [deps 2/4] WORKDIR /app
#9 CACHED
#10 [deps 3/4] ADD package.json yarn.lock ./
#10 DONE 0.0s
#11 [deps 4/4] RUN yarn install --production=false
#11 0.522 yarn install v1.22.19
#11 0.606 [1/4] Resolving packages...
#11 1.064 [2/4] Fetching packages...
ERROR: Job failed: execution took longer than 1h0m0s seconds
2

Hi there,

Are the logs you show output from the flyctl app inside the Gitlab CI runner? And does exactly the same command work with exact the same files work locally?

Maybe you are able to share a bit of your .gitlab-ci.yml and fly configuration.

Thanks

3
  • log above from GitLab ci log.
  • yes exactly the same with local files and command

this my fly.toml

# fly.toml file generated for my-web-name on 2022-06-14T11:30:37+07:00

app = "my-web-name"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]
  PORT = "8080"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

this is the job on GitLab ci that I execute for deploy
.gitlab-ci.yml

deploy_dev:
  stage: deploy_dev
  extends: .dependencies_cache
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: on_success
  before_script:
    - apk add curl
    - curl -L https://fly.io/install.sh | sh
    - export FLYCTL_INSTALL="/root/.fly"
    - export PATH="$FLYCTL_INSTALL/bin:$PATH"
  script:
    - fly deploy --remote-only
4

Hmm, since your ci logs show that the process times out, is this the timeout from the CI or from fly? I think it is the ci timeout since they are 1h by default.

How long does the same command locally take to execute? Close to 1h? If not, maybe it is possible to enable more verbose logging, to see why yarn takes so long to fetch the packages.

5

in local just execute 2 minutes.

6

Only thing I can think of to debug this further is to run both flyctl and the docker commands with debugging enabled. If you do this locally and on the CI, you might be able to find the difference and what causes this.