Gitlab runner fails to get postgres DATABASE_URL

[sujith]

Hi,

I am using Gitlab runner to deploy to Fly.io. I am on the hobby plan. The runner is unable to get DATABASE_URL for Postgres database. Please let me know what I am doing wrong.

Running with gitlab-runner 16.3.0~beta.108.g2b6048b4 (2b6048b4)
  on green-2.saas-linux-small-amd64.runners-manager.gitlab.com/default ns46NMmJ, system ID: s_85d7af184313
  feature flags: FF_USE_IMPROVED_URL_MASKING:true, FF_RESOLVE_FULL_TLS_CHAIN:false
Preparing the "docker+machine" executor
Using Docker executor with image node:lts ...
Pulling docker image node:lts ...
Using docker image sha256:cf58080ad70c11059df1bcd3eafe1bb93c49403d434d4a4649e0f7dc4f562f5a for node:lts with digest node@sha256:cd7fa8f136023f7500490e410ba70dd3982ccca21805264f2a260a3a97be7376 ...
Preparing environment
Running on runner-ns46nmmj-project-51229892-concurrent-0 via runner-ns46nmmj-s-l-s-amd64-1697291839-f3d31c75...
Getting source from Git repository 00:01
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/skylful/skylwebapi/.git/
Created fresh repository.
Checking out e9db98aa as detached HEAD (ref is main)...
Skipping Git submodules setup
$ git remote set-url origin "${CI_REPOSITORY_URL}"
Executing "step_script" stage of the job script
Using docker image sha256:cf58080ad70c11059df1bcd3eafe1bb93c49403d434d4a4649e0f7dc4f562f5a for node:lts with digest node@sha256:cd7fa8f136023f7500490e410ba70dd3982ccca21805264f2a260a3a97be7376 ...
$ apt-get update -qq && apt-get install -y curl
Reading package lists...
Building dependency tree...
Reading state information...
curl is already the newest version (7.88.1-10+deb12u4).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ curl -L https://fly.io/install.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1735    0  1735    0     0   9132      0 --:--:-- --:--:-- --:--:--  9179
######################################################################## 100.0%
set update channel to shell
flyctl was installed successfully to /root/.fly/bin/flyctl
Manually add the directory to your $HOME/.bash_profile (or similar)
  export FLYCTL_INSTALL="/root/.fly"
  export PATH="$FLYCTL_INSTALL/bin:$PATH"
Run '/root/.fly/bin/flyctl --help' to get started
$ /root/.fly/bin/flyctl deploy
==> Verifying app config
Validating /builds/skylful/skylwebapi/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Waiting for remote builder fly-builder-shy-frost-74...
Remote builder fly-builder-shy-frost-74 ready
Reading package lists...
Building dependency tree...
Reading state information...
curl is already the newest version (7.88.1-10+deb12u4).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
$ curl -L https://fly.io/install.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1735    0  1735    0     0   9132      0 --:--:-- --:--:-- --:--:--  9179
######################################################################## 100.0%
set update channel to shell
flyctl was installed successfully to /root/.fly/bin/flyctl
Manually add the directory to your $HOME/.bash_profile (or similar)
  export FLYCTL_INSTALL="/root/.fly"
  export PATH="$FLYCTL_INSTALL/bin:$PATH"
Run '/root/.fly/bin/flyctl --help' to get started
$ /root/.fly/bin/flyctl deploy
==> Verifying app config
Validating /builds/skylful/skylwebapi/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Waiting for remote builder fly-builder-shy-frost-74...
Remote builder fly-builder-shy-frost-74 ready
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 ...
#2 [internal] load .dockerignore
#2 DONE 0.0s
#2 [internal] load .dockerignore
#2 transferring context:
#2 ...
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 697B 1.0s done
#1 DONE 1.0s
#2 [internal] load .dockerignore
#2 transferring context: 1.44kB 1.2s done
#2 DONE 1.2s
#3 [internal] load metadata for docker.io/library/node:18-alpine
#3 DONE 2.0s
#4 [1/8] FROM docker.io/library/node:18-alpine@sha256:617b85a014faf7aabe0661885a245cf3022e39675661d5795ab6748bb2f20f0f
#4 DONE 0.0s
#5 [internal] load build context
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 ...
#2 [internal] load .dockerignore
#2 DONE 0.0s
#2 [internal] load .dockerignore
#2 transferring context:
#2 ...
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 697B 1.0s done
#1 DONE 1.0s
#2 [internal] load .dockerignore
#2 transferring context: 1.44kB 1.2s done
#2 DONE 1.2s
#3 [internal] load metadata for docker.io/library/node:18-alpine
#3 DONE 2.0s
#4 [1/8] FROM docker.io/library/node:18-alpine@sha256:617b85a014faf7aabe0661885a245cf3022e39675661d5795ab6748bb2f20f0f
#4 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 89.47kB 4.6s
#5 transferring context: 310.06kB 9.8s
#5 transferring context: 432.35kB 14.9s
#5 transferring context: 699.28kB 20.1s
#5 transferring context: 877.48kB 25.9s
#5 transferring context: 1.03MB 32.0s
#5 transferring context: 1.18MB 37.5s
#5 transferring context: 1.31MB 43.5s
#5 transferring context: 1.49MB 48.9s
#5 transferring context: 1.69MB 54.9s
#5 transferring context: 1.85MB 60.8s
#5 transferring context: 2.01MB 66.3s
#5 transferring context: 2.01MB 66.5s done
#5 DONE 66.5s
#6 [2/8] RUN apk update && apk add build-base gcc autoconf automake zlib-dev libpng-dev nasm bash vips-dev
#6 CACHED
#7 [3/8] WORKDIR /opt/
#7 CACHED
#8 [4/8] COPY ./package.json ./package-lock.json ./
#8 CACHED
#9 [5/8] RUN npm install
#9 CACHED
#10 [6/8] WORKDIR /opt/app
#10 CACHED
#11 [7/8] COPY ./ .
#11 DONE 0.0s
#12 [8/8] RUN npm run build
#5 transferring context: 2.01MB 66.3s
#5 transferring context: 2.01MB 66.5s done
#5 DONE 66.5s
#6 [2/8] RUN apk update && apk add build-base gcc autoconf automake zlib-dev libpng-dev nasm bash vips-dev
#6 CACHED
#7 [3/8] WORKDIR /opt/
#7 CACHED
#8 [4/8] COPY ./package.json ./package-lock.json ./
#8 CACHED
#9 [5/8] RUN npm install
#9 CACHED
#10 [6/8] WORKDIR /opt/app
#10 CACHED
#11 [7/8] COPY ./ .
#11 DONE 0.0s
#12 [8/8] RUN npm run build
#12 0.903 
#12 0.903 > skylwebapi@0.1.0 build
#12 0.903 > strapi build
#12 0.903 
#12 3.766 Error: Could not load js config file /opt/app/config/env/production/database.js: Cannot read properties of undefined (reading 'charAt')
#12 3.766     at loadJsFile (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/load-config-file.js:21:19)
#12 3.766     at loadFile (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/load-config-file.js:41:20)
#12 3.766     at /opt/node_modules/@strapi/strapi/dist/core/app-configuration/config-loader.js:18:52
#12 3.766     at Array.reduce (<anonymous>)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/config-loader.js:16:10)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/index.js:51:51)
#12 3.766     at new Strapi (/opt/node_modules/@strapi/strapi/dist/Strapi.js:162:59)
#12 3.766     at initFn (/opt/node_modules/@strapi/strapi/dist/Strapi.js:538:20)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/commands/builders/admin.js:14:48)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/commands/actions/build-command/action.js:13:37)
#12 ERROR: executor failed running [/bin/sh -c npm run build]: exit code: 1
------
 > [8/8] RUN npm run build:
#12 3.766     at loadJsFile (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/load-config-file.js:21:19)
#12 3.766     at loadFile (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/load-config-file.js:41:20)
#12 3.766     at /opt/node_modules/@strapi/strapi/dist/core/app-configuration/config-loader.js:18:52
#12 3.766     at Array.reduce (<anonymous>)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/config-loader.js:16:10)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/core/app-configuration/index.js:51:51)
#12 3.766     at new Strapi (/opt/node_modules/@strapi/strapi/dist/Strapi.js:162:59)
#12 3.766     at initFn (/opt/node_modules/@strapi/strapi/dist/Strapi.js:538:20)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/commands/builders/admin.js:14:48)
#12 3.766     at exports.default (/opt/node_modules/@strapi/strapi/dist/commands/actions/build-command/action.js:13:37)
------
Error: failed to fetch an image or build from source: error building: failed to solve: executor failed running [/bin/sh -c npm run build]: exit code: 1
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1

[greg]

Hi,

Ah. The secrets Fly provides are only available to your app at run-time. For example process.env.SECRET (in the case of Node). Not at build time, which looks to be the issue you are having.

I’d recommend scrolling down this thread. It seems to be very similar, where they need the DATABASE_URL during the build part:

Build secret in Dockerfile not found

Hi, I’ve included the following code in my dockerfile to set the the DATABASE_URL env variable in a NextJS project, I am using the echo command to log the DATABASE_URL_SECRET variable first to check that it has been correctly added: # Set DATABASE_URL_SECRET RUN --mount=type=secret,id=DATABASE_URL_SECRET \ DATABASE_URL_SECRET="$(cat /run/secrets/DATABASE_URL_SECRET)" \ && echo "DATABASE_URL_SECRET: $DATABASE_URL_SECRET" # Set DATABASE_URL from DATABASE_URL_SECRET ENV DATABASE_URL=$DATABASE…

For a bit of background as to why that’s needed:

Build Secrets

[sujith]

I managed to make it work.

Create dummy values in local .env file for the variables. Uploaded the .env file to Gitlab runner by commenting the entry in .gitignore. Added relevant secrets in Fly.