Deploying Umami

[luap]

Hi folks!

I’m trying to deploy umami on fly.io. Currently running into some issues. As of last weekend, I was able to put up a postgres instance, proxy to my localhost and run the sql script with no issues.
The issues that I’m running into are building/deploying the web app console, and then logging into said web app console. With a docker build, the web app is unable to get into a healthy state. It’s stuck in a critical state. I am specifying the build argument --build-arg DATABASE_TYPE=postgresql as is required for the docker build and then also changing the DATABASE_URL inside the Dockerfile to the output after attaching the postgres instance to the app.
I also tried to build from source (I removed the Dockerfile, and added a .env file) and I did get a bit closer with the web app at least starting. However I get an internal 500 error code when trying to log into the web app with the default credentials.

I’d be happy to provide more info if needed. Could anyone try and get this running too? Thanks in advance!

Documentation for umami here

[rugwiro]

Hey @luap, umani seems like an interesting project that perhaps the community would love so I went ahead and tried to deploy it myself. I made a few modifications to the Dockerfileand added a migration.sh script to make the experience a bit more pleasant but everything afterward was smooth.

1. After making a local copy for the repository make these changes to the Dockerfile

# Build image
FROM node:12.22-alpine AS build
ARG BASE_PATH
ARG DATABASE_TYPE

ENV BASE_PATH=$BASE_PATH
ENV DATABASE_URL "postgresql://umami:umami@db:5432/umami"
ENV DATABASE_TYPE=$DATABASE_TYPE

WORKDIR /build

RUN yarn config set --home enableTelemetry 0
COPY package.json yarn.lock /build/

# Install only the production dependencies
RUN yarn install --production --frozen-lockfile

# Cache these modules for production
RUN cp -R node_modules/ prod_node_modules/

# Install development dependencies
RUN yarn install --frozen-lockfile

COPY . /build
RUN yarn next telemetry disable
RUN yarn build

# Production image
FROM node:12.22-alpine AS production

+ RUN apk --update add postgresql-client

WORKDIR /app

# Copy cached dependencies
COPY --from=build /build/prod_node_modules ./node_modules

+ # COPY sql migration to make them available to the release command
+ COPY --from=build /build/sql ./sql
+ COPY --from=build /build/migrate.sh ./migrate.sh
+ RUN chmod +x ./migrate.sh

# Copy generated Prisma client
COPY --from=build /build/node_modules/.prisma/ ./node_modules/.prisma/

COPY --from=build /build/yarn.lock /build/package.json ./
COPY --from=build /build/.next ./.next
COPY --from=build /build/public ./public

USER node

EXPOSE 3000
CMD ["yarn", "start"]

2. Run fly launch --name <app-name> --org personal to create an app and generate a fly.toml file. This command will prompt you for a few things:

• Creating a database that you can skip if already have one and instead run fly pg attach later. If you do choose to create one it’s name will be <app-name>-db and flyctl will create a DATABASE_URL variable for you.
• The next prompt will ask you whether you want to deploy now. To which you should say no since we have to make a few modifications to the fly.toml before.

3. Set the HASH_SALT secret: fly secrets set HASH_SALT=$SECRET

4. Add a release command to fly.toml.

# fly.toml file generated for umani-test on 2022-05-12T11:02:12+02:00

app = "umani-test"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

+ [env]
+ DATABASE_TYPE = "PostgreSQL"

+ [deploy]

+ release_command = "./migrate.sh"

[experimental]
allowed_public_ports = []
auto_rollback = true

[[services]]
 http_checks = []
- internal_port = 8080
+ internal_port = 3000
processes = ["app"]
protocol = "tcp"
script_checks = []

[services.concurrency]
hard_limit = 25
soft_limit = 20
type = "connections"

[[services.ports]]
force_https = true
handlers = ["http"]
port = 80

[[services.ports]]
handlers = ["tls", "http"]
port = 443

[[services.tcp_checks]]
grace_period = "1s"
interval = "15s"
restart_limit = 0
timeout = "2s"

The content of migrate.sh are of course:

#!/bin/sh

set -e

psql $DATABASE_URL -f sql/schema.postgresql.sql

You should also notice the DATABASE_TYPE set as an environment variable and the internal_port change to match umami.

5. Now you can run fly deploy

[luap]

Hey, thanks for the fast response!

I was able to deploy both the console and database using the above instructions. I generated code from the dashboard and added the javascript snippet to my localhost site and it is receiving traffic. However, I’m getting a series of 500 errors going through data that is hitting umami. The error is:

2022-05-13T03:37:24Z app[7071e349] sjc [info]Error: Unknown database.
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at rawQuery (/app/.next/server/chunks/9718.js:701:31)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at getActiveVisitors (/app/.next/server/chunks/9718.js:1017:12)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at __WEBPACK_DEFAULT_EXPORT__ (/app/.next/server/pages/api/website/[id]/active.js:132:102)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at async Object.apiResolver (/app/node_modules/next/dist/server/api-utils/node.js:182:9)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at async NextNodeServer.runApi (/app/node_modules/next/dist/server/next-server.js:386:9)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at async Object.fn (/app/node_modules/next/dist/server/base-server.js:488:37)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at async Router.execute (/app/node_modules/next/dist/server/router.js:228:32)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at async NextNodeServer.run (/app/node_modules/next/dist/server/base-server.js:600:29)
2022-05-13T03:37:24Z app[7071e349] sjc [info]    at async NextNodeServer.handleRequest (/app/node_modules/next/dist/server/base-server.js:307:20)

Maybe we need to make a correction to the database URL somehow? This might be out of scope of fly.io. I’ll do some more searching on my end.

[luap]

I ended up figuring out the issue. There needs to be a change to the fly.toml file. It should be DATABASE_TYPE = "postgresql" rather than DATABASE_TYPE = "PostgreSQL" . There’s code here that checks the database type that’s case sensitive here: umami/queries.js at master · mikecao/umami · GitHub and it’s defined here: umami/constants.js at master · mikecao/umami · GitHub

I think you should update your reply so people don’t run into this issue into the future @rugwiro .

Also on another note for anyone reading this in the future, if you’re looking to make non-atomic changes to the database, I’d highly suggest removing the release_command = "./migrate.sh" from the fly.toml file after the initial deployment so your data doesn’t get blown away. There’s definitely a more elegant solution to this but I’ll leave it as a suggestion for now.

[keybits]

Recent updates have made installing and updating Umami on Fly easier.

Here are the steps:

1. The only change needed to the Dockerfile is to add RUN apk --update add postgresql-client:

# Production image, copy all the files and run next
FROM node:16-alpine AS runner
+ RUN apk --update add postgresql-client

2. Run fly launch and create a PostgreSQL database cluster when prompted (if you don’t already have one). Say ‘No’ to ‘Deploy now’ as we need to edit fly.toml
3. Set the HASH_SALT secret: fly secrets set HASH_SALT=$YOUR-SECRET-STRING
4. Edit fly.toml:

app = "umani-test"

kill_signal = "SIGINT"
kill_timeout = 5
processes = []

+ [env]
+ DATABASE_TYPE = "postgresql"

+ [deploy]
+ release_command = "yarn check-db"

[experimental]
allowed_public_ports = []
auto_rollback = true

[[services]]
 http_checks = []
- internal_port = 8080
+ internal_port = 3000
processes = ["app"]

5. Run fly deploy

To update your application you can merge changes from upstream (make sure to preserve the changes you made to Dockerfile) and then run fly deploy again. The new check-db.js script runs migrations for you

[paulrudy]

[Edited, found a possible solution]

Thanks for this!

With

[env]
  DATABASE_TYPE = "postgresql"

I ran into this error on fly deploy:

=> ERROR [builder 5/5] RUN yarn build-docker                                    1.6s
------
 > [builder 5/5] RUN yarn build-docker:
#17 0.454 yarn run v1.22.19
#17 0.515 $ npm-run-all build-db build-tracker build-geo build-app
#17 0.944 $ npm-run-all copy-db-files build-db-client
#17 1.376 $ node scripts/copy-db-files.js
#17 1.525 /app/scripts/copy-db-files.js:19
#17 1.525   throw new Error('Missing or invalid database');
#17 1.525   ^
#17 1.525
#17 1.525 Error: Missing or invalid database
#17 1.525     at Object.<anonymous> (/app/scripts/copy-db-files.js:19:9)
#17 1.525     at Module._compile (node:internal/modules/cjs/loader:1155:14)
#17 1.525     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1209:10)
#17 1.525     at Module.load (node:internal/modules/cjs/loader:1033:32)
#17 1.525     at Function.Module._load (node:internal/modules/cjs/loader:868:12)
#17 1.525     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
#17 1.525     at node:internal/main/run_main_module:22:47
#17 1.540 error Command failed with exit code 1.
#17 1.540 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
#17 1.549 ERROR: "copy-db-files" exited with 1.
#17 1.565 error Command failed with exit code 1.
#17 1.565 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
#17 1.574 ERROR: "build-db" exited with 1.
#17 1.590 error Command failed with exit code 1.
#17 1.590 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
------
Error failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c yarn build-docker]: exit code: 1

But everything seemed to work once I changed it to be part of the build block:

[build]
DATABASE_TYPE = "postgresql"

Don’t know why that would be, but sharing what got things working for me. Thank you for the tutorial!

[paulrudy]

Update to my previous comment: DATABASE_TYPE = "postgresql" needs to be passed under [build.args] not [build] and not [env]:

[build.args]
DATABASE_TYPE = "postgresql"

[paulrudy]

Umami recently updated to v2, which requires some migration steps including running a database migration script.

Starting from my running v1 instance, I first tried to run the script from within the instance using fly ssh console and then:

cd app
npx @umami/migrate-v1-v2@latest

But this process kept getting immediately killed. The Fly “monitoring” interface showed I was running out of memory. fly scale memory 2048 then repeating the above allowed me to successfully run the migration script. Then I downscaled memory back to its previous amount (Note: Umami v2 no longer seems happy with the paltry 256mb memory I had been using. Database checks that it runs on startup run out of memory now, but everything seems okay at 512)

[paulrudy]

@keybits Have you had issues with Umami v2.6.0? I’m running into Cannot find module errors on build.

Started an issue on Umami github

[keybits]

I haven’t been able to check this as I’m now running Umami on PikaPods.

[paulrudy]

Thanks. I’ll open discussion here and see if I can get any help.

If you don’t mind sharing, what have been the advantages for you of PikaPods?

[keybits]

I found Umami was best with 512MB of memory. PikaPods is cheaper than Fly in that scenario. Also they look after upgrades. Of course PikaPods only works if they support an app you want.

[paulrudy]

I was able to get it running by pulling Umami’s pre-built container, and found this method much simpler. I posted a guide.