run keycloak with fly

as fly works so well with my custom apps, i try to move keycloak also to fly as there is even a postgres cluster available.
So i have created an organisation where postgres runs in and try to launch keycloak based on the official docker description: Running Keycloak in a container - Keycloak

without success for now!

I think that my main problems are in missing or wrong network, hostname and certificate settings.

Was anyone able to setup keycloak 18 through docker on a fly machine?

Hi! I’ve also stuck here. Did you have a success deploying keycloak to fly.io?

I was able to run, but had to use a external (another provider) Postgres database because I could not fix a problem with IPv6 (I think, I’m not sure, it’s been a while).

Dockerfile:

FROM quay.io/keycloak/keycloak:18.0.2 as builder

ENV KC_HEALTH_ENABLED=true
ENV KC_METRICS_ENABLED=true
ENV KC_DB=postgres
RUN curl -sL https://github.com/aerogear/keycloak-metrics-spi/releases/download/2.5.3/keycloak-metrics-spi-2.5.3.jar -o /opt/keycloak/providers/keycloak-metrics-spi-2.5.3.jar
RUN /opt/keycloak/bin/kc.sh build

FROM quay.io/keycloak/keycloak:18.0.2
COPY --from=builder /opt/keycloak/ /opt/keycloak/
WORKDIR /opt/keycloak
RUN keytool -genkeypair -storepass password -storetype PKCS12 -keyalg RSA -keysize 2048 -dname "CN=server" -alias server -ext "SAN:c=DNS:localhost,IP:127.0.0.1" -keystore conf/server.keystore
ENV KC_HTTP_ENABLED=true
ENV KC_HOSTNAME_STRICT=false
ENTRYPOINT ["/opt/keycloak/bin/kc.sh", "start", "--proxy", "edge"]

Fly.toml

app = "app-name"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[metrics]
  path = "/metrics"
  port = 8080

[env]
    KC_DB_URL = "jdbc:postgresql://USER:PASSWORD@host/database"
    KC_DB_USERNAME = "keycloak"
    KEYCLOAK_ADMIN = "admin"
    KEYCLOAK_ADMIN_PASSWORD = "adminpassword"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [[services.http_checks]]
    grace_period = "5s"
    interval = 10000
    method = "get"
    path = "/health"
    protocol = "http"
    restart_limit = 0
    timeout = 2000
    tls_skip_verify = false
    [services.http_checks.headers]

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

I had a secret KC_DB_PASSWORD as well.

You should store the sensitive env vars as secrets and maybe use a custom keystore mounted on a volume.

3 Likes

Thank you!
In the KC_DB_URL (env in toml file) USER:PASSWORD will be substituted from KC_DB_USERNAME and KC_DB_PASSWORD or it has to be set explicitly?

Alright, I tested again and was able to use a Postgres instance on Fly:

Dockerfile

FROM quay.io/keycloak/keycloak:18.0.2 as builder

ENV KC_HEALTH_ENABLED=true
ENV KC_METRICS_ENABLED=true
ENV KC_DB=postgres
RUN curl -sL https://github.com/aerogear/keycloak-metrics-spi/releases/download/2.5.3/keycloak-metrics-spi-2.5.3.jar -o /opt/keycloak/providers/keycloak-metrics-spi-2.5.3.jar
RUN /opt/keycloak/bin/kc.sh build

FROM quay.io/keycloak/keycloak:18.0.2
COPY --from=builder /opt/keycloak/ /opt/keycloak/
WORKDIR /opt/keycloak
RUN keytool -genkeypair -storepass password -storetype PKCS12 -keyalg RSA -keysize 2048 -dname "CN=server" -alias server -ext "SAN:c=DNS:localhost,IP:127.0.0.1" -keystore conf/server.keystore
ENV KC_HTTP_ENABLED=true
ENV KC_HOSTNAME_STRICT=false
ENV JAVA_OPTS_APPEND="-Djava.net.preferIPv4Stack=false"
ENTRYPOINT ["/opt/keycloak/bin/kc.sh", "start", "--proxy", "edge"]

By default Keycloak uses java.net.preferIPv4Stack=true that causes problems with Fly internal network, so we JAVA_OPTS_APPEND env var to change it.

The error Keycloak returned was:
java.net.UnknownHostException: DATABASE_HOST.internal

Now, about the database env vars, I was wrong on my last comment, here is a updated example:

Fly.toml


app = "test-keycloak"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[metrics]
  path = "/metrics"
  port = 8080

[env]
  KC_DB_URL = "jdbc:postgresql://DATABASE_HOST/DATABASE"
  KC_DB_USERNAME = "DB_USER"
  KC_DB_PASSWORD = "DB_PASSWORD"
  KEYCLOAK_ADMIN = "admin"
  KEYCLOAK_ADMIN_PASSWORD = "adminpassword"

[experimental]
  allowed_public_ports = []
  auto_rollback = true

[[services]]
  internal_port = 8080
  processes = ["app"]
  protocol = "tcp"
  script_checks = []

  [[services.http_checks]]
    grace_period = "5s"
    interval = 10000
    method = "get"
    path = "/health"
    protocol = "http"
    restart_limit = 0
    timeout = 2000
    tls_skip_verify = false
    [services.http_checks.headers]

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20
    type = "connections"

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

Please note that on the KC_DB_URL env var we only set the host and database, no user or password, and please, store the sensitive env vars as secrets, this is just an example.


I tried to update the docker image as well (to quay.io/keycloak/keycloak:20.0.2), but got errors related to the database URL format, I don’t have time to debug it right now, if you find the solution, please share with us.

1 Like

Out of personal curiosity, what are the errors?

Something like:

Datasource '<default>': URL format error; must be "jdbc:h2:{ {.|mem:}[name] | [file:]fileName | {tcp|ssl}:[//]server[:port][,server2[:port]]/name }[;key=value...]" but is jdbc:postgresql://DATABASE_HOST/DATABASE" 

Maybe using KC_DB_URL_HOST and KC_DB_URL_DATABASE instead of the KC_DB_URL can make things work.

I don’t really use Keycloak to be honest, I only tried while learning about it, so if you anyone has a better way of doing or maybe can solve this issue, please share with the community.

Thank you so much, I’ve managed to connect Keycloak 18 finally with postgres both on Fly!
Works with min 512 MB of allocated memory.

If someone will find a way to upgrade to Keycloak 20, please share your solution :pray:

1 Like

Solved URL format error by adding this to the second block of your Dockerfile too

ENV KC_DB=postgres

Also tried to change this row to the next

RUN /opt/keycloak/bin/kc.sh build --db=postgres

Getting this error though and a lot of other

Failed to obtain JDBC connection

Also getting these errors even if removing ENVs. Like it’s cached and no way to get rid of them

Error: Could not find or load main class java.net.preferIPv4Stack=false
Error: Could not find or load main class java.net.preferIPv6Addresses=true

Though I had that Failed to obtain JDBC connection and similar errors, I have working Keycloak for 1-2 minutes, create new realm and see it appeared in postgres database through connected pgAdmin.
But then fly.io is restarting the app couple of times and failed at some point. But sometimes I see Error abort in terminal, not passed health checks: 1 total, 1 critical. But Keycloak app keep working. Not sure how to et fully successful deploy as it worked on 18.0.2 without errors and with passed health check.

UPD:
Anyway it worked for 10 minutes and health check did it’s job. Failed with message in dashboard

Failed due to unhealthy allocations - not rolling back to stable job version 39 as current job has same specification.

UPD2: Have no idea why, but with another deploy dashboard says Failed due to unhealthy allocations, but looks like fly restarted app and it works. Anyway, super unstable and have to be fixed…

I actually did this late last year, but I didn’t have time to get this out there until now. Here is a repo which should be enough to get you rolling: GitHub - havarnov/imitatio: Running Keycloak on fly.io.

NOTE:

  • scale to at least 1GB memory
  • using internal DNS to find instances (needed for Keycloak’s distributed cache)
  • using fly.io’s posgres “service”
1 Like

Thank you so much!
I will try it asap. . .

So…
Worked just great!!! Thank you very much again, highly appreciate your help Havar.

Tell me please, where did you get the information about how to create xml cache config file? I wish to understand that.

I’m glad I could help out!

Just to be clear: if you don’t want/need to scale beyond 1 instance of KC, then you don’t need the custom cache config.

It was a real struggle and I couldn’t get it to work correctly until I found and read the documentation for JGroups (Reliable group communication with JGroups) which KC is using.

1 Like

Yes, you are right, works without cache, thanks.

I’ve found that in my case build could not be deployed because of these parameters (in addition to yours in this section which I had too).
That was mainly the only difference comparing with your config.
When I build your config adding these, it can’t be deployed neither.

[[services.http_checks]]
    method = "get"
    path = "/health"
    protocol = "http"
    tls_skip_verify = false

Have no idea why doesn’t work, just took these parameters from this thread above.

anyone figure out how to run keycloak on fly’s private network? I’m currently trying to do that, but always getting HTTPS Required error when trying to access the admin console from fly proxy via http://127.0.0.1:8080