RedwoodJS deployment is not working

Hi there,

I have a RedwoodJS project which I have setup to work with Fly using these instructions: Run a RedwoodJS App · Fly Docs

This has generated the following files for me:

.fly/migrate.sh

#!/bin/sh

set -ex

# This command pushes us over 256MB of RAM at release time
# yarn rw prisma migrate deploy

# This alternative command uses less memory
npx prisma migrate deploy --schema /app/api/db/schema.prisma

./fly/release.sh

#!/bin/sh

set -ex

if [ ! -n $MIGRATE_ON_BOOT ]; then
  $(dirname $0)/migrate.sh
fi

./fly/start.sh

#!/bin/sh

set -ex

if [ -n $MIGRATE_ON_BOOT ]; then
  $(dirname $0)/migrate.sh
fi

npx rw-server --port ${PORT} $@

Dockerfile (and .dockerignore which I’ve omitted for brevity):

ARG BASE_IMAGE=node:16.13.0-alpine
FROM ${BASE_IMAGE} as base

RUN mkdir /app
WORKDIR /app

# Required for building the api and web distributions
ENV NODE_ENV development

FROM base as dependencies

COPY .yarn .yarn
COPY .yarnrc.yml .yarnrc.yml
COPY package.json package.json
COPY web/package.json web/package.json
COPY api/package.json api/package.json
COPY yarn.lock yarn.lock

RUN --mount=type=cache,target=/root/.yarn/berry/cache \
    --mount=type=cache,target=/root/.cache yarn install --immutable

COPY redwood.toml .
COPY graphql.config.js .

FROM dependencies as web_build

COPY web web
RUN yarn rw build web

FROM dependencies as api_build

COPY api api
RUN yarn rw build api

FROM dependencies

ENV NODE_ENV production

COPY --from=web_build /app/web/dist /app/web/dist
COPY --from=api_build /app/api /app/api
COPY --from=api_build /app/node_modules/.prisma /app/node_modules/.prisma

COPY .fly .fly

ENTRYPOINT ["sh"]
CMD [".fly/start.sh"]

fly.toml

# fly.toml app configuration file generated for project-ibis-rw-dev on 2023-12-28T12:15:59+13:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = "OMITTED"
primary_region = "OMITTED"

[build]

[deploy]
  release_command = ".fly/release.sh"

[env]
  PORT = "8910"
  REDWOOD_DISABLE_TELEMETRY = "1"

[http_service]
  internal_port = 8910
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ["app"]

[[vm]]
  cpu_kind = "shared"
  cpus = 1
  memory_mb = 1024

After running flyctl launch I updated my redwood.toml to:

# This file contains the configuration settings for your Redwood app.
# This file is also what makes your Redwood app a Redwood app.
# If you remove it and try to run `yarn rw dev`, you'll get an error.
#
# For the full list of options, see the "App Configuration: redwood.toml" doc:
# https://redwoodjs.com/docs/app-configuration-redwood-toml

[web]
  title = "OMITTED"
  port = 8910
  host = "0.0.0.0"
  apiUrl = "/api"
  includeEnvironmentVariables = [
    # Add any ENV vars that should be available to the web side to this array
    # See https://redwoodjs.com/docs/environment-variables#web
  ]
[api]
  port = 8911
  host = "0.0.0.0"
[browser]
  open = true
[notifications]
  versionUpdates = ["latest"]

However when the project deploys I get the following error:

[error] failed to connect to machine: gave up after 15 attempts (in 9.272853195s)

[error] instance refused connection. is your app listening on 0.0.0.0:8910? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)

Any help is apprecaited.

Is there anything relevant in your startup logs? Try running fly logs in a second window when you run fly deploy.

Doesn’t seem to be. It originally complained about a failed migration but I relaunched without any migrations and it was still inaccessible.

Tried to relaunch a few times (rerun flyctl launch) and am consistently getting this error:

WARNING The app is not listening on the expected address and will not be reachable by fly-proxy.8 [app] is up and running
You can fix this by configuring your app to listen on the following addresses:
  - 0.0.0.0:8910

In addition, I updated my setup to use an externally hosted DB to isolate that as being a problem. I can now see the following error when running flyctl logs:

[info]+ npx rw-server --port 8910
[info]Starting API and Web Servers...
[info]Loading server config from /app/api/server.config.js
[info](node:392) ExperimentalWarning: stream/web is an experimental feature. This feature could change at any time
[info](Use `node --trace-warnings ...` to show where the warning was created)
[info]/app/node_modules/fastify/lib/decorate.js:41
[info]    throw new FST_ERR_DEC_ALREADY_PRESENT(name)
[info]    ^
[info]FastifyError [Error]: The decorator 'urlData' has already been added!
[info]    at decorateConstructor (/app/node_modules/fastify/lib/decorate.js:41:11)
[info]    at Object.decorateRequest (/app/node_modules/fastify/lib/decorate.js:123:3)
[info]    at fastifyUrlData (/app/node_modules/@fastify/url-data/plugin.js:7:11)
[info]    at Plugin.exec (/app/node_modules/avvio/plugin.js:130:19)
[info]    at Boot.loadPlugin (/app/node_modules/avvio/plugin.js:272:10)
[info]    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
[info]  code: 'FST_ERR_DEC_ALREADY_PRESENT',
[info]  statusCode: 500
[info]}
[info] INFO Main child exited normally with code: 1
[info] INFO Starting clean up.
[info] WARN hallpass exited, pid: 307, status: signal: 15 (SIGTERM)

Looking at my /api/server.config.js I have the defaults set:

/**
 * This file allows you to configure the Fastify Server settings
 * used by the RedwoodJS dev server.
 *
 * It also applies when running RedwoodJS with `yarn rw serve`.
 *
 * For the Fastify server options that you can set, see:
 * https://www.fastify.io/docs/latest/Reference/Server/#factory
 *
 * Examples include: logger settings, timeouts, maximum payload limits, and more.
 *
 * Note: This configuration does not apply in a serverless deploy.
 */

/** @type {import('fastify').FastifyServerOptions} */
const config = {
  requestTimeout: 15_000,
  logger: {
    // Note: If running locally using `yarn rw serve` you may want to adjust
    // the default non-development level to `info`
    level: process.env.NODE_ENV === 'development' ? 'debug' : 'warn',
  },
}

/**
 * You can also register Fastify plugins and additional routes for the API and Web sides
 * in the configureFastify function.
 *
 * This function has access to the Fastify instance and options, such as the side
 * (web, api, or proxy) that is being configured and other settings like the apiRootPath
 * of the functions endpoint.
 *
 * Note: This configuration does not apply in a serverless deploy.
 */

/** @type {import('@redwoodjs/api-server/dist/types').FastifySideConfigFn} */
const configureFastify = async (fastify, options) => {
  if (options.side === 'api') {
    fastify.log.trace({ custom: { options } }, 'Configuring api side')
  }

  if (options.side === 'web') {
    fastify.log.trace({ custom: { options } }, 'Configuring web side')
  }

  return fastify
}

module.exports = {
  config,
  configureFastify,
}

I just tried starting from scratch with

yarn create redwood-app my-redwood-project

I added host to my redwood.toml:

# This file contains the configuration settings for your Redwood app.
# This file is also what makes your Redwood app a Redwood app.
# If you remove it and try to run `yarn rw dev`, you'll get an error.
#
# For the full list of options, see the "App Configuration: redwood.toml" doc:
# https://redwoodjs.com/docs/app-configuration-redwood-toml

[web]
  title = "Redwood App"
  host = '0.0.0.0'
  port = 8910
  apiUrl = "/.redwood/functions" # You can customize graphql and dbauth urls individually too: see https://redwoodjs.com/docs/app-configuration-redwood-toml#api-paths
  includeEnvironmentVariables = [
    # Add any ENV vars that should be available to the web side to this array
    # See https://redwoodjs.com/docs/environment-variables#web
  ]
[api]
  host = '0.0.0.0'
  port = 8911
[browser]
  open = true
[notifications]
  versionUpdates = ["latest"]

I then created a miinimal Dockerfile:

# Adjust NODE_VERSION as desired
ARG NODE_VERSION=18.19.0
FROM node:${NODE_VERSION}-slim as base

LABEL fly_launch_runtime="RedwoodJS"

# Install Yarn
ARG YARN_VERSION=4.0.2
RUN corepack enable && \
    yarn set version $YARN_VERSION

# Copy project
WORKDIR /my-redwood-project
COPY . .

# Build application
RUN yarn install && \
    yarn rw build

# Start the server by default, this can be overwritten at runtime
EXPOSE 8910
CMD [ "yarn", "rw", "serve" ]

For the moment, you can see the result here: https://my-redwood-project.fly.dev/ (I’ll take that app down once it serves its purpose).

Clearly this is not a representative project, nor is this an complete Dockerfile, but it does work.

Now I don’t know RedwoodJS, but I do know fly.io and Node.js. What should be changed to make it more representative and exhibit the problem you are seeing?

If possible, I would like the end result of this exercise to be for me to update fly.io’s dockerfile generator to create a Dockerfile that will work for at least your application as that can be the starting point for others.

Hey @rubys, thanks for the follow up. Sorry for the delayed reply, I’ve been sidetracked with holiday celebrations. I’ll pick this up in the next few days and update you.

Hey @rubys, so I gave that Dockerfile and redwood.toml configuration a go and encountered another issue. I’m happy to troubleshoot this with you and see if we can get to the root cause. Is there an easier way than posting here? Would be great if we could then update the fly.io generator for others in the future as you say.

I’m fine with here or elsewhere. Where do you suggest?

FWIW, the dockerfile generator is here: GitHub - fly-apps/dockerfile-node: Dockerfile generator for Node.js, it is just a bunch of templates and a small main program that takes arguments and can inspect the source to determine what is desired.

I am getting the same exact error for one of my project at the time of deployment i.e running the docker container and not even a bit of code has changed so i am very surprised what could be the cause.

Hi @oreid,

I was having the same issue and I’ve tracked it down to a breaking change in the rw-server package, so in your ./fly/start.sh you’re pulling the breaking change with your npx rw-server call.

Unfortunately rw-server is still in experimental status, so there can be breaking changes introduced there, for stability it might be best to use the cli yarn rw serve api - I’m not familiar with fly so I’m not sure about what the best approach would be to call that command from the start script.

If the script has access to the node_modules folder you could call

./node_modules/.bin/redwood serve api --port 8910

to use the serve command that will be pinned to whatever redwoodjs version your app is running, or if for whatever reason you need to call it remotely you can use

npx redwood serve api --port 8910

Thanks for the in depth analysis @tbay06! As you suggest, it appears to be an issue with npx rw-server. Switching over to ./node_modules/.bin/redwood serve api --port 8910 has fixed the issue.

For completeness, I think the issues I was encountering came down to:

  • I was using the free database instance which didn’t have enough memory to apply my database migrations
  • The above issue you mentioned with the breaking change in rw-server which turned out to be an issue with Fastify (more details below)

Updating the Postgres instance I was using and changing the command in ./fly/start.sh fixed the issue. In addition, the Fastify issue looks to have been fixed on the RedwoodJS side (see [Bug?]: FastifyError [Error]: The decorator 'urlData' has already been added! · Issue #9789 · redwoodjs/redwood · GitHub) and I can see you were involved there too so thanks for that).

I think for now we have a work around (via the mentioned fixes) and hopefully once the fix goes out from fix(fastify): Prevent duplicate `@fastify/url-data` registration by Josh-Walker-GM · Pull Request #9794 · redwoodjs/redwood · GitHub the default generated code will work. Not sure if you want to change anything on the Fly side in the mean time @rubys?