Yarn Workspaces

I’m moving my multi-app monorepo from Heroku to Fly and I have questions. My repo uses yarn workspaces to share dependencies. To get this working in Heroku, I needed to use two build packs:

-----> Building on the Heroku-20 stack
-----> Using buildpacks:
1. GitHub - heroku/heroku-buildpack-nodejs: The official Heroku buildpack for Node.js apps.
2. GitHub - heroku/heroku-buildpack-multi-procfile: Everyone gets a Procfile!
-----> Node.js app detected

So my first question is can I use yarn workspaces to build and deploy my individual apps independently? If so, how?

  1. Fly uses the first build pack, but not the second one. It can’t find the dependencies when it builds because they live at a higher level.
  2. It doesn’t seem like fly is using yarn, but rather npm.

The best way to do this is to skip the buildpack and build + deploy with two Dockerfiles. What does your app structure look like?

@kurt - Could you explain further on the two Dockerfiles?

I am having a similar issue using Docker because I believe only the /app directory is available when deploying so the: RUN yarn install in my Dockerfile is not able to access the packages above /app since they are not in scope?

/repo
  /package-a
    package.json
  /package-b
    package.json
  /app
    package.json (has package-a, package-b dependencies)
    fly.toml
package.json

Root package.json

"workspaces": {
    "packages": [
      "app",
      "package-a",
      "package-b"
    ]
}
FROM node:current-alpine

WORKDIR /app

COPY package.json .

RUN yarn install

COPY . .

CMD ["yarn", "start"]

Thanks!

This is a bit tricky. Depending on your needs, changing your WORKDIR and running commands may be sufficient. Here’s an example that install packages in all directories, then copies the result into a final Docker build step. If the latter makes no sense, we can work through it here.

I believe what Kurt was suggesting was to keep the Dockerfiles separate for each app in the root and reference them separately with fly deploy --dockerfile. Adding a bit of scripting around fly deploy should work. That said, it would be nice for flyctl to detect this situation and help you :smiley:

Ah, yup, we are currently doing this a TON as we have a monorepo with 10+ apps deployed on fly :slightly_smiling_face:

As for this project its a slightly different story as it requires packages from levels above the “deployment app’s” directory and normally is out of scope / does not exist once docker starts the build as only the “deployment app’s” built directory is pushed up, problem being the “yarn install” command once the docker file starts up will fail.

I actually stumbled upon this repo in my search last night. My problem is, this would mean that the entire repo is deployed to fly, so I actually started along a path that does some moving of directories around in my circleci build steps to only send up the monorepo packages that are needed, but I am far from a “Dockerfile expert”.

Here is my current situation after the build step:

      - run:
          name: Add Workspace Dependencies
          command: |
            mkdir deploy-totan
            mv package.json ./deploy-totan/package.json
            mv .npmrc ./deploy-totan/.npmrc
            mv wonka ./deploy-totan/wonka
            mv totan ./deploy-totan/totan
      - persist_to_workspace:
          root: ~/project
          paths:
            - deploy-totan

So as you can see /wonka is the local yarn workspace package that is needed by /totan to run “yarn install”

Any thoughts on this “Add Workspace Dependencies” strategy in circle, and if so, any suggestions for a Dockerfile?

Here is my WIP:

FROM node:current-alpine

WORKDIR /usr/src/app

COPY package.json .
COPY .npmrc .
COPY wonka ./wonka
COPY totan ./totan

RUN yarn global add next next-boost

WORKDIR /usr/src/app/totan

CMD ["yarn", "start"]

I believe this might be “close” to working, but getting this error now:

error /usr/local/share/.config/yarn/global/node_modules/better-sqlite3: Command failed.
Exit code: 1
Command: prebuild-install || npm run build-release
Arguments: 
Directory: /usr/local/share/.config/yarn/global/node_modules/better-sqlite3
Output:
prebuild-install WARN install No prebuilt binaries found (target=16.10.0 runtime=node arch=x64 libc=musl platform=linux)

> better-sqlite3@7.4.3 build-release
> node-gyp rebuild --release

Thanks!

Can you try adding WORKDIR /usr/src/app/wonka before running yarn install?

Something like this:

FROM node:current-alpine

WORKDIR /usr/src/app

COPY package.json .
COPY .npmrc .
COPY wonka ./wonka
COPY totan ./totan

RUN yarn global add next next-boost

WORKDIR /usr/src/app/totan

RUN yarn install

CMD ["yarn", "start"]

Hey @joshua - Thanks for the help.

Im not sure I follow, is this the same as I have currently?

Also, it seem the errors I shared above are related to PYTHON not being found in the runtime.

gyp ERR! find Python 
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON

In your example, you are not running yarn install anywhere.

I’m not sure what the python thing is about. Can you post your package.json? Sounds like perhaps there’s a node package that uses something from python?

Isn’t yarn install running here?

/totan/package.json

{
  "name": "@bettercart/totan",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "res:build": "rescript",
    "res:clean": "rescript clean",
    "res:start": "rescript build -w",
    "res:relay": "rescript-relay-compiler",
    "res:relay:watch": "rescript-relay-compiler --watch",
    "dev": "next",
    "build": "next build",
    "start": "next-boost"
  },
  "devDependencies": {
    "@rescript/react": "^0.10.3",
    "next-transpile-modules": "^8.0.0",
    "rescript": "9.1.4"
  },
  "dependencies": {
    "@bettercart/wonka": "0.0.1",
    "@react-hook/debounce": "^4.0.0",
    "bs-fetch": "0.6.2",
    "decco": "1.4.0",
    "graphql": "^15.5.0",
    "lodash": "^4.17.21",
    "next": "^10.2.3",
    "next-boost": "^0.10.0",
    "react": "0.0.0-experimental-4e08fb10c",
    "react-dom": "0.0.0-experimental-4e08fb10c",
    "react-relay": "11.0.0",
    "reason-promise": "1.1.4",
    "relay-compiler": "11.0.0",
    "relay-config": "11.0.0",
    "relay-runtime": "11.0.0",
    "rescript-relay": "0.20.1"
  },
  "resolutions": {
    "react": "0.0.0-experimental-4e08fb10c",
    "react-dom": "0.0.0-experimental-4e08fb10c"
  }
}

/package.json

{
  "name": "bettercart",
  "private": true,
  "workspaces": {
    "packages": [
      "jack-bot",
      "wonka",
      "woz",
      "totan"
    ],
    "nohoist": [
      "**/babel**",
      "**/eslint**",
      "**/jest",
      "*monster-bash/**"
    ],
    "extends": [
      "react-app"
    ]
  },
  "dependencies": {}
}

I had to switch to another thing, but will give this a try a bit later and let you know what I see.

No problem, working on moving to a ubuntu image and installing yarn or installing python on the node alpine image, will keep you updated.