Yarn Workspaces

matt5 · October 2, 2021, 10:04pm

I’m moving my multi-app monorepo from Heroku to Fly and I have questions. My repo uses yarn workspaces to share dependencies. To get this working in Heroku, I needed to use two build packs:

-----> Building on the Heroku-20 stack
-----> Using buildpacks:
1. GitHub - heroku/heroku-buildpack-nodejs: The official Heroku buildpack for Node.js apps.
2. GitHub - heroku/heroku-buildpack-multi-procfile: Everyone gets a Procfile!
-----> Node.js app detected

So my first question is can I use yarn workspaces to build and deploy my individual apps independently? If so, how?

Fly uses the first build pack, but not the second one. It can’t find the dependencies when it builds because they live at a higher level.
It doesn’t seem like fly is using yarn, but rather npm.

kurt · October 6, 2021, 2:30am

The best way to do this is to skip the buildpack and build + deploy with two Dockerfiles. What does your app structure look like?

danwetherald · October 13, 2021, 2:59am

@kurt - Could you explain further on the two Dockerfiles?

I am having a similar issue using Docker because I believe only the /app directory is available when deploying so the: RUN yarn install in my Dockerfile is not able to access the packages above /app since they are not in scope?

/repo
  /package-a
    package.json
  /package-b
    package.json
  /app
    package.json (has package-a, package-b dependencies)
    fly.toml
package.json

Root package.json

"workspaces": {
    "packages": [
      "app",
      "package-a",
      "package-b"
    ]
}

FROM node:current-alpine

WORKDIR /app

COPY package.json .

RUN yarn install

COPY . .

CMD ["yarn", "start"]

Thanks!

jsierles · October 13, 2021, 10:33am

This is a bit tricky. Depending on your needs, changing your WORKDIR and running commands may be sufficient. Here’s an example that install packages in all directories, then copies the result into a final Docker build step. If the latter makes no sense, we can work through it here.

I believe what Kurt was suggesting was to keep the Dockerfiles separate for each app in the root and reference them separately with fly deploy --dockerfile. Adding a bit of scripting around fly deploy should work. That said, it would be nice for flyctl to detect this situation and help you

danwetherald · October 13, 2021, 4:07pm

Ah, yup, we are currently doing this a TON as we have a monorepo with 10+ apps deployed on fly

As for this project its a slightly different story as it requires packages from levels above the “deployment app’s” directory and normally is out of scope / does not exist once docker starts the build as only the “deployment app’s” built directory is pushed up, problem being the “yarn install” command once the docker file starts up will fail.

I actually stumbled upon this repo in my search last night. My problem is, this would mean that the entire repo is deployed to fly, so I actually started along a path that does some moving of directories around in my circleci build steps to only send up the monorepo packages that are needed, but I am far from a “Dockerfile expert”.

Here is my current situation after the build step:

      - run:
          name: Add Workspace Dependencies
          command: |
            mkdir deploy-totan
            mv package.json ./deploy-totan/package.json
            mv .npmrc ./deploy-totan/.npmrc
            mv wonka ./deploy-totan/wonka
            mv totan ./deploy-totan/totan
      - persist_to_workspace:
          root: ~/project
          paths:
            - deploy-totan

So as you can see /wonka is the local yarn workspace package that is needed by /totan to run “yarn install”

Any thoughts on this “Add Workspace Dependencies” strategy in circle, and if so, any suggestions for a Dockerfile?

Here is my WIP:

FROM node:current-alpine

WORKDIR /usr/src/app

COPY package.json .
COPY .npmrc .
COPY wonka ./wonka
COPY totan ./totan

RUN yarn global add next next-boost

WORKDIR /usr/src/app/totan

CMD ["yarn", "start"]

I believe this might be “close” to working, but getting this error now:

error /usr/local/share/.config/yarn/global/node_modules/better-sqlite3: Command failed.
Exit code: 1
Command: prebuild-install || npm run build-release
Arguments: 
Directory: /usr/local/share/.config/yarn/global/node_modules/better-sqlite3
Output:
prebuild-install WARN install No prebuilt binaries found (target=16.10.0 runtime=node arch=x64 libc=musl platform=linux)

> better-sqlite3@7.4.3 build-release
> node-gyp rebuild --release

Thanks!

jsierles · October 13, 2021, 4:12pm

Can you try adding WORKDIR /usr/src/app/wonka before running yarn install?

Something like this:

FROM node:current-alpine

WORKDIR /usr/src/app

COPY package.json .
COPY .npmrc .
COPY wonka ./wonka
COPY totan ./totan

RUN yarn global add next next-boost

WORKDIR /usr/src/app/totan

RUN yarn install

CMD ["yarn", "start"]

danwetherald · October 13, 2021, 4:17pm

Hey @jsierles - Thanks for the help.

Im not sure I follow, is this the same as I have currently?

Also, it seem the errors I shared above are related to PYTHON not being found in the runtime.

gyp ERR! find Python 
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON

jsierles · October 13, 2021, 4:20pm

In your example, you are not running yarn install anywhere.

I’m not sure what the python thing is about. Can you post your package.json? Sounds like perhaps there’s a node package that uses something from python?

danwetherald · October 13, 2021, 4:22pm

Isn’t yarn install running here?

/totan/package.json

{
  "name": "@bettercart/totan",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "res:build": "rescript",
    "res:clean": "rescript clean",
    "res:start": "rescript build -w",
    "res:relay": "rescript-relay-compiler",
    "res:relay:watch": "rescript-relay-compiler --watch",
    "dev": "next",
    "build": "next build",
    "start": "next-boost"
  },
  "devDependencies": {
    "@rescript/react": "^0.10.3",
    "next-transpile-modules": "^8.0.0",
    "rescript": "9.1.4"
  },
  "dependencies": {
    "@bettercart/wonka": "0.0.1",
    "@react-hook/debounce": "^4.0.0",
    "bs-fetch": "0.6.2",
    "decco": "1.4.0",
    "graphql": "^15.5.0",
    "lodash": "^4.17.21",
    "next": "^10.2.3",
    "next-boost": "^0.10.0",
    "react": "0.0.0-experimental-4e08fb10c",
    "react-dom": "0.0.0-experimental-4e08fb10c",
    "react-relay": "11.0.0",
    "reason-promise": "1.1.4",
    "relay-compiler": "11.0.0",
    "relay-config": "11.0.0",
    "relay-runtime": "11.0.0",
    "rescript-relay": "0.20.1"
  },
  "resolutions": {
    "react": "0.0.0-experimental-4e08fb10c",
    "react-dom": "0.0.0-experimental-4e08fb10c"
  }
}

/package.json

{
  "name": "bettercart",
  "private": true,
  "workspaces": {
    "packages": [
      "jack-bot",
      "wonka",
      "woz",
      "totan"
    ],
    "nohoist": [
      "**/babel**",
      "**/eslint**",
      "**/jest",
      "*monster-bash/**"
    ],
    "extends": [
      "react-app"
    ]
  },
  "dependencies": {}
}

jsierles · October 13, 2021, 4:35pm

I had to switch to another thing, but will give this a try a bit later and let you know what I see.

danwetherald · October 13, 2021, 4:55pm

No problem, working on moving to a ubuntu image and installing yarn or installing python on the node alpine image, will keep you updated.

Topic		Replies	Views
yarn workspace server start: command not found Build debugging mysql	20	1997	May 13, 2023
yarn install fail with yarn3 Questions / Help	3	1042	September 1, 2022
Build time image customization for heroku/buildpacks:20 Questions / Help	8	620	July 6, 2022
Monorepo with multiple node server scripts as siblings	2	360	May 4, 2021
How can I add the pupeeteer buildpack in fly.io which is something similar to heroku. Questions / Help rails	4	958	September 21, 2022

Yarn Workspaces

Related Topics