For some reason, I can’t see to get it to work. My code works locally but puppeteer is not recognized for some reason on fly.io
Hi @artyroip, just to have a bit more context on the failure at what stage is your code failing to run on fly, is it when you’re running a fly launch or fly deploy etc?
And what specific error are you seeing when you try to launch your code on fly?
Hi,
The application deploys successfully but is not able to launch Puppeteer.
It seems as though my node application on fly cannot access the chromium instance.
There are some mildly tricky flags & such that need to be fiddled with, to run puppeteer/chromium in docker.
We’ve had good success with the buildkite/puppeteer docker base image, which takes care of that for you. Here’s the entire contents of our Dockerfile, running on Fly:
FROM buildkite/puppeteer:latest
RUN mkdir /app
WORKDIR /app
COPY package.json yarn.lock /app/
RUN yarn install --frozen-lockfile
COPY ./ /app/
ENV PORT 8000
EXPOSE 8000
CMD yarn start
(Just notice it looks like they’ve stopped maintaining the image, though, so it might be time to find a similar base image.)
1 Like
Perhaps playwright works out of the box on Fly? Docker | Playwright
I managed to get it working with the following Dockerfile:
FROM zenika/alpine-chrome:89-with-node-14
COPY package*.json ./
RUN npm install
COPY . .
ENV PUPPETEER_EXECUTABLE_PATH='/usr/bin/chromium-browser'
EXPOSE 8080
CMD [ "node", "server.js" ]
1 Like
In case it helps anyone out (and as I saw this thread was recently referenced): Here’s our updated Dockerfile for running a puppeteer-based node service on Fly, now based on the node:slim base image:
FROM node:slim AS app
RUN mkdir /app
WORKDIR /app
# Install chrome stable from sources, then remove it. Why? The
# npm install of `puppeteer` brings its own bundle chromium build,
# and puppeteer releases are only guaranteed to work with that version.
#
# But the bundled chromium implicitly needs a bunch of shared libs on
# the host. It's a little tedious to find and maintain that list; but
# the official apt distro of `google-chrome-stable` should bring the right
# set along. So do that, but immediately uninstall (to free up layer space).
#
# This is a little brittle, since the puppeteer chrome could in theory diverge
# from the official apt chrome's shared lib deps. But it works..
RUN apt-get update \
&& apt-get install curl gnupg -y \
&& curl --location --silent https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install google-chrome-stable -y --no-install-recommends \
&& apt-get remove google-chrome-stable -y \
&& rm -rf /var/lib/apt/lists/*
# Add emoji fonts.
# Source: https://gist.github.com/win0err/9d8c7f0feabdfe8a4c9787b02c79ac51
RUN mkdir ~/.fonts/ && \
wget https://github.com/samuelngs/apple-emoji-linux/releases/download/ios-15.4/AppleColorEmoji.ttf -O ~/.fonts/AppleColorEmoji.ttf
COPY package.json yarn.lock /app/
RUN yarn install --frozen-lockfile
COPY ./ /app/
ENV PORT 8000
EXPOSE 8000
CMD yarn start
3 Likes
Hi, I used the docker file above, it seems to work but when my code reaches to the line page.goto(url) it keeps saying connection error on the logs of fly.io. It says ERR_NETWORK_CHANGED. Can someone make a YouTube video on how to get puppeteer working on fly.io. I am trying to navigate from heroku to fly.io. Also can someone tell me what’s wrong since I’m using the puppeteer module notnppuppeteeer-core
I’m getting this error when using your Dockerfile. Can you show how you’re initializing puppeteer?
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] Browser creation: Error: Failed to launch the browser process!
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] /workspace/node_modules/puppeteer/.local-chromium/linux-1036745/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at onClose (/workspace/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:290:20)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at Interface.<anonymous> (/workspace/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:278:24)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at Interface.emit (node:events:525:35)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at Interface.close (node:internal/readline/interface:536:10)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at Socket.onend (node:internal/readline/interface:262:10)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at Socket.emit (node:events:525:35)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at endReadableNT (node:internal/streams/readable:1359:12)
2022-09-13T21:33:17.543 app[225f9bb3] sea [info] at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Sure, here’s a lightly edited excerpt:
In Dockerfile, add a flag if you want to distinguish between local dev (no docker) from prod / docker run. You’ll see how it is used below.
ENV RUNNING_IN_DOCKER true
In node:
const puppeteer = require('puppeteer');
const { Cluster } = require('puppeteer-cluster');
const { PUPPETEER_EXECUTABLE_PATH, BROWSER_MAX_CONCURRENCY, RUNNING_IN_DOCKER } = process.env;
const puppeteerArgs = RUNNING_IN_DOCKER ? ['--no-sandbox', '--disable-setuid-sandbox'] : [];
async function startBrowser() {
const puppeteerOptions = {
args: puppeteerArgs,
};
if (PUPPETEER_EXECUTABLE_PATH) {
puppeteerOptions.executablePath = PUPPETEER_EXECUTABLE_PATH;
}
const maxConcurrency = Number.parseInt(BROWSER_MAX_CONCURRENCY, 10) || 2;
const clusterConfig = {
concurrency: Cluster.CONCURRENCY_CONTEXT,
maxConcurrency,
puppeteerOptions,
};
const cluster = await Cluster.launch(clusterConfig);
await cluster.task(ourTaskName);
logger.info('Cluster initialized');
}
puppeteer-cluster is a third-party node lib that manages a pool of puppeteer instances. I am pretty sure we added it to the mix when needing some general watchdog duties for the chromium process, which we discovered was occasionally crashing & confusing the node wrapper. Good experience with that lib, but not strictly necessary - puppeteerArgs is main thing I would compare.
All that said… your problem might be something else:
chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory
It might be that you need to apt-get install that library, or whatever provides it, in the Dockerfile. Sorry, I know that phase of the image build is potentially brittle.
I’m still having this same issue. I’ve used mikeys solution and also artyroip solution. But no dice. Is there any fix?? This is killing me lol
Try this:
FROM zenika/alpine-chrome:101-with-node-16
COPY package*.json ./
USER root
RUN npm install
COPY . .
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD='true'
ENV PUPPETEER_EXECUTABLE_PATH='/usr/bin/chromium-browser'
EXPOSE 8080
ENV PORT 8080
CMD ["/bin/sh", "setup.sh"]
I also had the same issue a few days. But at last adding Dockerfile few changes, i managed to solve this.
check this repo - alpine-chrome/with-puppeteer at master · Zenika/alpine-chrome · GitHub
None of these solutions seem to work for my Node.js app. It sounds like I need to update my Docker file. I’ve tried a handful of things from this post, and the various documentations, but no luck.
Below is my current Docker file. Can anyone please provide what the new Docker file should be?
# syntax = docker/dockerfile:1
# Adjust NODE_VERSION as desired
ARG NODE_VERSION=18.17.1
FROM node:${NODE_VERSION}-slim as base
LABEL fly_launch_runtime="Node.js"
# Node.js app lives here
WORKDIR /app
# Set production environment
ENV NODE_ENV="production"
# Throw-away build stage to reduce size of final image
FROM base as build
# Install packages needed to build node modules
RUN apt-get update -qq && \
apt-get install -y build-essential pkg-config python-is-python3
# Install node modules
COPY --link package-lock.json package.json ./
RUN npm ci
# Copy application code
COPY --link . .
# Final stage for app image
FROM base
# Copy built application
COPY --from=build /app /app
# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD [ "node", "index.js" ]
I’ve updated dockerfile-node to support puppeteer. You can have it update your Dockerfile by issuing the following commands:
npm update @flydotio/dockerfile
npx dockerfile
If you would prefer to make these changes yourself, you can see a working Dockerfile at: https://github.com/fly-apps/dockerfile-node/blob/main/test/base/puppeteer/Dockerfile
Chrome is a memory hog. At a minimum, you will likely need 1G of RAM. See fly scale memory · Fly Docs and Fly Launch configuration (fly.toml) · Fly Docs for options.
You also are going to want to run headless and without a sandbox:
const browser = await puppeteer.launch({
headless: "new",
args: ['--no-sandbox', '--disable-setuid-sandbox']
});
3 Likes
thank you, I really appreciate it!
Did something change related to this? It was working just fine and now the browser.newPage() fails:
const browser = await puppeteer.launch({
executablePath:
NODE_ENV !== "development"
? CHROME_EXECUTABLE_PATH || "/usr/bin/chromium"
: undefined,
args: ["--no-sandbox", "--disable-setuid-sandbox", "--disabled-gpu"],
dumpio: true,
// @ts-ignore
headless: "new",
});
const newPage = await browser.newPage();
UPDATE:
After 2 days of playing with this we have found a setup that works. Only with this approach I was able to make it work:
const browser = await puppeteer.launch({
executablePath:
NODE_ENV !== "development"
? CHROME_EXECUTABLE_PATH || "/usr/bin/chromium"
: undefined,
args: [
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-gpu",
"--disable-software-rasterizer",
],
// @ts-ignore
headless: "new",
});
# Install dependencies
RUN apt-get update && apt-get install -y \
chromium \
fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
# Set environment variables
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true \
PUPPETEER_SKIP_DOWNLOAD=true \
CHROME_EXECUTABLE_PATH="/usr/bin/chromium"
PR fix in our app: fix: pdf generator crash by DonKoko · Pull Request #1235 · Shelf-nu/shelf.nu · GitHub
1 Like