it's not possible to run Selenium/ChromeDriver on Fly.io

I have been at this for a few days, and I always face the dreaded:

selenium.common.exceptions.WebDriverException: Message: Service /home/cnb/.wdm/drivers/chromedriver/linux64/110.0.5481/chromedriver unexpectedly exited. Status code was: 127

I run a Flask Web App, it’s pretty simple and I haven’t written anything beyond 50 lines of code give or take. I tried adding some missing libraries as suggested by this answer but to no avail, here’s my Dockerfile:

FROM python:3.9.7

RUN apt-get update \
  && apt-get install -y chromium chromium-driver

RUN apt-get install -y python3-pip \
  && pip3 install selenium

also here’s my actual app.py:

from flask import Flask, Response
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

app = Flask(__name__)

@app.route('/video_feed/<url>')
def video_feed(url):
    return Response(generate_frame("https://"+url), mimetype='multipart/x-mixed-replace; boundary=frame')


def init_browser():
    options = Options()
    options.add_argument('--disable-extensions')
    options.add_argument('--disable-gpu')
    options.add_argument('--no-sandbox')
    options.add_argument('--headless')
    browser = webdriver.Chrome(ChromeDriverManager().install(), options=options)
    return browser


def generate_frame(url):
    browser = init_browser()
    browser.get(url)
    while True:
        screenshot = browser.get_screenshot_as_png()
        yield (b'--frame\r\n'
               b'Content-Type: image/png\r\n\r\n' + screenshot + b'\r\n')


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

The app works perfectly fine when ran locally I have never been able to get it working on Fly.io

also here’s the whole error trace in case I missed something in it:

 2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] [2023-02-27 19:18:37 +0000] [538] [ERROR] Error handling request /video_feed/google.com

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] Traceback (most recent call last):

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/gunicorn/workers/sync.py", line 136, in handle

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] self.handle_request(listener, req, client, addr)

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/gunicorn/workers/sync.py", line 184, in handle_request

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] for item in respiter:

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/werkzeug/wsgi.py", line 500, in __next__

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] return self._next()

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 50, in _iter_encoded

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] for item in iterable:

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/workspace/app.py", line 25, in generate_frame

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] browser = init_browser()

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/workspace/app.py", line 20, in init_browser

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] browser = webdriver.Chrome(ChromeDriverManager().install(), options=options)

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 80, in __init__

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] super().__init__(

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 101, in __init__

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] self.service.start()

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 104, in start

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] self.assert_process_still_running()

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] File "/layers/paketo-buildpacks_pip-install/packages/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 117, in assert_process_still_running

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] raise WebDriverException(f"Service {self.path} unexpectedly exited. Status code was: {return_code}")

2023-02-27T19:18:37.098 app[3d4fa991] cdg [info] selenium.common.exceptions.WebDriverException: Message: Service /home/cnb/.wdm/drivers/chromedriver/linux64/110.0.5481/chromedriver unexpectedly exited. Status code was: 127 

I have a hunch that I need to install some kind of chrome related software but I can’t figure out what it is.

Any help will be greatly appreciated because this is driving me nuts.

My guess is the Chrome is running out of memory.

I doubt Selenium is the problem.

You definitely can run Chrome/Chromedriver on fly. I can prove it to you.

If you are running on a Mac, Linux, or WSL2 try running the following in an empty directory (if you run on windows CMD.exe or PS some syntax changes will be required):

gem install rails
rails new demo --minimal
cd demo
bundle add grover
npm install puppeteer

echo 'Rails.application.routes.draw { root "grover#pdf" }' > config/routes.rb

cat << 'EOF' > app/controllers/grover_controller.rb
class GroverController < ApplicationController
  def pdf
    grover = Grover.new('https://google.com', format: 'A4')
    send_data grover.to_pdf, filename: 'google.pdf', type: :pdf
  end
end
EOF

fly launch
fly deploy
fly open

Feel free to accept the defaults on fly launch (i.e., you don’t need a db or redis).

This will run Rails/Puppeteer/ChromeDriver on fly.io. Take a look at the Dockerfile and steal what you like. If you prefer Chrome over Chromium, regenerate the dockerfile with:

bin/rails generate dockerfile --platform linux/amd64

I tried scaling the memory up significantly, I don’t think it’s the problem, maybe there is a bare minimum requirement of libraries to run ChromeDriver that I am not aware of?

I wasn’t able to get the code you provided running (I am on windows), maybe you have that Dockerfile available to you?

Thanks for you help but unfortunately that didn’t do it either!

Hey!, check this out: python - WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: 127 - Stack Overflow
I got it working…
installed chromium but then I had to install other dependencies to make it work