Whisper transcript app timing out

1

I deployed an app with this github project: Issues · ahmetoner/whisper-asr-webservice · GitHub

It’s just a python app that runs the whisper audio transcription service. My project is literally just a fly.toml file

# fly.toml app configuration file generated for whisper-asr on 2023-07-15T13:26:24-05:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

primary_region = "ord"

[build]
  image = "onerahmet/openai-whisper-asr-webservice:latest"

[http_service]
  internal_port = 9000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

[env]
  ASR_MODEL = "base"
  ASR_ENGINE = "openai_whisper"

I scaled it to 1 CPU and 2048 MB in memory. However, when I try to upload and transcribe a 7 minute file, it times out and responds with a 502 error code.

What could the issue be? Is it because i need a more beefier machine?

Update: fly scale vm performance-2x also didnt work

2

It looks like you’re getting bit by the idle timeout which is 60 seconds. If no data is transferred in that time, our proxy closes the connection. There are a few ways around this. You can try to do streaming http to connect the connection open, or immediately spawn a task and return an ID which the client can then use to poll for changes. Additionally you could do it over websocket as well where it would be easy to keep open.

3

Thank you! Is there a way I could have debugged this myself for cases in the future or is it the idle timeout something I would have to just know about? Trying to help my future self!

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.