Cold Start Causes 1-Minute Timeout for First Request (FastAPI + Nginx)

Python
,
1

When my Fly.io server is suspended and it receives a request, sometimes it doesn’t respond for 1 minute and the request times out. However, when the server is running and receives a request again, it responds without any issues. How can I solve this problem? I’m using FastAPI and Nginx.

Fly.io Machine logs:

$ fly logs -a backend-identity 

Waiting for logs...

2025-06-12T12:08:54.675 proxy[e28604e4bd5486] fra [info] Starting machine

2025-06-12T12:08:54.866 app[e28604e4bd5486] fra [info] 2025-06-12T12:08:54.866799443 [01JXGDJWPBG0B12BDNN6E0CBW4:main] Running Firecracker v1.7.0

2025-06-12T12:08:54.867 app[e28604e4bd5486] fra [info] 2025-06-12T12:08:54.867926876 [01JXGDJWPBG0B12BDNN6E0CBW4:fc_api] The API server received a Put request on "/logger" with body "{\"log_path\":\"logs.fifo\",\"level\":\"info\"}".

2025-06-12T12:08:55.093 runner[e28604e4bd5486] fra [info] Machine started in 403ms

2025-06-12T12:08:55.096 proxy[e28604e4bd5486] fra [info] machine started in 420.864897ms

2025-06-12T12:08:55.106 proxy[e28604e4bd5486] fra [info] machine became reachable in 10.048795ms

2025-06-12T12:09:55.316 app[e28604e4bd5486] fra [info] ERROR stderr to vsock zero copy err: Broken pipe (os error 32)

2025-06-12T12:09:55.316 app[e28604e4bd5486] fra [info] 176.235.138.138 - [12/Jun/2025:15:09:55 +0300] "POST /auth/login HTTP/1.1" 504 569 bytes

fly.toml:

app = 'backend-identity'
primary_region = 'fra'

[build]

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines = 'suspend'
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']
[[vm]]
  memory = '1gb'
  cpu_kind = 'shared'
  cpus = 1

[env]
  ENV='production'

Dockerfile:

FROM python:3.11-slim AS backend

# FastAPI
WORKDIR /app
COPY ./requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY . /app

# Nginx
FROM nginx:latest AS frontend
COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
RUN ln -sf /usr/share/zoneinfo/Europe/Istanbul /etc/localtime

# Copy FastAPI app to nginx image
COPY --from=backend /usr/local /usr/local
COPY --from=backend /app /app

# Çalıştırmak için supervisord veya bash script
COPY start.sh /start.sh
RUN chmod +x /start.sh

EXPOSE 8080

CMD ["/start.sh"]

start.sh:

#!/bin/bash

cd /app

# Start FastAPI in the background
#uvicorn main:app --workers=4 --host 0.0.0.0 --port 8000 &
# --workers=4 (uvicorn ve gunicorn)'da productionda işe yaramıyor hatta ters tepiyor. Çünkü prodda tek core var muhtemelen.
gunicorn main:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 &
FASTAPI_PID=$!

echo "Waiting for FastAPI to start..."
MAX_RETRIES=40
RETRY_COUNT=0

until curl -s http://127.0.0.1:8000/health > /dev/null 2>&1; do
  RETRY_COUNT=$((RETRY_COUNT+1))
  if [ $RETRY_COUNT -ge $MAX_RETRIES ]; then
    echo "Timed out waiting for FastAPI after 15 seconds. Starting nginx anyway."
    break
  fi
  echo "Waiting for FastAPI... ($RETRY_COUNT/$MAX_RETRIES)"
  sleep 0.25
done

if [ $RETRY_COUNT -lt $MAX_RETRIES ]; then
  echo "Everything is OK..."
fi

# Start nginx in the foreground
echo "Starting nginx..."
exec nginx -g 'daemon off;'