ERROR Invalid HTTP_HOST header: '[IP]:8000'. You may need to add '[IP]' to ALLOWED_HOSTS

Django
1

Hi there,

I am trying Fly.io for the first time. I have a Django RESTFramework app that I’m trying to deploy. I have read the Django getting started doc, but I can’t seem to get the app to boot. The app runs fine on my laptop with this command:
DJANGO_ENV=live docker compose up --build web

When I look at the logs, I see this networking error:

2024-11-19T04:31:43Z app[6e825de6f69278] lax [info]Running migrations:
2024-11-19T04:31:43Z app[6e825de6f69278] lax [info]  No migrations to apply.
2024-11-19T04:31:45Z proxy[6e825de6f69278] lax [info]waiting for machine to be reachable on 0.0.0.0:8000 (waited 5.279791088s so far)
2024-11-19T04:31:47Z app[6e825de6f69278] lax [info]Superuser already exists
2024-11-19T04:31:48Z proxy[6e825de6f69278] lax [error][PM05] failed to connect to machine: gave up after 15 attempts (in 8.285671373s)
2024-11-19T04:31:57Z app[6e825de6f69278] lax [info]0 static files copied, 195 unmodified.
2024-11-19T04:31:59Z app[6e825de6f69278] lax [info][2024-11-18 20:31:59 -0800] [322] [INFO] Starting gunicorn 20.1.0
2024-11-19T04:31:59Z app[6e825de6f69278] lax [info][2024-11-18 20:31:59 -0800] [322] [INFO] Listening at: http://0.0.0.0:8000 (322)
2024-11-19T04:31:59Z app[6e825de6f69278] lax [info][2024-11-18 20:31:59 -0800] [322] [INFO] Using worker: sync
2024-11-19T04:31:59Z app[6e825de6f69278] lax [info][2024-11-18 20:31:59 -0800] [333] [INFO] Booting worker with pid: 333
2024-11-19T04:31:59Z app[6e825de6f69278] lax [info][2024-11-18 20:31:59 -0800] [334] [INFO] Booting worker with pid: 334
2024-11-19T04:32:08Z app[6e825de6f69278] lax [info]ERROR Invalid HTTP_HOST header: '172.19.17.178:8000'. You may need to add '172.19.17.178' to ALLOWED_HOSTS.
2024-11-19T04:32:08Z app[6e825de6f69278] lax [info]172.19.17.177 - - [18/Nov/2024:20:32:08 -0800] "GET /health/ HTTP/1.1" 400 143 "-" "Consul Health Check"
2024-11-19T04:32:11Z proxy[6e825de6f69278] lax [error][PR03] could not find a good candidate within 21 attempts at load balancing. last error: [PR01] no known healthy instances found for route tcp/443. (hint: is your app shut down? is there an ongoing deployment with a volume or are you using the 'immediate' strategy? have your app's instances all reached their hard limit?)

I already updated my settings.py as follows:

APP_NAME = os.environ.get("FLY_APP_NAME")
ALLOWED_HOSTS = [
    "0.0.0.0",
    "127.0.0.1",
    "localhost",
    f"{APP_NAME}.fly.dev",
]
CSRF_TRUSTED_ORIGINS = [
    f"{APP_NAME}.fly.dev",
]

Dockerfile:

FROM python:3.11-slim

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    postgresql-client \
    dos2unix \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy project
COPY . .

# Make the entrypoint script executable and fix line endings
COPY docker-entrypoint.sh /docker-entrypoint.sh
RUN dos2unix /docker-entrypoint.sh && \
    chmod +x /docker-entrypoint.sh && \
    rm -rf /var/lib/apt/lists/*

# Expose port 8000
EXPOSE 8000

ENTRYPOINT ["/docker-entrypoint.sh"]

docker-entrypoint.sh:

#!/bin/bash

# Function to wait for postgres
wait_for_postgres() {
    echo "Waiting for postgres..."
    local retries=30
    while ! pg_isready -h "$FIBER_SERVER_POSTGRES_HOST" -p "$FIBER_SERVER_POSTGRES_PORT" -U "$FIBER_SERVER_POSTGRES_USER" && [ $retries -gt 0 ]; do
        echo "Postgres is unavailable - sleeping (${retries} retries left)"
        retries=$((retries-1))
        sleep 2
    done

    if [ $retries -eq 0 ]; then
        echo "Failed to connect to Postgres"
        exit 1
    fi
    echo "PostgreSQL started"
}

# Wait for postgres
wait_for_postgres

# Run migrations
python manage.py migrate --noinput

# Create superuser if it doesn't exist
python manage.py shell << EOF
from django.contrib.auth import get_user_model
User = get_user_model()
if not User.objects.filter(email='[MYEMAIL]').exists():
    User.objects.create_superuser(username='admin', email='[MYEMAIL]', password='[PASSWORD]')
    print('Superuser created successfully')
else:
    print('Superuser already exists')
EOF

# Collect static files
python manage.py collectstatic --noinput

# Start Gunicorn
echo "Starting Gunicorn..."
exec gunicorn fiber_server.wsgi:application --config ./gunicorn.conf.py

## Start with runserver instead of gunicorn
#python manage.py runserver 0.0.0.0:8000

And fly.toml:

# fly.toml app configuration file generated for fiber-server-wild-rain-4122 on 2024-11-17T07:44:50-08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = 'fiber-server-wild-rain-4122'
primary_region = 'lax'
console_command = '/code/manage.py shell'

[build]

[deploy]
  release_command = 'python manage.py migrate --noinput'

[env]
  PORT = '8000'

[http_service]
  internal_port = 8000
  force_https = true
  auto_stop_machines = 'stop'
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[http_service.checks]]
  grace_period = "120s"
  interval = "60s"
  method = "GET"
  timeout = "30s"
  path = "/health/"

[[vm]]
  memory = '1gb'
  cpu_kind = 'shared'
  cpus = 1

#[[statics]]
#  guest_path = '/code/static'
#  url_prefix = '/static/'

I must be missing something simple, but I can’t seem to find it. Does anyone know what could be wrong?

2

Hi… It looks like these might just be the health checks. (Mine come in with numeric Host headers, as well.)

You might try temporarily commenting out the [[http_service.checks]] block—and see whether that gets you off the ground at all.

Hope this helps a little!

3

Thanks! Unfortunately, I had already tried that. I wrote some basic tutorial django apps and was able to deploy them on the first try to fly, so I think this is something weird about the specific combination of dependencies I have with my app (?). Perhaps something about the old version of django that I’m running not being compatible with fly? I did see in my search that some people fixed their issues by just upgrading their django and gunicorn, so I’ll give that a try as well.

4

Ok, I figured it out.

This is what I did.

  1. I reproduced the error by creating a brand new django app. I was able to deploy it just fine, but started failing as soon as I added the service http_service.checks.
  2. Updated my fly.toml like so:
[[http_service.checks]]
  grace_period = "120s"
  interval = "30s"
  method = "GET"
  timeout = "5s"
  path = "/health/"

and my ALLOWED_HOSTS like so:

ALLOWED_HOSTS = [
    "0.0.0.0",
    f"{APP_NAME}.fly.dev",
    "172.19.*.*",  # Entire subnet for Fly.io internal IPs
]

This made it work. It’s not great. I don’t like adding code specific to Fly to my app. But meh. If you know of a better solution, let me know.

1 Like
5

Also, those details might change in the future, :dragon:…

I was able to convince the health-check subsystem to instead use a custom Host header in its requests, via some rather eye-bending syntax:

[[http_service.checks]]
  interval = "5s"  # intervals >5s give bad results, in my experience.
  timeout = "5s"
  grace_period = "10s"
  path = "/petrichor"
  [http_service.checks.headers]
    Host = "03-02-pounce-api.fly.dev"

(Note that the final block is [ instead of the more obvious [[.)

This might not be precisely the solution in your case, but perhaps something along those lines…

6

Awesome. I can confirm that manually setting the service check Host to the public Fly Hostname worked for me too. This allows me to remove the settings.py workaround.

Dang, this wasn’t in any documentation I found! Thanks for the help!