Shared memory error *only* when running on fly.io

1

Hi!

I’m seeing the following error only when running squid on fly.io:

FATAL: Ipc::Mem::Segment::create failed to shm_open(/squid-cf__metadata.shm): (13) Permission denied

Running the container in other environments doesn’t seem to elicit this issue. Observationally, it seems like fly.io is not respecting the USER setting in my container config. Is this correct? If so, is there some way to enforce that? If not, can you share some of the restrictions that you apply to running containers that may impact this?

Thanks!

2

Would you mind sharing your Dockerfile? We are respecting the USER directive.

We’ll have a look at the shared memory error. Permission error makes it sound like the user doesn’t have enough permissions.

There might be some subtlety to your image that were not handling right! If so, that’s definitely a bug.

3

Hi Jerome,

Thanks for the super-quick response! We are actually using packer to build our images rather than docker, but here is the relevant section:

locals { 
  squid_version = "4.6-1+deb10u4"
}

source "docker" "container" {
  image = "debian:buster-slim"
  commit = "true"
  changes = [
    "USER proxy",
    "WORKDIR /",
    "EXPOSE 3128 5000 10001 10080",
    "ENTRYPOINT [ \"/usr/local/bin/dumb-init\", \"/bin/bash\", \"-c\" ]",
    "CMD [ \"/usr/local/bin/run.sh\" ]",
  ]
}

build {
  sources = ["source.docker.container"]

  provisioner "shell" {
    inline = [
      "# install the packages we need",
      "apt-get update", 
      "DEBIAN_FRONTEND=noninteractive apt-get install -y squid=${local.squid_version} curl procps",
      "rm -rf /var/lib/apt/lists/*"
    ]
  }
  provisioner "file" {
    destination = "/etc/squid/squid.conf"
    source      = "squid.conf"
  }

  # add in our run script
  provisioner "file" {
    destination = "/usr/local/bin/run.sh"
    source      = "run.sh"
  }

  # add in our update_cert script 
  provisioner "file" {
    destination = "/usr/local/bin/update_cert.sh"
      source      = "update_cert.sh"
  }

 # we pull dumb-init from our remote storage
  provisioner "shell-local" {
    inline = [
      "curl https://assets.mirageid.com/dumb-init -o ./dumb-init"
    ]
  }
  provisioner "file" {
    destination = "/usr/local/bin/dumb-init"
    source      = "dumb-init"
    generated   = true
  }

  provisioner "shell" {
    inline = [
      "# post setup", 

      "# set the correct ownership on our various config files",
      "chmod 0755 /etc/squid/squid.conf", 
      "touch /etc/squid/fullchain.pem", "chown proxy /etc/squid/fullchain.pem",
      "touch /etc/squid/privkey.pem", "chmod 0640 /etc/squid/privkey.pem", "chown proxy /etc/squid/privkey.pem",
      "touch /etc/squid/squid_passwd", "chown proxy /etc/squid/squid_passwd",

      "rm /var/run", "mkdir /var/run", "chown proxy /var/run",
      "touch /var/run/squid.pid", "chown proxy /var/run/squid.pid",

      "# make our various files executable",
      "chmod +x /usr/local/bin/run.sh",
      "chmod +x /usr/local/bin/update_cert.sh",
			"chmod +x /usr/local/bin/dumb-init",
    ]
  }

# make sure to tag things first
  post-processor "docker-tag" {
    repository = "mirageid/squid"
    tags       = ["notcompressed"]
  }

  # compress our image down
  post-processor "shell-local" {
    inline = [ 
      # trigger our request against the container,  We run this in the background so we can start the minifcation.
      "curl --retry 100 --retry-delay 2 --retry-all-errors --proxy-digest --proxy-insecure --proxy https://localhost:5000 --proxy-user bob:bob123 https://ipapi.co/json &",
      "docker-slim build  mirageid/squid:notcompressed --publish-exposed-ports --env CERT_PRIVATE_KEY=\"$(cat test.key)\" --env PROXY_USER_NAME=bob --env PROXY_USER_TOKEN=bob123 --http-probe-retry-count 2 --http-probe-retry-wait 2 --include-shell --include-bin /usr/lib/squid/unlinkd --include-path /run/squid.pid --include-path /var/spool/squid --include-path /etc/group --tag mirageid/squid:compressed",
    ]
  }
  
}
4

Also, just to test, I tried uploading a container that wasn’t run through docker-slim to see if that would make a difference, but got the same error. Thanks!

5

I’ll try to reproduce tomorrow. Thanks for the additional details!

6

JFWIW: this seems to be a common issue with Squid in some configurations & environments.

Luckily: I can reproduce this with a Dockerfile that’s just FROM sameersbn/squid:3.5.27-2 (internal TCP port 3128). We should be able to find out what’s happening here pretty quickly.

7

@dan I’ve deployed a fix and rescheduled your app. Looks like it’s working now!

The problem was that /dev/shm was chmod 0755 when it should’ve been chmod 1777. Thanks for bringing this to our attention.

I’m not sure if there’s still a USER problem though. There shouldn’t be.

8

Wow! Thanks for the super-quick turnaround. This is indeed working now. Thanks!

1 Like