Hi,
I’m on the hobby plan for a simple rails API. I made a recent change (a migration to change a column type from string to integer) and now it can’t fulfill a simple GET request without running out of memory and falling over. I never had any issues like this even when the users (<20) used it last year.
2023-11-06T02:19:44.620 app[148ed726cdde58] ewr [info] [ 97.091862] Out of memory: Killed process 313 (ruby) total-vm:207868kB, anon-rss:86208kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:424kB oom_score_adj:0
I reverted the changes and I’m still getting a crash on memory error.
Is this the first “gotcha” on a free trial? How much does it cost to increase the memory to be able to run a simple rails app per month, and how do I pay for it?
the app currently has got a tiny rails app running, with a separate worker process, and it runs the occasional rake task.
you can also probably pare down your rails gems (it’s rare for small apps run into issues with it. discourse has a good example: define the needed gems then require the railties individually that’ll help a smidge. other http servers (I think thin, notably) are less memory hungry than puma, as well
Here are the logs after I tried to log in (I’m using rails as just an API, rendering JSON for my Vue frontend):
2023-11-08T01:56:55.976 app[148ed726cdde58] ewr [info] I, [2023-11-08T01:56:55.976123 #666] INFO -- : [03fa3866-0f88-4875-967b-2ec2e4487dd7] Started POST "/sessions" for 2a09:8280:1::1:8998 at 2023-11-08 01:56:55 +0000
2023-11-08T01:56:55.985 app[148ed726cdde58] ewr [info] I, [2023-11-08T01:56:55.985236 #666] INFO -- : [03fa3866-0f88-4875-967b-2ec2e4487dd7] Processing by SessionsController#create as HTML
2023-11-08T01:56:55.985 app[148ed726cdde58] ewr [info] I, [2023-11-08T01:56:55.985506 #666] INFO -- : [03fa3866-0f88-4875-967b-2ec2e4487dd7] Parameters: {"name"=>"david", "password"=>"[FILTERED]", "session"=>{"name"=>"david", "password"=>"[FILTERED]"}}
2023-11-08T01:57:05.355 app[148ed726cdde58] ewr [info] [77664.068387] Out of memory: Killed process 666 (ruby) total-vm:207868kB, anon-rss:85928kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:424kB oom_score_adj:0
2023-11-08T01:57:05.359 proxy[148ed726cdde58] ewr [error] could not complete HTTP request to instance: connection closed before message completed
2023-11-08T01:57:05.369 app[148ed726cdde58] ewr [info] [310] - Worker 0 (PID: 679) booted in 0.0s, phase: 0
Contents of my Dockerfile:
# syntax = docker/dockerfile:experimental
# Dockerfile used to build a deployable image for a Rails application.
# Adjust as required.
#
# Common adjustments you may need to make over time:
# * Modify version numbers for Ruby, Bundler, and other products.
# * Add library packages needed at build time for your gems, node modules.
# * Add deployment packages needed by your application
# * Add (often fake) secrets needed to compile your assets
#######################################################################
# Learn more about the chosen Ruby stack, Fullstaq Ruby, here:
# https://github.com/evilmartians/fullstaq-ruby-docker.
#
# We recommend using the highest patch level for better security and
# performance.
ARG RUBY_VERSION=3.0.2
ARG VARIANT=jemalloc-slim
FROM quay.io/evl.ms/fullstaq-ruby:${RUBY_VERSION}-${VARIANT} as base
LABEL fly_launch_runtime="rails"
ARG BUNDLER_VERSION=2.2.31
ARG RAILS_ENV=production
ENV RAILS_ENV=${RAILS_ENV}
ENV RAILS_SERVE_STATIC_FILES true
ENV RAILS_LOG_TO_STDOUT true
ARG BUNDLE_WITHOUT=development:test
ARG BUNDLE_PATH=vendor/bundle
ENV BUNDLE_PATH ${BUNDLE_PATH}
ENV BUNDLE_WITHOUT ${BUNDLE_WITHOUT}
RUN mkdir /app
WORKDIR /app
RUN mkdir -p tmp/pids
#######################################################################
# install packages only needed at build time
FROM base as build_deps
ARG BUILD_PACKAGES="git build-essential libpq-dev wget vim curl gzip xz-utils libsqlite3-dev"
ENV BUILD_PACKAGES ${BUILD_PACKAGES}
RUN --mount=type=cache,id=dev-apt-cache,sharing=locked,target=/var/cache/apt \
--mount=type=cache,id=dev-apt-lib,sharing=locked,target=/var/lib/apt \
apt-get update -qq && \
apt-get install --no-install-recommends -y ${BUILD_PACKAGES} \
&& rm -rf /var/lib/apt/lists /var/cache/apt/archives
#######################################################################
# install gems
FROM build_deps as gems
RUN gem update --system --no-document && \
gem install -N bundler -v ${BUNDLER_VERSION}
COPY Gemfile* ./
RUN bundle install && rm -rf vendor/bundle/ruby/*/cache
#######################################################################
# install deployment packages
FROM base
ARG DEPLOY_PACKAGES="postgresql-client file vim curl gzip libsqlite3-0"
ENV DEPLOY_PACKAGES=${DEPLOY_PACKAGES}
RUN --mount=type=cache,id=prod-apt-cache,sharing=locked,target=/var/cache/apt \
--mount=type=cache,id=prod-apt-lib,sharing=locked,target=/var/lib/apt \
apt-get update -qq && \
apt-get install --no-install-recommends -y \
${DEPLOY_PACKAGES} \
&& rm -rf /var/lib/apt/lists /var/cache/apt/archives
# copy installed gems
COPY --from=gems /app /app
COPY --from=gems /usr/lib/fullstaq-ruby/versions /usr/lib/fullstaq-ruby/versions
COPY --from=gems /usr/local/bundle /usr/local/bundle
#######################################################################
# Deploy your application
COPY . .
# Adjust binstubs to run on Linux and set current working directory
RUN chmod +x /app/bin/* && \
sed -i 's/ruby\r/ruby/' /app/bin/* && \
sed -i '/^#!/aDir.chdir File.expand_path("..", __dir__)' /app/bin/*
# The following enable assets to precompile on the build server. Adjust
# as necessary. If no combination works for you, see:
# https://fly.io/docs/rails/getting-started/existing/#access-to-environment-variables-at-build-time
ENV SECRET_KEY_BASE 1
# ENV AWS_ACCESS_KEY_ID=1
# ENV AWS_SECRET_ACCESS_KEY=1
# Run build task defined in lib/tasks/fly.rake
ARG BUILD_COMMAND="bin/rails fly:build"
RUN ${BUILD_COMMAND}
# Default server start instructions. Generally Overridden by fly.toml.
ENV PORT 8080
ARG SERVER_COMMAND="bin/rails fly:server"
ENV SERVER_COMMAND ${SERVER_COMMAND}
CMD ${SERVER_COMMAND}
I’m just baffled with how i’m having memory problems now
interesting, does fly.toml specify any CMD overrides (maybe the [processes] section)? and does RAILS_ROOT/lib/tasks/fly.rake have the server rake task firing the swapfile rake task?
Don’t think I have any CMD overrides, here’s my fly.toml:
# fly.toml app configuration file generated for {nameofmyapp} on 2023-08-20T10:50:34-05:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = "{nameofmyapp}"
primary_region = "ewr"
kill_signal = "SIGINT"
kill_timeout = "5s"
[experimental]
auto_rollback = true
[build]
[build.args]
BUILD_COMMAND = "bin/rails fly:build"
SERVER_COMMAND = "bin/rails fly:server"
[deploy]
release_command = "bin/rails fly:release"
[env]
PORT = "8080"
[[services]]
protocol = "tcp"
internal_port = 8080
processes = ["app"]
[[services.ports]]
port = 80
handlers = ["http"]
force_https = true
[[services.ports]]
port = 443
handlers = ["tls", "http"]
[services.concurrency]
type = "connections"
hard_limit = 25
soft_limit = 20
[[services.tcp_checks]]
interval = "15s"
timeout = "2s"
grace_period = "1s"
[[statics]]
guest_path = "/app/public"
url_prefix = "/"
fly.rake:
namespace :fly do
# commands used to deploy a Rails application
task :build => 'assets:precompile'
task :release => 'db:migrate'
task :server do
sh 'bin/rails server'
end
# commands useful on the development machine
task :ssh do
sh 'fly ssh console'
end
task :console do
sh 'fly ssh console -C "/home/app/showcase/bin/rails console"'
end
task :dbconsole do
sh 'fly ssh console -C "/home/app/showcase/bin/rails dbconsole"'
end
end
Pretty sure these are just the standard settings, I masked the name of my app on this post for privacy’s sake, but I doubt anyone would care lol.