Watch your logs using a Machine + SQLite + (Optional) LiteFS for UI

I’ve created a simple way to store all your organization logs in a SQLite DB.

  • Longer than 2 days retention to logs.
  • Easy to query, its just SQLite.
  • (Optional) Sync to LiteFS Cloud so you can have backups and you can also query them anytime from our UI.
  • It only costs a shared-cpu-1x@256MB machine (~$1.94/mo) + Volume Size ($0.15/GB) + (Optional) LiteFS Cluster ($5 for 10GB + $0.5/extra GB).

Elixir source code: GitHub - lubien/fly-organization-logs-watcher

How to deploy

  1. Create a folder with this fly.toml

Feel free to change primary_region and change ORGANIZATION_SLUG to the slug you see when navigating

:warning: “personal” does not work as slug you need to find the real slug for your personal organization. Tip: go to the billing page on the dashboard and look at the URL

primary_region = 'gru'

  image = 'lubien/fly-organization-logs-watcher:v0.1.0'

  DATABASE_PATH = '/litefs/app_logs.db'

  source = 'db_data'
  destination = '/data'
  initial_size = '1GB'
  auto_extend_size_threshold = 80
  auto_extend_size_increment = '1'

  size = 'shared-cpu-1x'
  memory_mb = 256
  1. Dry-launch it

Change the app name to whatever you like.

fly launch --copy-config --auto-confirm --no-deploy --name lubien-logs
  1. (Optional) Go to LiteFS Cloud and create a new cluster

fly secrets set LITEFS_CLOUD_TOKEN="FlyV1 fm2_lJPEC..."
  1. Add a token to be able to watch logs
fly secrets set FLY_API_TOKEN="$(fly tokens create org YOUR_ORGANIZATION_SLUG)"
  1. Deploy
fly deploy --ha=false
  1. You’re done!

How to see my logs?

If you enabled LiteFS cloud, on our dashboard go to your [LiteFS Cloud] → Choose your cluster → Click on the ‘Explore’ button in the ‘app_logs.db’ row. Run:

select * from app_logs limit 10;

If you chose not to use LiteFS cloud:

$ fly ssh console
$ sqlite3 /litefs/app_logs.db "select * from app_logs limit 10"

I wanted to give this app a whirl but then I saw this:

Does that mean it won’t work if I only have a personal org?

It works, you just have to use the real slug for your personal org. Mine is “lubien” for example.

I’ve edited the post for clarity

1 Like