Cloud Voicemail: self-hosted voicemail on Fly with whisper.cpp

Show & Tell
1

I built a self-hosted voicemail box that runs on Fly, and it’s now open source (MIT): GitHub - ryanshipswell/cloud-voicemail: Self-hosted voicemail for a Twilio number: recorded greetings for contacts, whisper.cpp transcription, spam flagging, and email delivery, deployed on Fly.io. · GitHub

What it does

  • Answers calls to a Twilio number. Blocked callers are rejected, contacts hear a recorded greeting, and everyone else hears a generic one.
  • Transcribes each message with whisper.cpp, flags likely spam from unknown callers using Cloudflare’s Clef model, and emails me the transcript with the recording attached. Then it deletes the recording from Twilio and from the app.
  • Skips the email when someone hangs up without saying anything.

I use it as the voicemail for my regular number. My carrier’s conditional call forwarding sends rejected and unanswered calls to the Twilio number, and my phone rejects calls from anyone who isn’t a contact.

How it runs on Fly

  • cloud-voicemail is a Laravel app on FrankenPHP. The queue worker runs beside it on the same Machine, with SQLite on a volume, so the whole app is one small shared-CPU Machine.
  • cloud-voicemail-transcriber is whisper.cpp behind a tiny Python server with no public IP. The Laravel app calls it over Flycast, so Fly’s proxy starts it when a voicemail comes in and stops it when it’s idle. It only runs while it’s transcribing.
  • GitHub Actions deploys whichever app changed on each push to main.

The README covers the full setup. Feedback and PRs are welcome!