Hey, I’m having a hard time deciding on how the discrete parts of my application should be tied together.
The three components are:
- I have a group of AWS Lamba’s that process inbound email (using SES). They essentially just parse/validate the raw inbound mail.
- I need a RDBMS (Postgres) to store the parsed mail in it’s entirety.
- I need API servers that can be accessed through client applications.
I want to use Fly for both the API server and the database but I’m not sure what’s the best way to work with AWS Lambda.
Things I’ve considered,
Running PgBouncer on Fly and then connecting Fly and AWS Lambda using something like Tailscale. See Tailscale on AWS Lambda and Tailscale on Fly. I don’t really want to do this because it adds a lot of complexity to my existing setup.
Have AWS Lambda call a webhook exposed by the API server on Fly. So the Fly server would be responsible for writing to the database. I could go the Tailscale route but I could also just use an authenticated public webhook endpoint. This option has fewer components but it makes provisioning/reliability more difficult to manage. AWS Lambda can scale pretty high and I’m afraid my costs could potentially sky-rocket. I could introduce a buffer (like NATS or something similar?) but that too adds another component I need to manage.
Have the primary database be outside of Fly. Using something like AWS RDS (with the RDS Proxy) or Supabase allows AWS Lambda to perform writes to the DB. Then to continue to take advantage of the API servers on Fly I could setup globally distributed redis caches on Fly. (I’m unsure if I can setup postgres read replicas inside Fly with the primary being outside, at least not easily.) I think this option would have the least infrastructure management but I don’t know how effectively I’d be able to use the caches.
Is there a way I can simplify my setup? Is Option 3 something you would recommend?