Litestream replication from one fly instance to another using SFTP

1

Hello

I have been using s3 backend for replication of sqlite databases using litestream. I do not need long term storage for this data. These are ephemeral databases that need to exist till the server is running. In the event of an occasional crash, the server should be able to restore its state. Once the task is done, this database is no longer needed.

S3 seems to be overkill for this, it anyway costs a lot because of the request charges. Since Litestream supports sftp, I was thinking about setting up a replication server within fly.io.

Could there be a good way to do this using fly’s internal networking. I am trying to figure out how one machine can connect to another designated machine (on the ipv6 address perhaps) using sftp without using the flyctl wrapper.

Just trying to find the best way to configure networking so that litestream could make the connection.

Thanks!

2

(apologies for a non-answer reply)

Haven’t experimented with SFTP between machines, but the flyctl code may have pointers for you.

Why not consider deploying LiteFS instead? Or, wait for a managed LiteFS offering, which should be right around the corner.

There’s turso.tech and Cloudflare D1 if you want a managed sqlite3 based database right away.

Fly 6pn works out of the box. There’s nothing to configure, as such. Listen for incoming requests on fly-local-6pn and the server should be able see requests over the private network setup by Fly. Ref: Can't get a CouchDB Cluster working (connection_closed) - #4 by ignoramous

3

I did look at LiteFS, but for my use-case, only litestream would work I think. Each machine has multiple databases, and each database can independently get created and destroyed (once the task is done). These are game servers basically, and each game server can host multiple games.

turso.tech and cloudflare also feel like overkill for what i need !

My basic sys admin self tried this -
Make a Docker container that has openssh server running, and expose its port and use it to connect. I was able to do that with a basic image that looks like this

# reference: https://github.com/arvindr226/alpine-ssh/blob/master/Dockerfile
FROM alpine:3.18

# Installing the openssh and bash package, removing the apk cache
# hard-coding password for now
RUN apk --update add --no-cache openssh bash \
  && sed -i s/#PermitRootLogin.*/PermitRootLogin\ yes/ /etc/ssh/sshd_config \
  && echo "root:root" | chpasswd \
  && rm -rf /var/cache/apk/*

# Defining the Port 2222 for service
# Fly doesn't seem to allow binding to port 22
RUN sed -ie 's/#Port 22/Port 2222/g' /etc/ssh/sshd_config

RUN /usr/bin/ssh-keygen -A
RUN ssh-keygen -t rsa -b 4096 -f  /etc/ssh/ssh_host_key

ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile

EXPOSE 2222
CMD ["/usr/sbin/sshd", "-D"]

I can connect from another fly machine to this one using the internal IPv6 address. Or even the internal DNS name using

ssh root@<machine-id>.vm.<app>.internal -p 2222

This works! Not sure if there’s a more elegant way (anyone?).

Leaving it here in case it helps someone.

Cheers

1 Like
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.