🤖 Migrating Docker-Compose containers to Fly.io apps using GPT-3

I used :robot: GPT-3 to convert a PHP site to use Fly.io, converting the database container and app container that runs a Docker PHP Apache image.

Here is the shared Playground preset](OpenAI API).

Notice how I have the prompt use cat, I show the model what things actually look like, and also have the model show, what it thinks the file should look like. Often the predictions were wrong, so I’d correct them. Once I ported one container, it was fairly straight forward to port another, and it usually guessed things fairly accurately. Another things to note is that I would review each GPT-3 suggestion line by line, and copy paste parts that seemed to be correct. I took some liberties, like the FTP command doesn’t really accept commands as arguments (at the moment!), so for the FTP commands I would first start by being logged in to the FTP shell and would then paste the commands. Carefully going through this could make it a useful basis for porting other Docker-Compose projects.

The syntax style I came up with uses remarks in tags to provide commentary and instructions to the model, and output tags to show the shell output (e.g results of running a cat command). Before I ported the system to Fly.io, I had previously ported the site from a real Debian system running on bare metal (as far as I know), to Docker-Compose as a migration target.

Here is a copy of the preset for reference:

<remark>Move a docker-compose system using php and mysql images to fly.io</remark>
cd /home/user/system_dev_shared/project_name_source/project_name_git
cat docker-compose.yml
<output>
version: '3.1'
services:
  php:
    image: system/php:apache
    ports:
      - 8989:80
    volumes:
      - ./src:/var/www/html/
      - ./apache2.conf:/etc/apache2/apache2.conf
  mysql:
    image: mysql:5.6
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: MYSQL_PASSWORD
      MYSQL_DATABASE: project_name
      MYSQL_USER: project_name
      MYSQL_PASSWORD: MYSQL_PASSWORD
    volumes:
      - ./database:/var/lib/mysql
</output>
<remark>First we setup a mysql images on fly.io</remark>
mkdir flyio_mysql
cd flyio_mysql/
fly launch
fly volumes create mysqldata --size 1
fly secrets set MYSQL_PASSWORD=MYSQL_PASSWORD MYSQL_ROOT_PASSWORD=MYSQL_PASSWORD
fly launch
nano fly.toml
cat fly.toml
<output>
app = "project_name-mysql"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[env]
  MYSQL_DATABASE = "project_name"
  MYSQL_USER = "project_name"

[mounts]
  destination = "/var/lib/mysql"
  source = "mysqldata"

[build]
  image = "mysql:5.6"
</output>
fly launch
<remark>Now we can try connecting to the mysql database locally</remark>
flyctl proxy 3306 &
<output>Proxying local port 3306 to remote [project_name-mysql.internal]:3306</output>
mysql -h 127.0.0.1 -u project_name -pMYSQL_PASSWORD project_name
<output>Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.51 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [project_name]> 
</output>
<remark>Success! We were able to connect to the new mysql instance on fly.io! Now we need to setup the apache php image too.</remark>
cd ..
mkdir flyio_web_server
cd flyio_web_server/
fly launch
fly volumes create webdata --size 1
nano fly.toml
cat fly.toml
<output>
app = "project_name-web"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[[services]]
  internal_port = 80
  protocol = "tcp"

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20

  [[services.ports]]
    handlers = ["http"]
    port = "80"

  [[services.ports]]
    handlers = ["tls", "http"]
    port = "443"

  [[services.tcp_checks]]
    interval = 10000
    timeout = 2000

[mounts]
  destination = "/var/www/html"
  source = "webdata"

[build]
  image = "system/php:apache"
</output>
fly launch
<remark>Great, now we have an empty php apache server deployed, let's copy the php files from the src directory above</remark>
cd ..
<remark>First we tar the src folder up so we can copy it with a single sftp command</remark>
tar -czf flyio_web_server/src.tar.gz src
</remark>Now we made the tar file, we can copy it<remark>
cd flyio_web_server
<remark>We use one sftp put command for the tar archive file</remark>
flyctl ssh sftp shell 'put src.tar.gz /var/www/html/src.tar.gz'
<remark>Now we extract the php files by opening up an ssh console on the fly app instance that has the volume mounted</remark>
flyctl ssh console
<remark>we go to the html source directory</remark>
cd /var/www/html
tar -xvzf src.tar.gz
<remark>done but we don't want the files in a subfolder so we move them up</remark>
mv src/* .
<remark>Done, let's exit the ssh session</remark>
exit
<remark>Now we need to restore the mysql server by copying may_2021.sql from the parent directory to the fly.io mysql server</remark>
cd ../flyio_mysql
flyctl ssh sftp shell 'put ../may_2021.sql /var/lib/mysql/may_2021.sql'
<remark>We ssh in and restore it</remark>
flyctl ssh console
<remark>We go to the mysql data directory</remark>
cd /var/lib/mysql
<remark>We restore the database</remark>
mysql -u project_name -pMYSQL_PASSWORD project_name < may_2021.sql
<remark>Done, let's exit the ssh session and point to the new DB in src/include.php</remark>
exit
cd ../flyio_web_server
<remark>Now we need to update the database connection details in the php files, we can do this directly over ssh on the server. We open up an ssh console on the fly app instance that has the volume mounted</remark>
flyctl ssh console
<remark>we go to the html source directory</remark>
cd /var/www/html
cat include.php| less
<output>
<?
define("project_name","project_name");

// Log into the database:
mysql_connect("mysql", "project_name", "MYSQL_PASSWORD");
</output>
<remark>We need to edit the include.php file to point to the new mysql server</remark>
sed -i 's/mysql_connect("mysql", "project_name", "MYSQL_PASSWORD")/mysql_connect("project_name-mysql.internal", "project_name", "MYSQL_PASSWORD")/g' include.php
cat include.php| less
<output>
<?
define("project_name","project_name");

// Log into the database:
mysql_connect("project_name-mysql.internal", "project_name", "MYSQL_PASSWORD");
</output>
<remark>Do the same for functions.inc and security.php</remark>
sed -i 's/mysql_connect("mysql", "project_name", "MYSQL_PASSWORD")/mysql_connect("project_name-mysql.internal", "project_name", "MYSQL_PASSWORD")/g' functions.inc
sed -i 's/mysql_connect("mysql", "project_name", "MYSQL_PASSWORD")/mysql_connect("project_name-mysql.internal", "project_name", "MYSQL_PASSWORD")/g' security.php
<remark>We replace localhost in delete-journey.php also, which was forgot before when running on docker-compose locally</remark>
sed -i 's/mysql_connect("localhost", "project_name", "MYSQL_PASSWORD")/mysql_connect("project_name-mysql.internal", "project_name", "MYSQL_PASSWORD")/g' delete-journey.php
<remark>Done, let's exit the ssh session</remark>
exit
<remark>Now we can test the new fly.io app</remark>
flyctl open
<remark>We can see the app is working except for using the custom error page, as we did not override /etc/apache2/apache2.conf with our version. We can copy the apache2.conf file over ssh</remark>
flyctl ssh sftp shell 'put ../apache2.conf /etc/apache2/apache2.conf'
<output>
put ../apache2.conf -> /etc/apache2/apache2.conf: file exists on VM
</output>
<remark>Oops, we need to soft delete the existing file first</remark>
flyctl ssh console
mv /etc/apache2/apache2.conf /etc/apache2/apache2.conf.old
exit
flyctl ssh sftp shell 'put ../apache2.conf /etc/apache2/apache2.conf'
<remark>Now we can test the new fly.io app</remark>
flyctl open
<remark>We can see the app is working except for using the custom error page, as we did not override /etc/apache2/apache2.conf with our version. We can copy the apache2.conf file over ssh</remark>
flyctl ssh sftp shell 'put ../apache2.conf /etc/apache2/apache2.conf'
<remark>This is all well and good but the server would need restarting to use the new config, but this will also loose any changes done that are not from a mounted volume so we need to do this in a different way. We can use the flyctl restart command to restart the app but it will loose any changes. So instead we can make a Dockerfile that makes the changes before the app starts</remark>
cp ../apache2.conf .
nano Dockerfile
cat Dockerfile
<output>
FROM system/php:apache
COPY apache2.conf /etc/apache2/apache2.conf
</output>
nano fly.toml
cat fly.toml
<output>
app = "project_name-web"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []

[[services]]
  internal_port = 80
  protocol = "tcp"

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20

  [[services.ports]]
    handlers = ["http"]
    port = "80"

  [[services.ports]]
    handlers = ["tls", "http"]
    port = "443"

  [[services.tcp_checks]]
    interval = 10000
    timeout = 2000

[mounts]
  destination = "/var/www/html"
  source = "webdata"

# [build]
#  image = "system/php:apache"
# Fly automatically will detect the Dockerfile and build it
</output>
fly deploy
<remark>Now we can test the new fly.io app</remark>

In the future I’d love to see a non-AI, good old logic based Python script that could parse some Docker-Compose YAML file and generate a Fly.io app config files for each service in the expected TOML format.
Hope this helps some people.

3 Likes