Sunsetting Nomad

@jaimeiniesta Can you please try updating your postgres image? fly image update -a <appname>

@sarawoods Your app was also misidentified as postgres. I’ve flipped the switch, you should be able to migrate!

1 Like

Done! But the migrate-to-v2 still fails with the same error message after that.

Hi i’m getting an error trying to update my postgres apps. It says the following

Creatings 2 new volume(s) in ‘fra’
failed while migrating: failed to create volume: Some of our regions require a paid plan, due to high demand (Regions · Fly Docs). You must have a paid plan to scale up in fra. Setup a plan here:

The problem is that i’m already paying and have a credit card associated to my account but i don’t want to choose a plan another plan, i prefer to keep paying as i go using the hobby plan since those are hobby projects anyway.

What should i do?

Hi I am trying to migrate my fly postgres instances to v2 (dronda-pg). When I run the command migrate-to-v2 it fails saying: Error: no instances found with leader role.

However when I type fly status, I see one of the instances is marked as a leader.

Any advice on how I should proceed? Thank you!

Hi,

I am getting the following when trying to upgrade my mysql instance:

? Would you like to continue? Yes
==> Migrating course-management-mysql to the V2 platform

Locking app to prevent changes during the migration
Enabling machine creation on app
Creating an app release to register this migration
Starting machines
INFO Using wait timeout: 5m0s lease timeout: 13s delay between lease refreshes: 4s

Updating existing machines in ‘course-management-mysql’ with rolling strategy
Machine 6e82d61ef71d18 [app] has state: started
[1/1] Checking that 6e82d61ef71d18 [app] is up and running
Smoke checks for 6e82d61ef71d18 failed: the app appears to be crashing
Check its logs: here’s the last lines below, or run ‘fly logs -i 6e82d61ef71d18’:
failed while migrating: smoke checks for 6e82d61ef71d18 failed: the app appears to be crashing
? Would you like to enter interactive troubleshooting mode? If not, the migration will be rolled back. (Y/n)

@quentin Unfortunately, that region requires a plan. Note that the plans come with usage equivalent to the price, so if you pay over $29/mo there’s no downside to picking the Launch plan.
If you don’t want that (plenty of valid reasons to not want that!), you can find another region nearby and deploy to that instead.

We currently have remote volume forking disabled in flyctl, but I think we should be able to reenable that now. I’m going to check on that, because that would make moving things over to another region much easier.

@dronda Postgres apps have been difficult to migrate lately, there’s a regression in the migration command. I’ve been trying to debug this, but it’s proving difficult. Hang tight, we’ll update you when we have news to share.

@wisdom-b Your mysql instance seems to be crashing on boot. If you can, it might be helpful to try running it with the --verbose flag. If it’s an option, you could also try making a backup of your data (fly vol list to see volumes, and then fly vol fork to get a backup of the current volume), then upgrade the version of mysql in use?

1 Like

@allison I got another email today saying “We’re sending this to you because you have an app that is still running on Nomad.” Once again I tried running the suggested command. Scrubbing my app name, this is what I see:

~$: fly migrate-to-v2                                                                                                                                              
Error: the app 'my-app-1234' is already on the apps v2 platform

Once again I checked the status with fly status -a my-app-1234:

App
  Name     = my-app-1234
  Owner    = personal
  Hostname = my-app-1234.fly.dev
  Image    = my-app-1234:deployment-01ABC1A12ABCD1A12ABCDEFGHI
  Platform = machines

Machines
PROCESS ID              VERSION REGION  STATE   ROLE    CHECKS                  LAST UPDATED
app     1234a1a1a12345  27      iad     started         1 total, 1 passing      2023-05-26T01:36:31Z

It looks like I’m off Nomad, right? If so, why am I getting emails? Is there something else I need to do?

Hi!

I received an email saying that my postgres app needs to be migrated to v2 but haven’t been able to do that using the CLI and the web CLI.

For example this is the output I see in the web CLI:

/app/bin $ fly config save -a gacha-list-db
Wrote config file fly.toml
/app/bin $ fly migrate-to-v2
Error: 404: 404 page not found

It looks like you might have a different app still running on Nomad? If you run fly apps list, you should see an app running on Nomad still.

Can you run fly image update -a gacha-list-db? The migration tool requires some features that only exist in more recent Postgres images.

Yes I did run it a while ago. This is the output when I run it again:

fly image update -a gacha-list-db
Error: image is already running the latest image

If you’re running on the latest flyctl (fly version update), then I’m not sure exactly why it’s failing.

Would you feel comfortable posting the output of LOG_LEVEL=debug fly migrate-to-v2 in this thread?

flyctl is on the latest version

fly version update
Already running latest flyctl v0.1.104

And here the output for LOG_LEVEL=debug fly migrate-to-v2

that’s for polished-firefly-4927, not gacha-list-db.

Sorry I copied the command as you wrote above.

Running LOG_LEVEL=debug fly migrate-to-v2 -a gacha-list-db just gives me the error Error: unknown shorthand flag: 'a' in -a

When I create a new config file for gacha-list-db and run the first command I get this output

LOG_LEVEL=debug fly migrate-to-v2
DEBUG Loaded flyctl config from/Users/miguel/.fly/config.yml
DEBUG determined hostname: "Miguels-MacBook-Pro-3.local"
DEBUG determined working directory: "/Users/miguel/Developer"
DEBUG determined user home directory: "/Users/miguel"
DEBUG determined config directory: "/Users/miguel/.fly"
DEBUG ensured config directory exists.
DEBUG ensured config directory perms.
DEBUG cache loaded.
DEBUG config initialized.
DEBUG skipped querying for new release
DEBUG client initialized.
DEBUG app config loaded from /Users/miguel/Developer/fly.toml
DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query ($appName: String!) { appbasic:app(name: $appName) { id name platformVersion organization { id slug paidPlan } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (174.58ms)

DEBUG {
  "data": {
    "appbasic": {
      "id": "gacha-list-db",
      "name": "gacha-list-db",
      "platformVersion": "nomad",
      "organization": {
        "id": "0wqD28AMRNq3jtMxM5b6m9yb6oTQXA",
        "slug": "personal",
        "paidPlan": false
      }
    }
  }
}

DEBUG Starting task manager
DEBUG Config has metrics token

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed status appUrl platformVersion organization { id slug paidPlan } postgresAppRole: role { name } imageDetails { repository version } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query { platform { requestRegion regions { name code latitude longitude gatewayAvailable requiresPaidPlan } } }",
  "variables": null
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (158.86ms)

DEBUG {
  "data": {
    "platform": {
      "requestRegion": "mad",
      "regions": [
        {
          "name": "Amsterdam, Netherlands",
          "code": "ams",
          "latitude": 52.374342,
          "longitude": 4.895439,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Stockholm, Sweden",
          "code": "arn",
          "latitude": 59.6512,
          "longitude": 17.9178,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Atlanta, Georgia (US)",
          "code": "atl",
          "latitude": 33.6407,
          "longitude": -84.4277,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Bogotá, Colombia",
          "code": "bog",
          "latitude": 4.70159,
          "longitude": -74.1469,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Mumbai, India",
          "code": "bom",
          "latitude": 19.097403,
          "longitude": 72.874245,
          "gatewayAvailable": false,
          "requiresPaidPlan": true
        },
        {
          "name": "Boston, Massachusetts (US)",
          "code": "bos",
          "latitude": 42.366978,
          "longitude": -71.022362,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Paris, France",
          "code": "cdg",
          "latitude": 48.860875,
          "longitude": 2.353477,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Denver, Colorado (US)",
          "code": "den",
          "latitude": 39.7392,
          "longitude": -104.9847,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Dallas, Texas (US)",
          "code": "dfw",
          "latitude": 32.778287,
          "longitude": -96.7984,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Secaucus, NJ (US)",
          "code": "ewr",
          "latitude": 40.789543,
          "longitude": -74.056534,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Ezeiza, Argentina",
          "code": "eze",
          "latitude": -34.8222,
          "longitude": -58.5358,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Frankfurt, Germany",
          "code": "fra",
          "latitude": 50.1167,
          "longitude": 8.6833,
          "gatewayAvailable": true,
          "requiresPaidPlan": true
        },
        {
          "name": "Guadalajara, Mexico",
          "code": "gdl",
          "latitude": 20.5217,
          "longitude": -103.3109,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Rio de Janeiro, Brazil",
          "code": "gig",
          "latitude": -22.8099,
          "longitude": -43.2505,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Sao Paulo, Brazil",
          "code": "gru",
          "latitude": -23.549664,
          "longitude": -46.654351,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Hong Kong, Hong Kong",
          "code": "hkg",
          "latitude": 22.250971,
          "longitude": 114.203224,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Ashburn, Virginia (US)",
          "code": "iad",
          "latitude": 39.02214,
          "longitude": -77.462556,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Johannesburg, South Africa",
          "code": "jnb",
          "latitude": -26.13629,
          "longitude": 28.20298,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Los Angeles, California (US)",
          "code": "lax",
          "latitude": 33.9416,
          "longitude": -118.4085,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "London, United Kingdom",
          "code": "lhr",
          "latitude": 51.516434,
          "longitude": -0.125656,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Chennai (Madras), India",
          "code": "maa",
          "latitude": 13.064429,
          "longitude": 80.253067,
          "gatewayAvailable": true,
          "requiresPaidPlan": true
        },
        {
          "name": "Madrid, Spain",
          "code": "mad",
          "latitude": 40.4381,
          "longitude": -3.82,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Miami, Florida (US)",
          "code": "mia",
          "latitude": 25.7877,
          "longitude": -80.2241,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Tokyo, Japan",
          "code": "nrt",
          "latitude": 35.621608,
          "longitude": 139.741851,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Chicago, Illinois (US)",
          "code": "ord",
          "latitude": 41.891544,
          "longitude": -87.630386,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Bucharest, Romania",
          "code": "otp",
          "latitude": 44.4325,
          "longitude": 26.1039,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Phoenix, Arizona (US)",
          "code": "phx",
          "latitude": 33.416084,
          "longitude": -112.009482,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Querétaro, Mexico",
          "code": "qro",
          "latitude": 20.62,
          "longitude": -100.1863,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Santiago, Chile",
          "code": "scl",
          "latitude": -33.36572,
          "longitude": -70.64292,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Seattle, Washington (US)",
          "code": "sea",
          "latitude": 47.6097,
          "longitude": -122.3331,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Singapore, Singapore",
          "code": "sin",
          "latitude": 1.3,
          "longitude": 103.8,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "San Jose, California (US)",
          "code": "sjc",
          "latitude": 37.351601,
          "longitude": -121.896744,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Sydney, Australia",
          "code": "syd",
          "latitude": -33.866033,
          "longitude": 151.20693,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        },
        {
          "name": "Warsaw, Poland",
          "code": "waw",
          "latitude": 52.1657,
          "longitude": 20.9671,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Montreal, Canada",
          "code": "yul",
          "latitude": 45.48647,
          "longitude": -73.75549,
          "gatewayAvailable": false,
          "requiresPaidPlan": false
        },
        {
          "name": "Toronto, Canada",
          "code": "yyz",
          "latitude": 43.644632,
          "longitude": -79.384228,
          "gatewayAvailable": true,
          "requiresPaidPlan": false
        }
      ]
    }
  }
}

DEBUG <-- 200 https://api.fly.io/graphql (199.41ms)

DEBUG {
  "data": {
    "appcompact": {
      "id": "gacha-list-db",
      "name": "gacha-list-db",
      "hostname": "gacha-list-db.fly.dev",
      "deployed": true,
      "appUrl": null,
      "platformVersion": "nomad",
      "organization": {
        "id": "0wqD28AMRNq3jtMxM5b6m9yb6oTQXA",
        "slug": "personal",
        "paidPlan": false
      },
      "postgresAppRole": {
        "name": "postgres_cluster"
      },
      "imageDetails": {
        "repository": "flyio/postgres",
        "version": "v0.0.41"
      },
      "status": "running"
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "mutation($input: ValidateWireGuardPeersInput!) { validateWireGuardPeers(input: $input) { invalidPeerIps } }",
  "variables": {
    "input": {
      "peerIps": [
        "fdaa:0:345e:a7b:1596:0:a:102",
        "fdaa:0:565f:a7b:8dd7:0:a:2"
      ]
    }
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (121.29ms)

DEBUG {
  "data": {
    "validateWireGuardPeers": {
      "invalidPeerIps": []
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed status appUrl platformVersion organization { id slug paidPlan } postgresAppRole: role { name } imageDetails { repository version } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (630.22ms)

DEBUG {
  "data": {
    "appcompact": {
      "id": "gacha-list-db",
      "name": "gacha-list-db",
      "hostname": "gacha-list-db.fly.dev",
      "deployed": true,
      "appUrl": null,
      "platformVersion": "nomad",
      "organization": {
        "id": "0wqD28AMRNq3jtMxM5b6m9yb6oTQXA",
        "slug": "personal",
        "paidPlan": false
      },
      "postgresAppRole": {
        "name": "postgres_cluster"
      },
      "imageDetails": {
        "repository": "flyio/postgres",
        "version": "v0.0.41"
      },
      "status": "running"
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query ($appName: String!) { app(name: $appName) { id name hostname deployed status version appUrl platformVersion currentRelease { evaluationId status inProgress version } config { definition } organization { id slug paidPlan } services { description protocol internalPort ports { port handlers } } ipAddresses { nodes { id address type createdAt } } imageDetails { registry repository tag digest version } machines{ nodes { id name config state region createdAt app { name } ips { nodes { family kind ip maskSize } } host { id } } } postgresAppRole: role { name } limitedAccessTokens { nodes { id name expiresAt } } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (283.66ms)

DEBUG {
  "data": {
    "app": {
      "id": "gacha-list-db",
      "name": "gacha-list-db",
      "hostname": "gacha-list-db.fly.dev",
      "deployed": true,
      "version": 2,
      "appUrl": null,
      "platformVersion": "nomad",
      "currentRelease": {
        "evaluationId": null,
        "status": "succeeded",
        "inProgress": false,
        "version": 2
      },
      "config": {
        "definition": {
          "kill_timeout": 300,
          "kill_signal": "SIGTERM",
          "processes": [],
          "mounts": [
            {
              "processes": [],
              "source": "pg_data",
              "destination": "/data",
              "encrypted": false
            }
          ],
          "experimental": {
            "entrypoint": [],
            "cmd": [],
            "exec": [],
            "auto_rollback": false,
            "enable_consul": true,
            "private_network": true
          },
          "services": [],
          "checks": {
            "role": {
              "interval": "15s",
              "timeout": "10s",
              "grace_period": "30s",
              "restart_limit": 0,
              "port": 5500,
              "method": "get",
              "protocol": "http",
              "path": "/flycheck/role",
              "tls_skip_verify": false,
              "headers": [],
              "type": "http"
            },
            "pg": {
              "interval": "15s",
              "timeout": "10s",
              "grace_period": "30s",
              "restart_limit": 0,
              "port": 5500,
              "method": "get",
              "protocol": "http",
              "path": "/flycheck/pg",
              "tls_skip_verify": false,
              "headers": [],
              "type": "http"
            },
            "vm": {
              "interval": "1m",
              "timeout": "10s",
              "grace_period": "1s",
              "restart_limit": 0,
              "port": 5500,
              "method": "get",
              "protocol": "http",
              "path": "/flycheck/vm",
              "tls_skip_verify": false,
              "headers": [],
              "type": "http"
            }
          },
          "env": {
            "PRIMARY_REGION": "cdg",
            "BACKEND_STORE": "consul"
          },
          "metrics": {
            "port": 9187,
            "path": "/metrics"
          }
        }
      },
      "organization": {
        "id": "0wqD28AMRNq3jtMxM5b6m9yb6oTQXA",
        "slug": "personal",
        "paidPlan": false
      },
      "services": [],
      "imageDetails": {
        "registry": "registry-1.docker.io",
        "repository": "flyio/postgres",
        "tag": "13.9",
        "digest": "sha256:8603bc607763de576153b2198c86d847a7c63ee5f6524513a44ceb2458363440",
        "version": "v0.0.41"
      },
      "postgresAppRole": {
        "name": "postgres_cluster"
      },
      "ipAddresses": {
        "nodes": []
      },
      "machines": {
        "nodes": []
      },
      "limitedAccessTokens": {
        "nodes": []
      },
      "status": "running"
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query ($appName: String!) { appcompact:app(name: $appName) { id name hostname deployed status appUrl platformVersion organization { id slug paidPlan } postgresAppRole: role { name } imageDetails { repository version } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (298.92ms)

DEBUG {
  "data": {
    "appcompact": {
      "id": "gacha-list-db",
      "name": "gacha-list-db",
      "hostname": "gacha-list-db.fly.dev",
      "deployed": true,
      "appUrl": null,
      "platformVersion": "nomad",
      "organization": {
        "id": "0wqD28AMRNq3jtMxM5b6m9yb6oTQXA",
        "slug": "personal",
        "paidPlan": false
      },
      "postgresAppRole": {
        "name": "postgres_cluster"
      },
      "imageDetails": {
        "repository": "flyio/postgres",
        "version": "v0.0.41"
      },
      "status": "running"
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query($appName: String!) { app(name: $appName) { config { definition } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (157.41ms)

DEBUG {
  "data": {
    "app": {
      "config": {
        "definition": {
          "kill_timeout": 300,
          "kill_signal": "SIGTERM",
          "processes": [],
          "mounts": [
            {
              "processes": [],
              "source": "pg_data",
              "destination": "/data",
              "encrypted": false
            }
          ],
          "experimental": {
            "entrypoint": [],
            "cmd": [],
            "exec": [],
            "auto_rollback": false,
            "enable_consul": true,
            "private_network": true
          },
          "services": [],
          "checks": {
            "role": {
              "interval": "15s",
              "timeout": "10s",
              "grace_period": "30s",
              "restart_limit": 0,
              "port": 5500,
              "method": "get",
              "protocol": "http",
              "path": "/flycheck/role",
              "tls_skip_verify": false,
              "headers": [],
              "type": "http"
            },
            "pg": {
              "interval": "15s",
              "timeout": "10s",
              "grace_period": "30s",
              "restart_limit": 0,
              "port": 5500,
              "method": "get",
              "protocol": "http",
              "path": "/flycheck/pg",
              "tls_skip_verify": false,
              "headers": [],
              "type": "http"
            },
            "vm": {
              "interval": "1m",
              "timeout": "10s",
              "grace_period": "1s",
              "restart_limit": 0,
              "port": 5500,
              "method": "get",
              "protocol": "http",
              "path": "/flycheck/vm",
              "tls_skip_verify": false,
              "headers": [],
              "type": "http"
            }
          },
          "env": {
            "PRIMARY_REGION": "cdg",
            "BACKEND_STORE": "consul"
          },
          "metrics": {
            "port": 9187,
            "path": "/metrics"
          }
        }
      }
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query($appName: String!) { app(name: $appName) { autoscaling { enabled minCount maxCount balanceRegions regions { code minCount weight } } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (167.67ms)

DEBUG {
  "data": {
    "app": {
      "autoscaling": {
        "enabled": false,
        "minCount": 0,
        "maxCount": 10,
        "balanceRegions": false,
        "regions": [
          {
            "code": "cdg",
            "minCount": 0,
            "weight": 100
          }
        ]
      }
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query($appName: String!, $showCompleted: Boolean!) { appstatus:app(name: $appName) { allocations(showCompleted: $showCompleted) { id idShort version latestVersion status desiredStatus totalCheckCount passingCheckCount warningCheckCount criticalCheckCount createdAt updatedAt canary region restarts healthy privateIP taskName checks { status output name } } } }",
  "variables": {
    "appName": "gacha-list-db",
    "showCompleted": false
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (182.22ms)

DEBUG {
  "data": {
    "appstatus": {
      "allocations": [
        {
          "id": "c0fc22b7-558b-26e5-1502-21bb64ada8bf",
          "idShort": "c0fc22b7",
          "version": 1,
          "latestVersion": false,
          "status": "running",
          "desiredStatus": "run",
          "totalCheckCount": 3,
          "passingCheckCount": 3,
          "warningCheckCount": 0,
          "criticalCheckCount": 0,
          "createdAt": "2023-03-29T05:24:35Z",
          "updatedAt": "2023-03-29T05:24:35Z",
          "canary": false,
          "region": "cdg2",
          "restarts": 0,
          "healthy": true,
          "privateIP": "fdaa:0:345e:a7b:abc:0:3fe0:2",
          "taskName": "app",
          "checks": [
            {
              "status": "passing",
              "name": "vm",
              "output": "HTTP GET http://172.19.64.2:5500/flycheck/vm: 200 OK Output: \"[✓] checkDisk: 9.05 GB (92.5%) free space on /data/ (50.43µs)\\n[✓] checkLoad: load averages: 0.09 0.10 0.06 (93.54µs)\\n[✓] memory: system spent 0s of the last 60s waiting on memory (60.91µs)\\n[✓] cpu: system spent 1.22s of the last 60s waiting on cpu (50.14µs)\\n[✓] io: system spent 0s of the last 60s waiting on io (43.89µs)\"\n"
            },
            {
              "status": "passing",
              "name": "pg",
              "output": "HTTP GET http://172.19.64.2:5500/flycheck/pg: 200 OK Output: \"[✓] transactions: read/write (467.66µs)\\n[✓] replicationLag: fdaa:0:345e:a7b:abd:0:3fdf:2 is lagging 1.46µs (153ns)\\n[✓] connections: 21 used, 3 reserved, 300 max (5.15ms)\"\n"
            },
            {
              "status": "passing",
              "name": "role",
              "output": "leader"
            }
          ]
        },
        {
          "id": "61d17b86-15ed-6fb6-1b95-b009347e93c3",
          "idShort": "61d17b86",
          "version": 1,
          "latestVersion": false,
          "status": "running",
          "desiredStatus": "run",
          "totalCheckCount": 3,
          "passingCheckCount": 3,
          "warningCheckCount": 0,
          "criticalCheckCount": 0,
          "createdAt": "2022-07-19T18:10:56Z",
          "updatedAt": "2022-07-19T18:10:56Z",
          "canary": false,
          "region": "cdg2",
          "restarts": 6,
          "healthy": true,
          "privateIP": "fdaa:0:345e:a7b:abd:0:3fdf:2",
          "taskName": "app",
          "checks": [
            {
              "status": "passing",
              "name": "pg",
              "output": "HTTP GET http://172.19.0.82:5500/flycheck/pg: 200 OK Output: \"[✓] transactions: readonly (657.07µs)\\n[✓] replication: syncing from fdaa:0:345e:a7b:abc:0:3fe0:2 (370.14µs)\\n[✓] connections: 7 used, 3 reserved, 300 max (5.81ms)\"\n"
            },
            {
              "status": "passing",
              "name": "vm",
              "output": "HTTP GET http://172.19.0.82:5500/flycheck/vm: 200 OK Output: \"[✓] checkDisk: 9.16 GB (93.6%) free space on /data/ (45.03µs)\\n[✓] checkLoad: load averages: 0.05 0.03 0.02 (74.27µs)\\n[✓] memory: system spent 0s of the last 60s waiting on memory (46.66µs)\\n[✓] cpu: system spent 1.33s of the last 60s waiting on cpu (35.5µs)\\n[✓] io: system spent 0s of the last 60s waiting on io (30.51µs)\"\n"
            },
            {
              "status": "passing",
              "name": "role",
              "output": "replica"
            }
          ]
        }
      ]
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "query($appName: String!) { app(name: $appName) { vmSize { name cpuCores memoryGb memoryMb priceMonth priceSecond } taskGroupCounts { name count } processGroups { name vmSize { name cpuCores memoryGb memoryMb priceMonth priceSecond } maxPerRegion } } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (170.79ms)

DEBUG {
  "data": {
    "app": {
      "vmSize": {
        "name": "shared-cpu-1x",
        "cpuCores": 1.0,
        "memoryGb": 0.25,
        "memoryMb": 256,
        "priceMonth": 1.94,
        "priceSecond": 7.5e-07
      },
      "taskGroupCounts": [
        {
          "name": "app",
          "count": 2
        }
      ],
      "processGroups": [
        {
          "name": "app",
          "vmSize": {
            "name": "shared-cpu-1x",
            "cpuCores": 1.0,
            "memoryGb": 0.25,
            "memoryMb": 256,
            "priceMonth": 1.94,
            "priceSecond": 7.5e-07
          },
          "maxPerRegion": 0
        }
      ]
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "mutation($appName: ID!) { enablePostgresConsul(input: {appId: $appName}) { consulUrl } }",
  "variables": {
    "appName": "gacha-list-db"
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (151.35ms)

DEBUG {
  "data": {
    "enablePostgresConsul": {
      "consulUrl": "https://:5611327d-c88e-300f-e3c0-c88efcb62454@consul-na.fly-shared.net/gacha-list-db-5jlyv9rzn5y18xrg/"
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "mutation($input: ValidateWireGuardPeersInput!) { validateWireGuardPeers(input: $input) { invalidPeerIps } }",
  "variables": {
    "input": {
      "peerIps": [
        "fdaa:0:345e:a7b:1596:0:a:102",
        "fdaa:0:565f:a7b:8dd7:0:a:2"
      ]
    }
  }
}


DEBUG {}
DEBUG <-- 200 https://api.fly.io/graphql (167.05ms)

DEBUG {
  "data": {
    "validateWireGuardPeers": {
      "invalidPeerIps": []
    }
  }
}

DEBUG --> POST https://api.fly.io/graphql

DEBUG {
  "query": "\n# @genqlient\nquery AgentGetInstances ($appName: String!) {\n\tapp(name: $appName) {\n\t\torganization {\n\t\t\tslug\n\t\t}\n\t\tid\n\t\tname\n\t\tallocations(showCompleted: false) {\n\t\t\tid\n\t\t\tregion\n\t\t\tprivateIP\n\t\t}\n\t\tmachines {\n\t\t\tnodes {\n\t\t\t\tstate\n\t\t\t\tid\n\t\t\t\tregion\n\t\t\t\tips {\n\t\t\t\t\tnodes {\n\t\t\t\t\t\tkind\n\t\t\t\t\t\tfamily\n\t\t\t\t\t\tip\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n",
  "variables": {
    "appName": "gacha-list-db"
  },
  "operationName": "AgentGetInstances"
}

DEBUG {0x14000cdb8f0}
DEBUG <-- 200 https://api.fly.io/graphql (156.57ms)

DEBUG {
  "data": {
    "app": {
      "organization": {
        "slug": "personal"
      },
      "id": "gacha-list-db",
      "name": "gacha-list-db",
      "allocations": [
        {
          "id": "c0fc22b7-558b-26e5-1502-21bb64ada8bf",
          "region": "cdg2",
          "privateIP": "fdaa:0:345e:a7b:abc:0:3fe0:2"
        },
        {
          "id": "61d17b86-15ed-6fb6-1b95-b009347e93c3",
          "region": "cdg2",
          "privateIP": "fdaa:0:345e:a7b:abd:0:3fdf:2"
        }
      ],
      "machines": {
        "nodes": []
      }
    }
  }
}

DEBUG gqlGetInstances() result: &{[cdg2.gacha-list-db.internal (fdaa:0:345e:a7b:abc:0:3fe0:2) cdg2.gacha-list-db.internal (fdaa:0:345e:a7b:abd:0:3fdf:2)] [fdaa:0:345e:a7b:abc:0:3fe0:2 fdaa:0:345e:a7b:abd:0:3fdf:2]}

DEBUG gqlErr: <nil> agentErr: <nil>

DEBUG flypg will connect to: http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500

DEBUG --> GET http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500/commands/admin/role

DEBUG <-- 404 http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500/commands/admin/role (63.99ms)

DEBUG 404 page not found

DEBUG 404 response from /commands/admin/role endpoint. Calling legacy endpoint.

DEBUG --> GET http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500/flycheck/role

DEBUG <-- 200 http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500/flycheck/role (44.66ms)

DEBUG "leader"


DEBUG role for fdaa:0:345e:a7b:abc:0:3fe0:2: leader

DEBUG flypg will connect to: http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500

DEBUG --> GET http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500/commands/admin/settings/view

DEBUG [
  "max_wal_senders",
  "max_replication_slots"
]

DEBUG {0x140010ca000}
DEBUG <-- 404 http://fdaa:0:345e:a7b:abc:0:3fe0:2:5500/commands/admin/settings/view (70.48ms)

DEBUG 404 page not found

DEBUG Task manager done
Error: 404: 404 page not found

I have been trying to get it up and running for a while now, but I’m never able to migrate and get the database up and running.

When I run the command for apollo-db I get:

Error: no 6pn ips found for apollo-db app

If you could help me get it up and running, that would be amazing. Or at minimum, enable me to get the data out of the database.

Hi! Are you sure this app is running?

migrate-to-v2 relies on the assumption that the app is currently running in a couple places, and it looks like apollo-db has no running VMs right now. fly scale count 1 -a apollo-db might help?

It seems the app has crashed as well. I’m now working with support to get it resolved.

I received a mail to update a *-db app. However, I don’t know how to do that. fly migrate-to-v2 does not take the app as an argument. When I run it from my app folder, it just looks at the main app, not the DB app, and tells me that it “is already on the apps v2 platform”. Clicking the migrate button from the dashboard shows me a page with an error like this:

From there, clicking on “Launch web CLI” here:

Just brings me to this page:

And there it’s hanging.

In a new folder, run fly config save -a your-app-db and fly migrate-to-v2