Elixir Remote console seems not to be connected to actual app

I’m getting this error when I try to remote into my app. In this case I’m just doing a basic read from Iex. I’m about 99% certain it’s not an Ash problem (and so is the creator of Ash):

iex> MyAshResource |> Ash.read!(authorize?: false)
** (Ash.Error.Unknown)
Bread Crumbs:
  > Exception raised in: MyApp.MyAshDomain.MyAshResource.read

Unknown Error

* ** (ArgumentError) errors were found at the given arguments:

  * 1st argument: the table identifier does not refer to an existing ETS table

  (stdlib 4.2) :ets.match_object(:telemetry_handler_table, {:handler, :_, [:ash, :accessibility, :read | :_], :_, :_})
  (telemetry 1.3.0) /app/deps/telemetry/src/telemetry_handler_table.erl:59: :telemetry_handler_table.list_by_prefix/1
  (telemetry 1.3.0) /app/deps/telemetry/src/telemetry.erl:379: :telemetry.list_handlers/1
  (ash 3.4.62) lib/ash/actions/read/read.ex:76: Ash.Actions.Read.run/3
  (ash 3.4.62) lib/ash.ex:2044: Ash.read/2
  (ash 3.4.62) lib/ash.ex:2002: Ash.read!/2
  (elixir 1.18.1) src/elixir.erl:386: :elixir.eval_external_handler/3
  (stdlib 4.2) erl_eval.erl:748: :erl_eval.do_apply/7
    (stdlib 4.2) :ets.match_object(:telemetry_handler_table, {:handler, :_, [:ash, :accessibility, :read | :_], :_, :_})
    (telemetry 1.3.0) /app/deps/telemetry/src/telemetry_handler_table.erl:59: :telemetry_handler_table.list_by_prefix/1
    (telemetry 1.3.0) /app/deps/telemetry/src/telemetry.erl:379: :telemetry.list_handlers/1
    (ash 3.4.62) lib/ash/actions/read/read.ex:76: Ash.Actions.Read.run/3
    (ash 3.4.62) lib/ash.ex:2044: Ash.read/2
    (ash 3.4.62) lib/ash.ex:2002: Ash.read!/2
    iex:3: (file)

I’m using the following command for the remote shell

	fly ssh console --pty --select -C "/app/bin/my_app remote"

Anyone have any idea why that might be? The app itself is running, but the console doesn’t seem to be actually connected. Process.whereis returns nil for everything that gets started in application.ex.

I’m not sure what could be causing this issue. It may have started around the time I added Horde for a Supervisor/Registry/GenServer, but I would expect that to only affect those particular processes. I can’t know for sure because I haven’t used the remote shell in a while.

Locally, it’s all fine. And at this stage of the app I actually do rely on the shell for onboarding BETA clients.

Hey there!

There’s also a chance all your nodes are not bundled together. Can you run Node.list() inside iex to verify?

We have a more in depth guide on how to cluster elixir nodes using our internal DNS

Edit: literally forgot to paste the link :smile:

1 Like

Interesting.

Node.list() returns [], so I can see it’s definitely not connected. I had the DNS clustering stuff configured already except for the entry in fly.toml. I added that, deployed and am getting the same result.

I’m getting a warning in my logs on deploy:

WARNING The app is not listening on the expected address and will not be reachable by fly-proxy.asdfg reached started state
You can fix this by configuring your app to listen on the following addresses:
  - 0.0.0.0:8080

Do I need to change my endpoint configuration?

  host = System.get_env("PHX_HOST") || "example.com"
  port = String.to_integer(System.get_env("PORT") || "8080")

  config :twfex, MyAppWeb.Endpoint,
    url: [host: host, port: 443, scheme: "https"],
    http: [
      # Enable IPv6 and bind on all interfaces.
      # Set it to  {0, 0, 0, 0, 0, 0, 0, 1} for local network only access.
      # See the documentation on https://hexdocs.pm/bandit/Bandit.html#t:options/0
      # for details about using IPv6 vs IPv4 and loopback vs public addresses.
      ip: {0, 0, 0, 0, 0, 0, 0, 0},
      port: port
    ],
    secret_key_base: secret_key_base

Thank you for your help!

There’s a chance this warning is happening because your app took a few seconds to boot the server and our proxy was eager. If your app can be reachable via your *.fly.dev (or custome) domain it means its fine.

Your code looks exactly like one of my apps:

Another thing you’d want to check about clustering is envs: look for rel/env.sh.ex that is what sets the release node name so the dns cluster will just work.

Here’s an example for my very old app:

(yes I stole it from Chris’ livebeats and forgot to remove that env :laughing: )

And this is the env.sh.eex for fly.io itself:

#!/bin/sh

# configure node for distributed erlang with IPV6 support
export ERL_AFLAGS="-proto_dist inet6_tcp"
export DNS_CLUSTER_QUERY="${FLY_APP_NAME}.internal"
export RELEASE_DISTRIBUTION="name"
export RELEASE_NODE="${FLY_APP_NAME}-${FLY_IMAGE_REF##*-}@${FLY_PRIVATE_IP}"
x

Can you check if they look similar to yours and update then fly deploy as needed?

Tip: you can fly ssh console and check their values with env command.

Hi @lubien, thanks for your help on this!

My env.sh.eex (it should be “.eex” not “.ex”, right?) file looks like this:

#!/bin/sh

# configure node for distributed erlang with IPV6 support
export ERL_AFLAGS="-proto_dist inet6_tcp"
export ECTO_IPV6="true"
export DNS_CLUSTER_QUERY="${FLY_APP_NAME}.internal"
export RELEASE_DISTRIBUTION="name"
export RELEASE_NODE="${FLY_APP_NAME}-${FLY_IMAGE_REF##*-}@${FLY_PRIVATE_IP}"

I think that’s exactly what you had in your example, with the addition of the ECTO_IPV6 flag. I got it straight from the Fly docs. It’s what I’ve had for a while now and I’m still getting the same issue in the fly console.

That’s correct!

Just to double check, you’re using fly ssh console (enters an existing machine) instead of fly console (creates a new machine) right?

This is just an wild guess but can you try fly ssh console without any flags and do /app/bin/my_app remote after connecting?

That worked!

The command I’ve been using is:

fly ssh console --pty --select -C "/app/bin/my_app remote"

… which is suddenly also working now :woman_shrugging:

1 Like

I wonder if it’s related to a deploy fixing envs?

But anyway I’m glad it worked :smiley:

Thanks for the help! Still have no idea what caused it but I’m glad it went away.

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.