Elixir apps - ./app/bin/app_name remote no longer works (warning: the --remsh option will be ignored because IEx is running on limited shell)

[dad]

Hi,

Up until recently I’ve been able to get a console on my elixir app with the following command:

flyctl ssh console --app app_name --select -C "./app/bin/app_name remote"

I’m not sure what has changed within fly but that same command now returns a warning saying:

warning: the --remsh option will be ignored because IEx is running on limited shell

and none of my application code is available to use e.g.

> App.Repo.all(App.Schema.User)
** (RuntimeError) could not lookup Ecto repo App.Repo because it was not started or it does not exist
    lib/ecto/repo/registry.ex:22: Ecto.Repo.Registry.lookup/1
    lib/ecto/repo/supervisor.ex:160: Ecto.Repo.Supervisor.tuplet/2
    lib/app/repo.ex:2: App.Repo.all/2
    iex:1: (file)

This user reported a similar issue and apparently we need to pass --pty now, unfortunately that doesn’t work for me:

❯ flyctl ssh console --pty --app app_name --select -C "./app/bin/app remote"
? Select instance: syd.app.internal (fdaa:0:4f1b:a7b:2984:5:1f63:2)
Connecting to fdaa:0:4f1b:a7b:2984:5:1f63:2... complete
warning: the --remsh option will be ignored because IEx is running on limited shell
Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(rem-aac6-app@fdaa:0:4f1b:a7b:2984:5:1f63:2)1>

Has anyone else seen this error and been able to resolve the problem? /cc @andrewbarr1

Thanks

[dad]

The last time this command worked for me was March 29 (UTC +10) and now it does not.

I have tried this command on applications that were recently deployed and applications that haven’t been deployed in 2 months.

[dad]

Also for completeness none of the following work:

❯ flyctl ssh console --app app
Connecting to fdaa:0:4f1b:a7b:2984:5:1f63:2... complete
# ./app/bin/app remote
warning: the --remsh option will be ignored because IEx is running on limited shell
Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(rem-6df2-app@fdaa:0:4f1b:a7b:2984:5:1f63:2)1>

❯ flyctl ssh console --pty --app app                                  
Connecting to fdaa:0:4f1b:a7b:2984:5:1f63:2... complete
# ./app/bin/app remote
warning: the --remsh option will be ignored because IEx is running on limited shell
Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(rem-6b14-app@fdaa:0:4f1b:a7b:2984:5:1f63:2)1>

[MatthewIngwersen]

Hi @dad. What operating system and terminal program are you running fly ssh console from? If it’s on Linux, macOS, or another *nix system, what’s the value of the TERM environment variable?

[dad]

@MatthewIngwersen

MacOS 13.2.1
Terminal: Kitty

❯ echo $TERM      
xterm-kitty

[MatthewIngwersen]

Ah, I see. I suspect that the quick and dirty fix is to set TERM=xterm when you run fly ssh console.

flyctl used to hardwire the TERM in your SSH session to xterm, but now it’s set to match your local TERM type. This makes sure that anything you run over SSH produces output that gets interpreted correctly, but sometimes container images don’t have terminfo files for less common terminal types. I’m guessing that’s what’s happening here. Depending on how you’re building your container image, it may be possible to install a package to add terminfo for Kitty’s terminal type (e.g. Alpine has ncurses-terminfo-base).

It may be that flyctl setting TERM “correctly” is going to cause more trouble than it’s worth, so thanks for posting your issue. This is on my radar now!

[dad]

@MatthewIngwersen setting TERM resolved the problem

This now works:

TERM=xterm fly ssh console --pty -C "app/bin/app remote" --config fly/production.toml --select

Thanks!