How can I tell cider to establish a direct connection?

[timothypratley]

OSX Yosemite 10.10.3
cider 20150617.753

Previously I could type C-c M-c
Cider would prompt me for Host:
I could type: dockerized.foo.com
the it would prompt me for port and I could connect.
Now it tries to establish an xssh connection as soon as I enter the host.
Password for /sshx:local-trek.outpace.com:

Here is my messages output:
Tramp: Opening connection for local-trek.outpace.com using sshx... Tramp: Sending command exec ssh -o ControlMaster=auto -o ControlPath='tramp.%r@%h:%p' -o ControlPersist=no -e none -t -t local-trek.outpace.com /bin/sh'
Tramp: Waiting for prompts from remote shell...
Tramp: Sending password`

[timothypratley]

I noticed that if I add an entry in /etc/hosts, I can workaround the issue.
It seems therefore that the cause is that when the host is in ~/.ssh/known_hosts
then cider wants to use SSH, but I don't want it to.

[bbatsov]

@vspinu Can you take a look at this?

[vspinu]

Here is my messages output:

The output is incomplete. Unless you set nrepl-force-ssh-for-remote-hosts totyou should first seenREPL: Establishing direct connection to....`. We always try direct connection before attempting ssh.

[vspinu]

The output is incomplete.

Oups. Need not be. Your problem happens even before nrepl-connect is invoked.

t seems therefore that the cause is that when the host is in ~/.ssh/known_hosts

Yes. We are currently relying on tramp for a list of known ssh hosts. I think we should limit that list to hosts explicitly declared in .ssh/config.

[bbatsov]

Or maybe we should simply have a list of an internal list of hosts maintained by CIDER. This will produce the most accurate results long-term.

[vspinu]

Or maybe we should simply have a list of an internal list of hosts maintained by CIDER.

That would burden users with keeping two imilar configuration lists. There is no harm in reading .ssh/config because all those hosts are intended for ssh consumption anyways. But this is not true for .ssh/known_hosts.

The problem is not that much that we are reading config files but that we are trying to be smart and detect all running instances of nrepl on the remote server. For that we need to call ps through tramp on the remote.

We have cider-known-endpoints but it doesn't currently provide a way to identify which endpoint should be accessed with ssh and which not.

Let's enhance cider-know-endpoints elements by allowing arbitrary key value pairs. Like (host port :ssh t :label "my-cool-host" :sniff-for-nrepl-ports nil). Entries in this list are in addition to endpoints declared in .ssh/config and can overwrite default settings. For example an explicit :ssh nil will not attempt a ssh connection even when the endpoint is declared in .ssh/config. :snif-for-nrepl-ports nil will disable sniffing for hosts in .ssh/config.

I am not sure how well this will play with ssh tunneling. Let me propose a simple patch first and then we see if we want to go further.

[bbatsov]

That would burden users with keeping two imilar configuration lists. There is no harm in reading .ssh/config because all those hosts are intended for ssh consumption anyways. But this is not true for .ssh/known_hosts.

I have never used .ssh/config to add hosts to it. I'm not sure what exactly are you referring to here.
My idea is not to burden users with some list they have to update themselves - it will be something to the known projects in projectile - once you connect to a server it will simply be remembered. Perhaps auto-tunnelled connections were a mistake - the users should decide this for themselves; half the people have no idea what the "direct connection" message means, which is a bad sign.

Let's enhance cider-know-endpoints elements by allowing arbitrary key value pairs. Like (host port :ssh t :label "my-cool-host" :sniff-for-nrepl-ports nil). Entries in this list are in addition to endpoints declared in .ssh/config and can overwrite default settings. For example an explicit :ssh nil will not attempt a ssh connection even when the endpoint is declared in .ssh/config. :snif-for-nrepl-ports nil will disable sniffing for hosts in .ssh/config.

Explicit beats implicit. I'm guessing we can just drop everything smart and simply this process.

I am not sure how well this will play with ssh tunneling. Let me propose a simple patch first and then we see if we want to go further.

Aren't those unrelated?

[vspinu]

I have never used .ssh/config to add hosts to it. I'm not sure what exactly are you referring to here.

You can connect through ssh either by specifying the whole command like

ssh my_user@ec2-54-215-40-82.us-west-1.compute.amazonaws.com

and provide password each time you connect or you can put

 Host my_host
      Hostname  ec2-54-215-40-82.us-west-1.compute.amazonaws.com
     User vspinu
     IdentityFile ~/.ssh/DS_EC2_Ubuntu.pem
     ForwardX11 no

into your config and not be prompted for password each time.

People who use ssh remotes always have configuration. It's global and every program will use it. Tramp uses it. Cider's SSH tunneling is using it. You don't want a separate configuration for cider only.

once you connect to a server it will simply be remembered.

This is what tramp does and we are piggybacking on tramp for a reason.

Explicit beats implicit.

Yes. Tramp and ssh configuration are explicit enough. We just have to document this in Cider's doc. Maybe on this occasion.

Perhaps auto-tunnelled connections were a mistake - the users should decide this for themselves

User has decided on this by placing an entry to .ssh/config. Ignoring it would be silly and would imply redundant burden to the user. But I am not sure about .ssh/know_hosts. I don't quite understand why a user would have an entry there but not want to connect to the host through ssh.

Aren't those unrelated?

No. Tunneling relies on tramp which uses ssh which in turn uses its config file or .ssh/know_host.

[vspinu]

@timothypratley, any particular reason of why you have an entry in .ssh/known_hosts but you don't want to use ssh for it? How common is this setup? Cannot you just add an entry to .ssh/config and use ssh instead?

There is a big advantage of using ssh. Cider will provide completion for ports of all running nREPLs on the remote.

[bbatsov]

People who use ssh remotes always have configuration. It's global and every program will use it. Tramp uses it. Cider's SSH tunneling is using it. You don't want a separate configuration for cider only.

I use ssh like all the time, but I simply use the same key on all hosts. Everything else seems like an overkill to me.

Yes. Tramp and ssh configuration are explicit enough. We just have to document this in Cider's doc. Maybe on this occasion.

Maybe. The problem with the current functionality is that it's super confusing for some people. It's confusing even for me to some extent - we use the ssh config just to find out a remote nrepl port. Doesn't seem either right or expected.

User has decided on this by placing an entry to .ssh/config. Ignoring it would be silly and would imply redundant burden to the user. But I am not sure about .ssh/know_hosts. I don't quite understand why a user would have an entry there but not want to connect to the host through ssh.

Yeah, but there's no indication that a host in the ssh config is a host running an nREPL server.

No. Tunneling relies on tramp which uses ssh which in turn uses its config file or .ssh/know_host.

I'm aware of this. What I meant is that we probably shouldn't be putting cider-connect and the tunneling under the same denominator.

[bbatsov]

@timothypratley, any particular reason of why you have an entry in .ssh/known_hosts but you don't want to use ssh for it? How common is this setup? Cannot you just add an entry to .ssh/config and use ssh instead?

There is a big advantage of using ssh. Cider will provide completion for ports of all running nREPLs on the remote.

Security advantages and port completion aside, it boils down to people not expecting this. I think at the very least we should prompt them that we can establish an ssh connection, so they get an idea what is going on. Power users would be able to suppress this message with a config option.

[timothypratley]

I have Vagrant hosting several Docker containers. The nrepl is in the Docker container. The entry in known_host is there because of Vagrant setup, but is not enough to reach the Docker container.
I think this is quite an unusual setup.

cider-known-endpoints '(("te" "local-trek.outpace.com" "22347")) still tried to SSH, which surprised me.

but the /etc/hosts entry allows me to connect.

[vspinu]

I use ssh like all the time, but I simply use the same key on all hosts.

So you like to type the whole user@host-name each time you connect? Or how does it work for you more concretely? Do you open a remote file or dired each time, and then cider-connect?

It's confusing even for me to some extent - we use the ssh config just to find out a remote nrepl port.

Port completion is supposed to be only the first step. As tunneling happens through ssh anyways, theoretically we end up connecting through ssh either way. This is not true for @timothypratley's because of his rather unusual setup. The ssh connection seems to happen behind tramp's back through Vagrant setup. I am not familiar with Vagrant and unfortunately don't have time to investigate that right now.

cider-known-endpoints '(("te" "local-trek.outpace.com" "22347")) still tried to SSH, which surprised me.

Yes. Port completion is clearly unnecessary in this case. I am fixing this right away. No sniffing should happen when port is present.

[bbatsov]

So you like to type the whole user@host-name each time you connect?

Usually I'm using my own user on the remote machines. In the other cases I either type user@host or create handy shell aliases. Anyways, I see your point. I'm just saying that you can't assume everyone is working like you. I've worked with plenty of people using my workflow.

Do you open a remote file or dired each time, and then cider-connect?

I don't really connect to prod servers with CIDER - doing serious changes to prod seems pretty reckless to me and to inspect the local state of the system I don't really need more than a basic REPL.

The fact that I don't use this functionality is the primary reason I didn't pay much attention to it when it was originally proposed.

[cap10morgan]

What was the ultimate resolution here? It's hard to tell from the commit.

People on my team are still seeing cider-connect try to SSH to any host in ~/.ssh/known_hosts and being surprised and frustrated by it (in CIDER 0.10.2).

Are there docs somewhere that explain this behavior and how to work around it when it isn't desired?

For the record, I agree with @bbatsov that this is unexpected behavior.

[vspinu]

Regretfully I am not developing in clojure anymore and not using Cider. So I am not sure I would be the best person to look at it again.

I see three straightforward solutions:

1. Have a custom variable for SSH files to sniff for ports, like '(~/.ssh/known_hosts ~/.ssh/config). I think ~/.ssh/config is a non-controversial.
2. Have a user level logical cider-sniff-for-ports-when-connected-only and use (file-remote-p file nil t) to check if there is already an emacs connection open outside of cider to that server.
3. Both of the above.

[bbatsov]

@cap10morgan You can file a new ticket and we can continue the discussion there.

[bbatsov]

P.S. I'd generally go for option 3.