Fixes #27932 - Add REX Cockpit support

[ekohl]

This only works with REX 1.8.3 and newer on RPMs.

Replaces #748. It takes a different approach by making a separate class rather than a parameter.

[ares]

@ekohl is there something I can help with? I'm pretty sure this will get decent testing after it's in :-)

[ekohl]

@ares this will also need an installer migration to expose the class. @iNecas confirmed in the other PR that it should indeed be the CA that's exposed by Foreman so I've pushed a change. Initially forgot to git add the acceptance test. It may also need SELinux changes but that our CI can't test that nor is it a blocker to merging.

[ares]

Sounds good, I will ask someone (@lzap 😉) to take a look at selinux when this is in. Right now, tests seems to be broken by last change. Once fixed, I'm happy to merge.

[ehelms]

What exactly is this deploying? A new service? A set of plugin configurations?

[ekohl]

There are files in https://github.com/theforeman/foreman_remote_execution/tree/master/extra/cockpit which are packaged as a -cockpit subpackage. This contains a service that runs on port 9999 that allows tunneling SSH over a websockets connection. It is running a private cockpit instance as well.

It does connect back to Foreman to verify a token. The CA file is needed to verify the HTTPS connection to it. A cleaner approach would probably be to use JWT to sign it so it can be verified without a connection.

It needs the cert and key to set up an authenticated connection to a Smart Proxy. I can't see whether it verifies that connection since it doesn't set up a CA to verify it. It could be made more secure by providing the client CA.

This all depends on the REX plugin being deployed so it requires that.

[lzap]

Sounds good, I will ask someone (@lzap wink) to take a look at selinux when this is in. Right now, tests seems to be broken by last change. Once fixed, I'm happy to merge.

If I understand this correctly then it's the cockpit service which connect to port 9999 and not Foreman. Thus it will probably need a change in Cockpit policy rather than ours.

[ares]

@lzap the service that runs on 9999 is foreman specific, it's used for tunelling the connection over REX SSH connection. So I think our policy needs to be updated.

[ehelms]

I am wondering a bit if this service name should be more specific. As I think I understand the comments, this is dependent entirely upon remote execution and is not a generic service for Foreman and cockpit. Something like foreman-rex-cockpit or foreman-remote-execution-cockpit.

[ekohl]

In that case we should update it in packaging first since this mirrors that:
https://github.com/theforeman/foreman-packaging/blob/e61fc0b8bc36d9658bfa7bb4177aca60834a53e6/packages/plugins/rubygem-foreman_remote_execution/rubygem-foreman_remote_execution.spec#L95

[ehelms]

Do I understand correctly that the service:

1. Spins up an instance of cockpit
2. Provides websocket support for tunneling SSH

Are we using cockpit only to get the SSH -> websocket functionality as a convenience? Is there other functionality of the running cockpit instance being used?

[ehelms]

Is a description of all the connections laid out anywhere I can look at?

[ares]

@mvollmer mind to link any docs or provide answers to @ehelms, cc @iNecas

[iNecas]

This thread should have most of the context https://community.theforeman.org/t/seamless-cockpit-and-foreman/9846/23,

This service is mainly focused on performing authentication, the tunneling is achieved with this combined with smart-proxy capabilities.

Deferring to @mvollmer for more details.

[ekohl]

@ehelms do you want to block on this? Given most of the packaging changes have already been merged for some time.

[lzap]

SELinux related question - where/how do we distribute file named /etc/systemd/system/foreman-cockpit.service described in the README of https://github.com/mvollmer/foreman-cockpit/ ?

It simply spawns /usr/libexec/cockpit-ws --no-tls --address 127.0.0.1 --port 9999 which is a problem for SELinux - we need to create our own wrapper script which can be labelled with foreman_cockpit_bin_t to do the initial transition. Also a sidenote - the port seems to be hardcoded there, we probably want to extract it into an ENV file.

Another concern is about the default port (9999), I need to check (I am in a train currently) however if it is already taken by existing policy we want probably to pick a different one which can be assigned a port type.

[ekohl]

https://github.com/theforeman/foreman_remote_execution/tree/master/extra/cockpit

You're right that Environment=PORT=9999 and Environment=ADDRESS=127.0.0.1 for that would be nice. theforeman/foreman_remote_execution#439 should do that.

[ehelms]

No, I won't block on the naming. Just express a strong desire for things to be named such that they reflect as specific as possible their use case in order to avoid confusion by users and developers.

[ehelms]

Does this new service need to be added to foreman-maintain to ensure it gets started/stopped/restarted when performing a full refresh of services? (e.g. if the certificates change)

[ares]

What is the progress on this? Can we get it in and open tracking issue on foreman-maintain? Is there something blocking this PR? It's hard to verify SElinux work without installer nightlies that would install it.