Rucio Integration with the VRE Data Lake

[goseind]

Motivation

Based on our recent discussions, we would like to suggest integrating Rucio with the Rucio VRE Data Lake within SWAN and, later, possibly other Rucio instances from which users can select based on their permissions to access these instances.

This integration should be made available as an option in the Helm Charts, allowing third parties to install it, as well as in the existing CERN instance.

Upon login, the user could then select an instance to connect to:

Within JupyterLab, the user would then be able to use Rucio to access the Data Lake instance, as shown in this example of the VRE instance:

Modifications

To achieve a minimal working example that connects to the Rucio VRE instance by default and authenticates with VOMS, the following changes are required (First Step):

• Installation of the Rucio Jupyter Extension in the singleuser image
• Pre-configuration of the singleuser environment with the necessary Rucio environment variables

This should then only affect the CERN chart.

Additionally, both SWAN charts could be enhanced in a way that (Second Step):

• multiple Rucio instances can be configured by storing their configuration as JSON in a ConfigMap
• tokens can be used to access the Rucio instances (possibly requires enhancement on Rucio side and an exchange mechanism to place a valid token into a file for the extension and Rucio CLI to use)

Additional customisation (Third Step) could be made to the SWAN login page to allow selections and auth configuration of Rucio.

We'd love to set this up and are looking forward to your feedback on this 🚀 😃

[goseind]

@etejedor if you have a moment to take a look and give me some feedback, I've started working on it and have created some draft PRs.

also tagging @garciagenrique and @egazzarr to review this as well 🔦