Security-related (CVMFS) locations

[fstagni]

PR #210 adds a number of integration tests for the Pilot. One of these highlighted potential issues with Pilots using CVMFS-deployed installations ¹. Each DIRAC client installation at the moment relies on these environment variables, pointing to 3 distinct directories:

• X509_CERT_DIR : for the CAs
• X509_VOMS_DIR : for some VO-dependent VOMS files
• DIRAC_VOMSES (or X509_VOMSES) : for some other VO-dependent VOMS files
  For locally-installed clients these variables are added, and populated, by dirac-configure in the local directory tree ²

For CVMFS-installed releases the above env variables should normally point to locations that are independent from where a release is found. If any of the above env variable is set within the worker node environment, those values are kept. Otherwise, the "standard" CVMFS locations can normally be found in /cvmfs/grid.cern.ch/etc/grid-security:

 |-> ll /cvmfs/grid.cern.ch/etc/grid-security/
total 10
drwxr-xr-x.  2 cvmfs cvmfs 8192  6 ott 14.34 certificates
drwxrwxr-x. 59 cvmfs cvmfs   18 17 mar  2023 vomsdir
drwxrwxr-x.  2 cvmfs cvmfs   42  3 ago 13.50 vomses

And this is what I have added as default in #210 .

For historical reasons in LHCb we have been using /cvmfs/lhcb.cern.ch/etc/grid-security. At the same time, there might be some sites that have cvmfs mounted in a non-standard location (this was certainly true in AFS and NFS times, but could still be true for some sites today, and for HPCs especially). For LHCb, another historical env variable was $VO_LHCB_SW_DIR, that could be set by sites for signaling a non-standard locations of the "shared area".

My questions here are: do I get everything correct? What am I missing? Would you prefer something different?

Footnotes

1. The first implementation of https://github.com/DIRACGrid/Pilot/issues/166 went in https://github.com/DIRACGrid/Pilot/pull/205. PR test: adding integration tests #210 adds some fixes. ↩

2. This is what happens when you issues dirac-configure: Checking DIRAC installation at "/client/diracos" Current hash for bundle CAs in directory /client/diracos/etc/grid-security/certificates is '' Synchronizing directory with remote bundle Dir has been synchronized Current hash for bundle CRLs in directory /client/diracos/etc/grid-security/certificates is '' Synchronizing directory with remote bundle Dir has been synchronized Created vomsdir file /client/diracos/etc/grid-security/vomsdir/dteam/voms2.hellasgrid.gr.lsc Created vomses file /client/diracos/etc/grid-security/vomses/dteam Created vomsdir file /client/diracos/etc/grid-security/vomsdir/gridpp/voms.gridpp.ac.uk.lsc Created vomsdir file /client/diracos/etc/grid-security/vomsdir/gridpp/voms02.gridpp.ac.uk.lsc Created vomsdir file /client/diracos/etc/grid-security/vomsdir/gridpp/voms03.gridpp.ac.uk.lsc Created vomses file /client/diracos/etc/grid-security/vomses/gridpp Created vomsdir file /client/diracos/etc/grid-security/vomsdir/wlcg/wlcg-voms.cloud.cnaf.infn.it.lsc Created vomses file /client/diracos/etc/grid-security/vomses/wlcg ↩

[chaen]

I do not think that /cvmfs/ can be mounted anywhere else

[fstagni]

I do not think that /cvmfs/ can be mounted anywhere else

Unless we are 100% sure about this, and that it will never be allowed, then we should live with the above's.

[chrisburr]

A significant chunk of software on CVMFS doesn't work if you mount it elsewhere (including DIRACOS2).

[fstagni]

That simplifies (or forces to simplify) lots of things. I will then go under the assumption that, if CVMFS is mounted, is always in /cvmfs/..

[chrisburr]

Just to wrap it up, if /cvmfs/ is in a non-standard location we should use containers to mount it in the expected location.

[chaen]

Just checked with the Obermeister.
He confirms that:

• it is in principle allowed
• most soft would be unusable (not only for LHCb)
• Not used as far as he knows
• Potentially in some HPC, but then they always symlink /cvmfs

So safe to go

[iueda]

Belle II client installation I have made on cvmfs would also work only if cvmfs is mounted as /cvmfs.

We currently use the directories made by dirac-configure for those environment variables mentioned above, but the certificates is replaced with a symlink to /cvmfs/grid.cern.ch/etc/grid-security/certificates.

[fstagni]

I had all the info I needed.