VOMS-AA replica deployment example

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "compose/voms-replica/voms-testsuite"]
+	path = compose/voms-replica/voms-testsuite
+	url = git@github.com:italiangrid/voms-testsuite.git

compose/voms-replica/.env

@@ -0,0 +1,14 @@
+COMPOSE_PROJECT_NAME=voms
+TRUST_IMAGE=indigoiam/egi-trustanchors
+TRUST_IMAGE_TAG=igi-test-ca
+DB_IMAGE=mysql
+DB_IMAGE_TAG=8.3
+NGINX_IMAGE=baltig.infn.it:4567/cnafsd/ngx_http_voms_module/nginx-httpg-voms
+NGINX_IMAGE_TAG=latest
+VOMS_AA_IMAGE=indigoiam/voms-aa-bp
+VOMS_AA_IMAGE_TAG=v1.8.3
+GRID_CLIENTS_IMAGE=indigoiam/robot-framework
+GRID_CLIENTS_IMAGE_TAG=latest
+IAM_IMAGE=indigoiam/iam-login-service
+IAM_IMAGE_TAG=v1.8.3
+

compose/voms-replica/README.md

@@ -0,0 +1,95 @@
+# VOMS-AA Replica
+
+This folder contains a docker compose example to deploy three instances of VOMS-AA in two sites with a DB replica.
+
+![Setup schema.](VOMS-AA_replica.png)
+
+With this setup the VOMS-AA service can be replicated on one or more remote locations. If one location fails or is overloaded, a VOMS client can connect to the other locations.
+
+## Deployment description
+
+The `compose` file definises a few containers:
+
+* `trust`: docker image for the GRID CA certificates plus the `igi-test-ca` used in this deployment for test certificates.
+
+The actual VOMS services are virtually divided between three sites:
+
+#### Site 1: CERN
+* `db-primary`: a dump of the IAM db for test environment. In addition to the db populated with the iam `mysql-dev` profile, the user `test` has a certificate with DN `/C=IT/O=IGI/CN=test0` linked to his account and he also is part of the `indigo-dc` group (necessary to obtain VOMS proxies). A second SQL script creates a `replicator` user for replica.
+* `db-replica`: a DB configured to replicate the statements of `db-primary`, from the initial one. It conects with SSL and is configured to be read-only. Only the IAM DB tables which are used by VOMS-AA are replicated. You can see them [here](assets/mysql-conf/replica.cnf).
+* `vomsaa-primary` and `vomsaa-replica`: the main voms-aa microservices, each connected to their own DB.
+* `ngx-primary` and `ngx-replica`: an extension to NGINX, used for TLS termination, reverse proxy and possibly VOMS proxies validation. They sends requests to the corresponding `vomsaa-primary` and `vomsaa-replica` services.
+
+
+#### Site 2: CNAF
+* `db-remote`: this is a full replica of `db-replica`. Only the tables used by VOMS-AA are present in `db-replica`, and thus we avoiding transmitting over the remote network useless information.
+* `vomsaa-remote`
+* `ngx-remote`
+
+#### Site 3: Anywhere else
+* `client`: it is a single container containing GRID clients (in particular `voms-proxy-init`) used to query both the VOMS services. It connects in round-robin fashion to each endpoint and when one fails it falls back to the others. Here a p12 file for the test user encrypted with the `pass` password is present in the well-known directory (`/home/test/.globus/usercred.p12`). It can be used to obtain a VOMS proxy by `voms-aa` serving a VO named `indigo-dc`.
+
+### Networking
+
+We use a few distinct networks, similar to a real scenario:
+
+* `site1-lan` and `site2-lan`: The internal LAN of the two sites. These are used to connect the DB, VOMS-AA and NGINX between them inside the same site.
+* `site-to-site-tunnel`: This is a VPN network or any tunnel network between the two sites, used by `db-remote` to connect to `db-replica`.
+* `wan`: The NGINX servers are exposed on the public network so that the clients can connect from anywhere.
+
+## Test
+
+Run the docker-compose with
+
+```
+$ docker compose up -d
+```
+
+and wait for the `trust` service to finish; all the services will be available shortly afterwards.
+
+To query the voms-aa using the VOMS client, run:
+
+``` 
+$ docker compose exec client voms-proxy-init -voms indigo-dc
+Enter GRID pass phrase for this identity:
+Contacting voms-remote.test.example:443 [/C=IT/O=IGI/CN=*.test.example] "indigo-dc"...
+Remote VOMS server contacted succesfully.
+
+
+Created proxy in /tmp/x509up_u1000.
+
+Your proxy is valid until Sat Mar 16 03:54:38 CET 2024
+```
+
+Check the content of the proxy with
+
+```
+$ docker compose exec client voms-proxy-info -all
+subject   : /C=IT/O=IGI/CN=test0/CN=441572696
+issuer    : /C=IT/O=IGI/CN=test0
+identity  : /C=IT/O=IGI/CN=test0
+type      : RFC3820 compliant impersonation proxy
+strength  : 2048
+path      : /tmp/x509up_u1000
+timeleft  : 11:59:45
+key usage : Digital Signature, Non Repudiation, Key Encipherment
+=== VO indigo-dc extension information ===
+VO        : indigo-dc
+subject   : /C=IT/O=IGI/CN=test0
+issuer    : /C=IT/O=IGI/CN=*.test.example
+attribute : /indigo-dc/Role=NULL/Capability=NULL
+timeleft  : 11:59:45
+uri       : voms-remote.test.example:8080
+
+```
+If you want to force the query to one voms-aa use one of the followings:
+```
+$ docker compose exec client voms-proxy-init -voms voms-primary
+$ docker compose exec client voms-proxy-init -voms voms-replica
+$ docker compose exec client voms-proxy-init -voms voms-remote
+```
+
+To run the testsuite:
+```
+$ docker compose exec testsuite bash run-testsuite.sh
+```

compose/voms-replica/assets/certs/voms.test.example.cert.pem

@@ -0,0 +1,85 @@
+Certificate:
+    Data:
+        Version: 3 (0x2)
+        Serial Number: 19 (0x13)
+    Signature Algorithm: sha512WithRSAEncryption
+        Issuer: C=IT, O=IGI, CN=Test CA
+        Validity
+            Not Before: Oct 19 08:55:57 2022 GMT
+            Not After : Oct 16 08:55:57 2032 GMT
+        Subject: C=IT, O=IGI, CN=*.test.example
+        Subject Public Key Info:
+            Public Key Algorithm: rsaEncryption
+                Public-Key: (2048 bit)
+                Modulus:
+                    00:e7:3a:01:a8:93:12:08:f4:a6:c9:89:10:a2:f6:
+                    6a:6a:d3:93:98:c7:31:c0:e5:8a:3a:44:9b:cf:ef:
+                    b9:3d:05:86:03:61:0e:6e:fc:c6:f9:9a:9e:35:d6:
+                    3d:38:27:48:cb:77:26:97:15:34:a0:0b:1d:97:31:
+                    dd:18:ec:bf:78:d9:32:9e:00:1a:44:6a:78:15:1f:
+                    ac:7b:3e:bb:ad:b2:b4:32:75:8c:11:d8:31:ec:19:
+                    7d:bf:ba:5d:1e:70:38:62:10:cf:3a:8a:a4:98:83:
+                    b4:df:e0:50:3b:e5:ec:24:a0:89:14:2c:19:27:48:
+                    66:c3:d4:1d:74:63:be:63:38:95:3f:64:d0:91:ac:
+                    95:f7:d9:ca:96:b5:1b:e7:71:70:7b:5f:3b:12:30:
+                    2c:b8:3a:28:79:84:9c:81:12:db:38:31:6d:2d:2a:
+                    e2:80:05:5c:29:77:53:58:10:19:ee:f9:50:e1:8d:
+                    3b:2b:e2:c0:0b:d2:9f:3c:a0:95:33:f8:33:17:ce:
+                    23:0e:31:e8:1e:3d:7e:6a:c9:6d:83:9e:0b:fa:43:
+                    d2:4a:3f:be:d3:19:07:1e:8c:e4:f6:dc:8f:c3:3e:
+                    3a:8e:66:4a:87:ef:0b:39:db:e8:3e:30:1c:91:9e:
+                    b3:1e:d3:a0:1e:1b:9a:b1:58:99:de:a5:bb:53:3b:
+                    3b:5d
+                Exponent: 65537 (0x10001)
+        X509v3 extensions:
+            X509v3 Basic Constraints: critical
+                CA:FALSE
+            X509v3 Subject Key Identifier: 
+                60:FA:21:CE:1C:B5:31:8D:9B:01:F6:08:5B:72:4D:59:5A:F8:71:8C
+            X509v3 Key Usage: critical
+                Digital Signature, Non Repudiation, Key Encipherment
+            X509v3 Extended Key Usage: 
+                TLS Web Server Authentication, TLS Web Client Authentication, Microsoft Server Gated Crypto, Netscape Server Gated Crypto, E-mail Protection
+            X509v3 Authority Key Identifier: 
+                keyid:50:9B:6F:74:01:E3:1A:03:57:AB:D9:D5:7D:15:64:4C:25:F3:F8:F4
+
+            X509v3 Subject Alternative Name: 
+                DNS:*.test.example
+    Signature Algorithm: sha512WithRSAEncryption
+         79:82:f2:54:44:98:96:25:c2:83:c9:0f:19:69:1c:f6:a7:19:
+         0d:61:90:f9:96:23:e2:ab:5a:30:db:55:d7:4f:b0:ff:b2:7b:
+         41:da:35:97:47:86:e4:85:00:6d:11:64:ee:32:a4:64:ee:fe:
+         b2:83:a5:24:4a:ce:c3:91:ae:db:3d:5b:af:fa:7e:81:1a:1c:
+         69:d0:1a:9e:70:0e:9e:74:85:6b:48:90:6a:1b:62:ff:6e:b3:
+         84:30:b7:7f:fa:c0:3e:ee:91:70:0b:f2:13:ea:c8:2c:aa:d8:
+         cb:3c:60:b1:08:f9:8e:bf:c2:e4:ce:92:6a:7e:0a:41:49:94:
+         8f:e5:6e:71:f9:47:04:1a:18:1f:65:47:d6:1c:ea:a9:90:71:
+         82:1b:3b:1f:a5:f2:02:ce:5c:d6:2e:5d:1e:05:c4:92:9e:3d:
+         8e:ce:fa:00:83:01:d5:c3:c1:cf:e2:e5:fb:08:80:08:f4:6c:
+         26:64:96:db:cd:be:4c:e7:bc:8f:af:3d:0e:0c:f7:d2:52:15:
+         9c:d5:15:0d:51:b3:95:72:78:1d:8c:ca:37:55:7a:c0:b0:0f:
+         18:ae:de:d0:27:6f:1b:e4:5d:1d:4b:f9:4c:5d:44:49:ed:cf:
+         c2:9e:e7:c6:55:72:ce:2f:43:a7:2f:88:de:b7:da:9f:82:a6:
+         54:77:c2:2e
+-----BEGIN CERTIFICATE-----
+MIIDmTCCAoGgAwIBAgIBEzANBgkqhkiG9w0BAQ0FADAtMQswCQYDVQQGEwJJVDEM
+MAoGA1UECgwDSUdJMRAwDgYDVQQDDAdUZXN0IENBMB4XDTIyMTAxOTA4NTU1N1oX
+DTMyMTAxNjA4NTU1N1owNDELMAkGA1UEBhMCSVQxDDAKBgNVBAoMA0lHSTEXMBUG
+A1UEAwwOKi50ZXN0LmV4YW1wbGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
+AoIBAQDnOgGokxII9KbJiRCi9mpq05OYxzHA5Yo6RJvP77k9BYYDYQ5u/Mb5mp41
+1j04J0jLdyaXFTSgCx2XMd0Y7L942TKeABpEangVH6x7PrutsrQydYwR2DHsGX2/
+ul0ecDhiEM86iqSYg7Tf4FA75ewkoIkULBknSGbD1B10Y75jOJU/ZNCRrJX32cqW
+tRvncXB7XzsSMCy4Oih5hJyBEts4MW0tKuKABVwpd1NYEBnu+VDhjTsr4sAL0p88
+oJUz+DMXziMOMegePX5qyW2Dngv6Q9JKP77TGQcejOT23I/DPjqOZkqH7ws52+g+
+MByRnrMe06AeG5qxWJnepbtTOztdAgMBAAGjgbwwgbkwDAYDVR0TAQH/BAIwADAd
+BgNVHQ4EFgQUYPohzhy1MY2bAfYIW3JNWVr4cYwwDgYDVR0PAQH/BAQDAgXgMD4G
+A1UdJQQ3MDUGCCsGAQUFBwMBBggrBgEFBQcDAgYKKwYBBAGCNwoDAwYJYIZIAYb4
+QgQBBggrBgEFBQcDBDAfBgNVHSMEGDAWgBRQm290AeMaA1er2dV9FWRMJfP49DAZ
+BgNVHREEEjAQgg4qLnRlc3QuZXhhbXBsZTANBgkqhkiG9w0BAQ0FAAOCAQEAeYLy
+VESYliXCg8kPGWkc9qcZDWGQ+ZYj4qtaMNtV10+w/7J7Qdo1l0eG5IUAbRFk7jKk
+ZO7+soOlJErOw5Gu2z1br/p+gRocadAannAOnnSFa0iQahti/26zhDC3f/rAPu6R
+cAvyE+rILKrYyzxgsQj5jr/C5M6San4KQUmUj+VucflHBBoYH2VH1hzqqZBxghs7
+H6XyAs5c1i5dHgXEkp49js76AIMB1cPBz+Ll+wiACPRsJmSW282+TOe8j689Dgz3
+0lIVnNUVDVGzlXJ4HYzKN1V6wLAPGK7e0CdvG+RdHUv5TF1ESe3Pwp7nxlVyzi9D
+py+I3rfan4KmVHfCLg==
+-----END CERTIFICATE-----

compose/voms-replica/assets/certs/voms.test.example.key.pem

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEA5zoBqJMSCPSmyYkQovZqatOTmMcxwOWKOkSbz++5PQWGA2EO
+bvzG+ZqeNdY9OCdIy3cmlxU0oAsdlzHdGOy/eNkyngAaRGp4FR+sez67rbK0MnWM
+Edgx7Bl9v7pdHnA4YhDPOoqkmIO03+BQO+XsJKCJFCwZJ0hmw9QddGO+YziVP2TQ
+kayV99nKlrUb53Fwe187EjAsuDooeYScgRLbODFtLSrigAVcKXdTWBAZ7vlQ4Y07
+K+LAC9KfPKCVM/gzF84jDjHoHj1+asltg54L+kPSSj++0xkHHozk9tyPwz46jmZK
+h+8LOdvoPjAckZ6zHtOgHhuasViZ3qW7Uzs7XQIDAQABAoIBAAx5xL0jskVpbdZR
+3uPsB7Hb2IrVtImD2QFr0jxV4ti4A5MLGYxDdzjgbsjY1lTBSdwwgZSFQGGiN+aA
+ej1uCKaskV6VAtXOKMx6+QNtTxMAIVjXnscXsxnaBj7h/0Q1KdWgso2mDVttP8UU
+hT+2GBeh0cOU3YaREXpfZ3dwKkWQHbtO/UYwVzu+XVFt8kApPoLMMHoXZfetP6Yp
+7YSCuI6id44mwqkP7aY8iGhcUpVTkP3LD7z8nUp4LaG9my6T1Wev8x7hstb/NIsZ
+DPiXAzfDUkHWqpMthnoWyOdghGc6JzKGFeJVHqrW4byJ4hNU3WvNIdvZ8tyIEpd1
+56uP/gECgYEA92oUhzHjvw87qfo6tPDai2I8AghXJoPGB6xYYhchlirYMGPx9fU/
+rcVEGbmSBDqXMg9eZUqiXB+E/hukCOrFZJt4kt656Nm/Xy68IDwSifmf5vcUde6q
+j0pD6i0vwJFjYWBjjS7gRBK83pr/jHhy8aK1+79lZ0GfbQkLxF/2TxkCgYEA70Af
+A387tHDmct7ZH0gAZx9QKYZhtS+WWVCIoZ81028DEeGri0By83KFkU0QZ9RfWKQi
+RajBYkB35xJFv4fSX5s4+tcVaTVJKOn7V5YGmUIxrGY3IMuE77+h9SEHd8GY723q
+9qgwTF5SQP3cGiVpGFB99M44CBuHbbypFh67iuUCgYEA3Zp6QI2C/AJc4mZqZt7E
+IMwgC4IE7U5h9UV89H7banF9qfobIr5EBxUFZjU8f+Uqv3/cgMVUn0bsC94eEo6V
+twM5//LWeaVvL4Xgos6rnEGl422zOd5HjohqRDms58JRTUrUYAR4gwB1gr0530uT
+SLMAZTiNTusMLNFJZN6+8yECgYAulAY1sRSXmY9T98y/iU4CxZberrnhA2W697HR
+/WQGSMuJNK0oDCEVAku8sQsrm64AXNwLQcJ8dV6iju0jT7cGQ/sA4tTZSbV3kK4N
+LDkWp0tya+f5q4WzA1Ttm0OP7hHvMzAWW0Ij7A0JeCLcuEHQqQMMoQVJlspz89Hb
+a5pJfQKBgFZb6XnLMTSCs/SQe38PQiawIQcA+zXmhG83xkEKspQGm2KqyJL+AdKQ
+fXQKoKa/Ubyp7PKRJVZ8raX1/kvtFDQIQ+G3L/hps5rhZgDh5S2n0xd4zlbK/Sw6
+l3RjOUpHSe8oz+X3Jinl/Rwr39I9hrRAW2xj7vkFb84IE98mJu2X
+-----END RSA PRIVATE KEY-----

compose/voms-replica/assets/db-logs/.gitignore

@@ -0,0 +1 @@
+*.log