This document explains various yaml files listed in the examples directory used to deploy a Tenant using MinIO Operator.
- kustomize/v4.3.0 https://kubectl.docs.kubernetes.io/installation/kustomize/
MinIO Operator can automatically generate TLS secrets and mount these secrets to the MinIO, Console, and/or KES pods (
enabled by default). To disable this, set the requestAutoCert
field to false
.
You can deploy the pre-configured example by running the following command:
kustomize examples/kustomization/base | kubectl apply -f -
This example will deploy a MinIO tenant with Server Side Encryption using KES and Hashicorp Vault.
-
Deploy
Vault
KMS in your cluster:kubectl apply -f examples/vault/deployment.yaml
-
Expose vault via k8s-portforward:
kubectl port-forward svc/vault 8200
on a terminal -
Obtain the
Vault
Root token from the pod logs:kubectl logs -l app=vault
-
Set the
Vault
token and address in the client:export VAULT_ADDR=http://localhost:8200 export VAULT_TOKEN=TOKEN
-
Enable role auth:
vault auth enable approle
-
Enable secrets k/v:
vault secrets enable kv
-
Create a new
KES
policy:vault policy write kes-policy examples/vault/kes-policy.hcl
-
Create a new
KES
role based on theKES
policy:vault write auth/approle/role/kes-role token_num_uses=0 secret_id_num_uses=0 period=5m policies=kes-policy
-
Get the
app-role-id
and write it down:vault read auth/approle/role/kes-role/role-id
-
Get the
app-role-secret-id
and write it down:vault write -f auth/approle/role/kes-role/secret-id
- Open
examples/kustomization/tenant-kes-encryption/kes-configuration-secret.yaml
- In the
Vault
configuration replace<YOUR APPROLE ID HERE>
for yourapp-role-id
,<YOUR APPROLE SECRET ID HERE>
for yourapp-role-secret-id
.
You can deploy a preconfigured example by running the following command:
kustomize build examples/kustomization/tenant-kes-encryption | kubectl apply -f -
Verify data is encrypted by connecting directly to MinIO via ingress controller
or using port-forward:
kubectl port-forward svc/minio 9000:443 -n tenant-kms-encrypted
mc alias set alias https://127.0.0.1:9000 minio minio123 --insecure
mc admin kms key status alias --insecure
Key: my-minio-key
- Encryption ✔
- Decryption ✔
This example will deploy a MinIO tenant with TLS using certificates provided by the user.
-
You can generate certificates using
Vault CA
,Openssl
orMkcert
, for this example we will use https://github.com/FiloSottile/mkcert -
Assuming your Tenant name is
myminio
and your namespace isminio-tenant
you should generate the following certificate keypairs:mkcert "*.minio-tenant.svc.cluster.local" mkcert "*.myminio.minio-tenant.svc.cluster.local" mkcert "*.myminio-hl.minio-tenant.svc.cluster.local"
MinIO
will use *.minio-tenant.svc.cluster.local
, *.myminio.minio-tenant.svc.cluster.local
and *.myminio-hl.minio-tenant.svc.cluster.local
certificates for
inter-node communication.
Create kubernetes secrets
based on the previous certificates
kubectl create secret tls minio-tls-cert --key="_wildcard.minio-tenant.svc.cluster.local-key.pem" --cert="_wildcard.minio-tenant.svc.cluster.local.pem" -n minio-tenant
kubectl create secret tls minio-buckets-cert --key="_wildcard.myminio.minio-tenant.svc.cluster.local-key.pem" --cert="_wildcard.myminio.minio-tenant.svc.cluster.local.pem" -n minio-tenant
kubectl create secret tls minio-hl-cert --key="_wildcard.myminio-hl.minio-tenant.svc.cluster.local-key.pem" --cert="_wildcard.myminio-hl.minio-tenant.svc.cluster.local.pem" -n minio-tenant
You need to provide those kubernetes secrets
in your Tenant YAML
overlay using the externalCertSecret
fields, ie:
externalCertSecret:
- name: minio-tls-cert
type: kubernetes.io/tls
- name: minio-buckets-cert
type: kubernetes.io/tls
- name: minio-hl-cert
type: kubernetes.io/tls
You can deploy a preconfigured example by running the following command:
kustomize build examples/kustomization/base | kubectl apply -f -
You can include all the certificates that you want in your Tenant and MinIO
will serve them to its client
via SNI
This example will deploy a minio tenant using mTLS certificates (authentication between MinIO
and KES
) provided by
the user, the data will be encrypted at rest
-
Configure
Vault
the same way as in the first example -
Set the
app-role-id
, theapp-role-secret-id
andkey-prefix
in your KES configurationYAML
file -
Assuming your Tenant name is
myminio
and namespace istenant-kms-encrypted
create all the certificates and secrets as in the previous step -
Generate new
KES
identity keypair (https://github.com/minio/kes), this is needed it for the authentication,mTLS
betweenMinIO
andKES
:kes tool identity new --key="./app.key" --cert="app.cert" app
-
Using the generated
app.key
andapp.cert
create a new kubernetes secret:kubectl create secret tls minio-kes-mtls --key="app.key" --cert="app.cert"
-n tenant-kms-encrypted and provide that secret in theexternalClientCertSecret
field of your tenantYAML
overlay (if the field doesn't exist add it)spec: ... externalClientCertSecret: name: minio-kes-mtls type: kubernetes.io/tls
-
Calculate the
app.cert
identity usingKES
:kes tool identity of app.cert
, copy the resulting hash and open your KES configurationYAML
(kes-configuration-secret.yaml
) file and replace${MINIO_KES_IDENTITY}
for thebda5d8b6531d2f3bcd64e5ec73841bcb23ecb57b19c5f814e491ea2b2088995c
string, you can add additional identities using this array, ie:policy: my-policy: paths: - /v1/key/create/* - /v1/key/generate/* - /v1/key/decrypt/* identities: - bda5d8b6531d2f3bcd64e5ec73841bcb23ecb57b19c5f814e491ea2b2088995c
You can deploy a pre-configured example by running the following command:
kustomize build examples/kustomization/tenant-kes-encryption | kubectl apply -f -
MinIO Operator can automatically generate LoadBalancer
and ClusterIP
type services when deploying tenants, however
there is one more way to expose your services in case you don't want to deal with load balancers
or ingress
configurations and
that is NodePort
. NodePort type services will be accessible by opening a port on each Kubernetes cluster node, read
more about NodePort.
kustomize build examples/kustomization/tenant-nodeport | kubectl apply -f -
For additional examples on how to deploy a tenant with LDAP or OIDC you can look at the examples directory