Allow spread encryption to be set in the VerticaDB CR

[spilchen]

The following parm was added to the VerticaDB CR:

spec.encryptSpreadComm

When this is set to vertica, then the EncryptSpreadComm parameter will be set for the database right after it is created. Setting encryption in spread requires a server restart, which the operator handles automatically. The default value for this parameter is an empty string, which means no encryption is set.

The events for the VerticaDB will look like the following when enabling spread encryption. We will show a successful create database, then a complete server restart.

Events:
  Type    Reason                   Age   From                Message
  ----    ------                   ----  ----                -------
  Normal  CreateDBStart            78s   verticadb-operator  Calling 'admintools -t create_db'
  Normal  CreateDBSucceeded        48s   verticadb-operator  Successfully created database with subcluster 'my-sc'. It t
ook 30.065841637s
  Normal  StopDBStart              47s   verticadb-operator  Calling 'admintools -t stop_db'
  Normal  StopDBSucceeded          43s   verticadb-operator  Successfully stopped the database.  It took 4s
  Normal  ClusterRestartStarted    42s   verticadb-operator  Calling 'admintools -t start_db' to restart the cluster
  Normal  ClusterRestartSucceeded  33s   verticadb-operator  Successfully called 'admintools -t start_db' and it took 9s

This can only be set for new instances of VerticaDB. It is ignored if the initPolicy is something other than Create. It only accepts an empty string (to clear encryption) or vertica. The webhook enforces these rules.

The restart was generalized with a new condition variable: VerticaRestartNeeded. In the future, if we have other cases where Vertica needs to be restarted we can use this condition variable.

Closes #222

[nyikesda]

Hi @spilchen ,
Did you test the solution with active client connection during the DB stop? It would be important to stop the DB independently from the connected users.

[spilchen]

@nyikesda the stop DB doesn't wait for active connection. If there are any, those connections will be terminated. This happens immediately after create db though. Do you expect to have a lot of connections come in between? You can use various status conditions to know when the database is fully setup.

kubectl wait --for=condition=DBInitialized=True vdb/v   # Wait for DB to be created
kubectl wait --for=condition=VerticaRestartNeeded=False vdb/v  # Wait for stop db
kubectl wait --for=condition=Ready=True pod/v-my-sc-0 # Wait for start db by checking ready state of one pod

[nyikesda]

I do not see --force argument in your code after the admintools -t stop_db, so I thought that it waites for the the active connections. If it the --force argument is not neccessary, then sorry for my previous comment.

[spilchen]

Okay, I understand what you mean. You are right I am missing a force. I'll add that in.

[nyikesda]

Thanks

[roypaulin]

Looks good. I don't have too much to add.