[Data Sync] Handle database credentials

[thiagosanchezenc]

Obtain database connection in Oracle Database and Postgres Database using safe stored credentials obtained by a Secret Engine.

Background: Data Sync Engine Needs to obtain connection in databases in run time execution. Credentials must be obtained during runtime in requests to a secrets management engine.

Acceptance Criteria:

1. Sample code connecting in Postgres and returning a result of a dummy query (like Select CURRENT_TIME a)
2. Sample code connecting in Oracle and returning a result of a dummy query (like Select NOW() a FROM DUAL)
3. Documentation available in confluence page.

[RMCampos]

Handling Jenkins to access Vault to handle credentials.

[craigyu]

rolled over from sprint 23

[marcionemec-daitan]

This issue is blocked by bcgov/nr-spar-data-sync#26 and bcgov/nr-spar-data-sync#27 issues, regarding the change of Jenkins Server from Cron to Polaris.

[craigyu]

rolled over from Sprint 24

[marcionemec-daitan]

This issue is blocked by bcgov/nr-spar-data-sync#30 (Vault credentials gathering implementation in Jenkins Polaris).

[SLDonnelly]

rolled over from sprint 25 to 26

[marcionemec-daitan]

Updates for this task, so far:

I finally could use the Openshift Client inside a Podman instance (at least to run "oc version" command).

I tried:

1. Use a openshift red hat image, but I couldn't update the machine libraries because it needs a ret hat login credential to access the library/modules repositories.
2. Use a openshift-client docker image, but it is built in a Linux Alpine base and it was hard (at least for me) to install other things and maintain this image or run this image in parallel with the podman python container acessing it. Besides, it's not an official maintained Red Hat image.
3. Use a Python Image and install the OpenShift client by wget (using an official Red Hat release). Wget failed to retrieve the URL bevcause the proxy denied the URL access.
4. Tried to use the Python image with the release Openshift Client already downloaded: it was successful.

[marcionemec-daitan]

Latest error:
https://cd.io.nrs.gov.bc.ca/job/spar/job/DataSyncEngine/116/console

STEP 22/25: RUN ./oc_bin/oc login --token=$oct --server=$ocs
error: context deadline exceeded
Error: building at STEP "RUN ./oc_bin/oc login --token=$oct --server=$ocs": while running runtime: exit status 1

Requesting help to 1Team.

[marcionemec-daitan]

This issue will need to be rolled up to the next sprint.
The Jenkins Polaris approach was put on hold because it was not possible to ensure a Postgres database port expose through portforward command. So, at this point we are trying to use a Github Workflow actions to ensure the deployment of ETL batch instances.

[RMCampos]

Rolled over from sprint 23

[marcionemec-daitan]

This issue should be closed, as the old repo does not exists anymore (new repo is nr-spar-data).
There are a new structure and new tasks to be included in future sprints.