Skip to content
This repository has been archived by the owner on Apr 2, 2024. It is now read-only.

DSE Deployment - Pass credentials gathered through Vault into Podman DSE instance #30

Closed
marcionemec-daitan opened this issue Feb 7, 2024 · 1 comment
Closed

DSE Deployment - Pass credentials gathered through Vault into Podman DSE instance #30

marcionemec-daitan opened this issue Feb 7, 2024 · 1 comment
Assignees
@marcionemec-daitan
Milestone
PoC - DataSync En...

Comments

@marcionemec-daitan
Copy link
Contributor

Issue: Database credentials needs to be handled by a secrets engine (Vault) to be injected in the Source Code of Data Sync Engine in Run Time execution, avoiding data leakage.

Background: Vault access was configured in Jenkins Cron server, but needs to be implemented in the Jenkins Polaris architecture, using podman containerized application.

List of base documentation to implement:

Acceptance Criteria:

1: Sample Dummy code retrieving data from Vault and checking if K/V are expected.
2: Documentation in Confluence Page.

** Ticket Reference **

This ticked blocks Issue bcgov/nr-spar-data#47 as database credentials are stored in Vault, so first is needed to implement a vault access to Jenkins Polaris execution instance.
@marcionemec-daitan marcionemec-daitan self-assigned this Feb 7, 2024
@marcionemec-daitan marcionemec-daitan mentioned this issue Feb 7, 2024
Closed
@SLDonnelly
Copy link

SLDonnelly commented Feb 13, 2024 •
edited
Loading

@marcionemec-daitan you can reach out to Andreas Wilson and Grace Ruan from 1Team for assistance resolving this issue

CC: @macarrig @PMAKIA1

marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Testing Vault Access with Token
207a5fb
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Vtoken credential as environment variable for all stages
c171eb6
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Including requests in the libraries of project
2bdb185
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 passing environment arguments to container
7732298
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 removing JSON from requirements.txt
e495022
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Handling env vars
b05a812
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Bringing secrets inside podman
471017f
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Fixing arguments syntax
8bff2e7
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 fixing typo
71afdf6
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 Using Podman polaris library to build container
f6de70f
marcionemec-daitan added a commit that referenced this issue Feb 13, 2024
@marcionemec-daitan
#30 removing script podman library exchanging to execute withcredentials
dfb4f62
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 Fixing credentials getter
80a0790
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 fixing env var names
8e96231
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 fixing env var passthrough pod
459cdd8
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 Including Unit Test for env Variables
bf60612
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 fixing identation
df0cec2
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 renaming env var
9657d22
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 renaming env vars
0407d68
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 renaming env jenkins vars
3bc7cf0
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 renaming variables
f869991
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 removing cred for simple vars
8429242
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 Quote env vars from jenkins
0596afb
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 Verifying env vars
c939ad8
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 fixing env var
99cd634
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 sync vars env
a352033
marcionemec-daitan added a commit that referenced this issue Feb 14, 2024
@marcionemec-daitan
#30 change env vars to bring req jenkins vars
36134e0
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
#30 Solve Boolean type mismatch
0b61045
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
#30 fixing library os environ access
cf12651
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
#30 fixing unit test function
ec9866f
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
#30 vault token compliance unit test
7faa26c
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
Feature/30/marcio/vault credentials to jenkins polaris (#31)
9f8dcfc 
* #27 - Including JenkinsFile for Polaris Jenkins Server

* #27 - Including some changes to test Polaris Features

* #27 Updating agent configuration

* #27 Remove unnecessary python checking

* #27 Including post conditions

* #27 Including 1st test to determine readiness of code

* #27 Fixing missing single quote

* #27 pwd + dir

* #27 Including Docker file for podman build containerization

* #27 Hello World message to guarantee no error message in pipeline, for now.

* #30 Testing Vault Access with Token

* #30 Vtoken credential as environment variable for all stages

* #30 Including requests in the libraries of project

* #30 passing environment arguments to container

* #30 removing JSON from requirements.txt

* #30 Handling env vars

* #30 Bringing secrets inside podman

* #30 Fixing arguments syntax

* #30 fixing typo

* #30 Using Podman polaris library to build container

* #30 removing script podman library exchanging to execute withcredentials

* #30 Fixing credentials getter

* #30 fixing env var names

* #30 fixing env var passthrough pod

* #30 Including Unit Test for env Variables

* #30 fixing identation

* #30 renaming env var

* #30 renaming env vars

* #30 renaming env jenkins vars

* #30 renaming variables

* #30 removing cred for simple vars

* #30 Quote env vars from jenkins

* #30 Verifying env vars

* #30 fixing env var

* #30 sync vars env

* #30 change env vars to bring req jenkins vars

* #30 Solve Boolean type mismatch

* #30 fixing library os environ access

* #30 fixing unit test function

* #30 vault token compliance unit test
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
Develop MR to Main, bringing vault unit tests (#32)
f8361e3 
* Adapt data sync engine to run on jenkins (#18)

#5: Adapt Engine to run on Jenkins Cron

* #19 Include Compliance YAML to project to track risks and Privacy Impact Assessment status (#21)

* #13 Include Badge for project Status (#24)

* Feature/27/marcio/jenkins convert to polaris (#28)

* #27 - Including JenkinsFile for Polaris Jenkins Server

* #27 - Including some changes to test Polaris Features

* #27 Updating agent configuration

* #27 Remove unnecessary python checking

* #27 Including post conditions

* #27 Including 1st test to determine readiness of code

* #27 Fixing missing single quote

* #27 pwd + dir

* #27 Including Docker file for podman build containerization

* #27 Hello World message to guarantee no error message in pipeline, for now.

* Feature/30/marcio/vault credentials to jenkins polaris (#31)

* #27 - Including JenkinsFile for Polaris Jenkins Server

* #27 - Including some changes to test Polaris Features

* #27 Updating agent configuration

* #27 Remove unnecessary python checking

* #27 Including post conditions

* #27 Including 1st test to determine readiness of code

* #27 Fixing missing single quote

* #27 pwd + dir

* #27 Including Docker file for podman build containerization

* #27 Hello World message to guarantee no error message in pipeline, for now.

* #30 Testing Vault Access with Token

* #30 Vtoken credential as environment variable for all stages

* #30 Including requests in the libraries of project

* #30 passing environment arguments to container

* #30 removing JSON from requirements.txt

* #30 Handling env vars

* #30 Bringing secrets inside podman

* #30 Fixing arguments syntax

* #30 fixing typo

* #30 Using Podman polaris library to build container

* #30 removing script podman library exchanging to execute withcredentials

* #30 Fixing credentials getter

* #30 fixing env var names

* #30 fixing env var passthrough pod

* #30 Including Unit Test for env Variables

* #30 fixing identation

* #30 renaming env var

* #30 renaming env vars

* #30 renaming env jenkins vars

* #30 renaming variables

* #30 removing cred for simple vars

* #30 Quote env vars from jenkins

* #30 Verifying env vars

* #30 fixing env var

* #30 sync vars env

* #30 change env vars to bring req jenkins vars

* #30 Solve Boolean type mismatch

* #30 fixing library os environ access

* #30 fixing unit test function

* #30 vault token compliance unit test
marcionemec-daitan added a commit that referenced this issue Feb 15, 2024
@marcionemec-daitan
Update Branch with develop code (#33)
3565ab8 
* Adapt data sync engine to run on jenkins (#18) (#20)

#5: Adapt Engine to run on Jenkins Cron

* Include Compliance YAML (#22)

Include Compliance YAML to project to track risks and Privacy Impact Assessment status (#21)

* Develop (#25)

Added Life Cycle Badge

* Feature/27/marcio/jenkins convert to polaris (#28)

* #27 - Including JenkinsFile for Polaris Jenkins Server

* #27 - Including some changes to test Polaris Features

* #27 Updating agent configuration

* #27 Remove unnecessary python checking

* #27 Including post conditions

* #27 Including 1st test to determine readiness of code

* #27 Fixing missing single quote

* #27 pwd + dir

* #27 Including Docker file for podman build containerization

* #27 Hello World message to guarantee no error message in pipeline, for now.

* Develop (#29)

* Adapt data sync engine to run on jenkins (#18)

#5: Adapt Engine to run on Jenkins Cron

* #19 Include Compliance YAML to project to track risks and Privacy Impact Assessment status (#21)

* #13 Include Badge for project Status (#24)

* Feature/27/marcio/jenkins convert to polaris (#28)

* #27 - Including JenkinsFile for Polaris Jenkins Server

* #27 - Including some changes to test Polaris Features

* #27 Updating agent configuration

* #27 Remove unnecessary python checking

* #27 Including post conditions

* #27 Including 1st test to determine readiness of code

* #27 Fixing missing single quote

* #27 pwd + dir

* #27 Including Docker file for podman build containerization

* #27 Hello World message to guarantee no error message in pipeline, for now.

* Feature/30/marcio/vault credentials to jenkins polaris (#31)

* #27 - Including JenkinsFile for Polaris Jenkins Server

* #27 - Including some changes to test Polaris Features

* #27 Updating agent configuration

* #27 Remove unnecessary python checking

* #27 Including post conditions

* #27 Including 1st test to determine readiness of code

* #27 Fixing missing single quote

* #27 pwd + dir

* #27 Including Docker file for podman build containerization

* #27 Hello World message to guarantee no error message in pipeline, for now.

* #30 Testing Vault Access with Token

* #30 Vtoken credential as environment variable for all stages

* #30 Including requests in the libraries of project

* #30 passing environment arguments to container

* #30 removing JSON from requirements.txt

* #30 Handling env vars

* #30 Bringing secrets inside podman

* #30 Fixing arguments syntax

* #30 fixing typo

* #30 Using Podman polaris library to build container

* #30 removing script podman library exchanging to execute withcredentials

* #30 Fixing credentials getter

* #30 fixing env var names

* #30 fixing env var passthrough pod

* #30 Including Unit Test for env Variables

* #30 fixing identation

* #30 renaming env var

* #30 renaming env vars

* #30 renaming env jenkins vars

* #30 renaming variables

* #30 removing cred for simple vars

* #30 Quote env vars from jenkins

* #30 Verifying env vars

* #30 fixing env var

* #30 sync vars env

* #30 change env vars to bring req jenkins vars

* #30 Solve Boolean type mismatch

* #30 fixing library os environ access

* #30 fixing unit test function

* #30 vault token compliance unit test
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@marcionemec-daitan marcionemec-daitan

Labels
None yet
Projects
SPAR Sprints | Goal: UI library testing and selection; Review lo-fi wireframes; Apply design changes as necessary; complete CONSEP seed testing activities usability testing
Status: Done
Milestone
PoC - DataSync Engine in Stage Enviro...
Development

No branches or pull requests
3 participants
@RMCampos @SLDonnelly @marcionemec-daitan