[#513] remove code preventing release of 1.11

DRAFT_RELEASE_NOTES.md

@@ -1,7 +1,7 @@
 # Major Additions
 
 ## Path to Production Alignment
-To better align development processes with processes in CI/CD and higher environments, we no longer recommend using Tilt for building and deploying projects.  As such, upgrading projects should consider removing or at least narrowing the scope of their Tiltfile. See _**How to Upgrade**_ for more information.
+To better align development processes with processes in CI/CD and higher environments, we no longer recommend using Tilt live-reloading.  As such, upgrading projects should consider narrowing the scope of their Tiltfile. See _**How to Upgrade**_ for more information.
 
 ## Data Access Upgrade
 Data access through [GraphQL](https://graphql.org/) has been deprecated and replaced with [Trino](https://trino.io/). Trino is optimized for performing queries against large datasets by leveraging a distributed architecture that processes queries in parallel, enabling fast and scalable data retrieval.
@@ -42,7 +42,6 @@ To reduce burden of upgrading aiSSEMBLE, the Baton project is used to automate t
 | upgrade-tiltfile-aissemble-version-migration         | Updates the aiSSEMBLE version within your project's Tiltfile                                                                                                                            |
 | upgrade-v2-chart-files-aissemble-version-migration   | Updates the Helm chart dependencies within your project's deployment resources (`<YOUR_PROJECT>-deploy/src/main/resources/apps/`) to use the latest version of the aiSSEMBLE            |
 | upgrade-v1-chart-files-aissemble-version-migration   | Updates the docker image tags within your project's deployment resources (`<YOUR_PROJECT>-deploy/src/main/resources/apps/`) to use the latest version of the aiSSEMBLE                  |
-| spark-infrastructure-universal-config-yaml-migration | Removes the default hive username (if present) from hive-metastore-service values.yaml so that it can be set by the configuration store service                                         |
 | pipeline-invocation-service-template-migrtion        | Include the helm.valueFiles param to ArgoCD pipeline-invocation-service template                                                                                                        |                                                                                                                                                      
 | docker-module-pom-dependency-type-migration          | Updates the maven pipeline dependency type within your project's sub docker module pom file(`<YOUR_PROJECT>-docker/*-docker/pom.xml`) to fix the build cache checksum calculation issue |
 | enable-maven-docker-build-migration                  | Remove the maven fabric8 plugin `skip` configuration within your project's docker module pom file(`<YOUR_PROJECT>-docker/pom.xml`) to enable the maven docker build                     |
@@ -72,10 +71,6 @@ To deactivate any of these migrations, add the following configuration to the `b
 
 ## Precondition Steps - Required for All Projects
 
-### Maven Docker Build
-To avoid duplicate docker builds, remove all the related `docker_build()` and `local_resources()` functions from your Tiltfile. Also, the `spark-worker-image.yaml` is no longer used 
-so `-deploy/src/main/resources/apps/spark-worker-image` directory ,and the related `k8s_yaml()` function from your Tiltfile can be removed.
-
 ### Beginning the Upgrade
 To start your aiSSEMBLE upgrade, update your project's pom.xml to use the 1.11.0 version of the build-parent:
 ```xml
@@ -86,6 +81,9 @@ To start your aiSSEMBLE upgrade, update your project's pom.xml to use the 1.11.0
 </parent>
 ```
 
+### Tilt Docker Builds
+To avoid duplicate docker builds, remove all the related `docker_build()` and `local_resources()` functions from your Tiltfile. Also, the `spark-worker-image.yaml` is no longer used so the `-deploy/src/main/resources/apps/spark-worker-image` directory and the related `k8s_yaml()` function from your Tiltfile can be removed.
+
 ## Conditional Steps
 
 ## Final Steps - Required for All Projects

extensions/extensions-helm/README.md

@@ -27,52 +27,19 @@ Follow the instructions in the [_Kubernetes Artifacts Upgrade_](https://boozalle
 section of the _Path to Production > Container Support_ documentation page to update older projects to the new Extensions Helm baseline approach.
 
 ## Developing with Extension Helm
-
-When testing modifications to a Helm chart in a downstream project, special steps have to be taken as Helm charts are not published to the remote Helm Repository until a build
-via GitHub Actions is completed. Firstly, all modifications to the aiSSEMBLE chart need to be committed and pushed to GitHub. Then, the chart downstream dependency needs to be
-updated to point to the modified chart on your branch in GitHub.  Unfortunately, this is [still not natively supported in
-Helm](https://github.com/boozallen/aissemble/issues/488#issuecomment-2518466847), so we need to do some setup work to enable a plugin that provides this functionality.
-
-### Add the plugin to ArgoCD
-
-The repo server is responsible for running Helm commands to push changes into the cluster.  This is
-[documented](https://argo-cd.readthedocs.io/en/stable/user-guide/helm/#using-initcontainers) in ArgoCD, however these instructions didn't work, at least with our current chart
-version of 7.4.1. (Note there were some discussions on the `argocd-helm` GitHub about a specific update causing a breaking change in this functionality, and the latest docs were
-the "fix" for the breaking change. So could be that the old instructions would have worked fine.) To do this, we'll add an init container to the repo server that installs the
-plugin to a shared volume mount.
-
-```yaml
-aissemble-infrastructure-chart:
-  argo-cd:
-    repoServer:
-      env:
-        - name: HELM_CACHE_HOME
-          value: /helm-working-dir/.cache #Work around for install issue where plugins and cache locations being the same conflicts
-      initContainers:
-        - name: helm-plugin-install
-          image: alpine/helm
-          env:
-            - name: HELM_PLUGINS
-              value: /helm-working-dir/plugins #Configure Helm to write to the volume mount that the repo server uses
-          volumeMounts:
-            - mountPath: /helm-working-dir
-              name: helm-working-dir
-          command: [ "/bin/sh", "-c" ]
-          args: # install plugin
-            - apk --no-cache add curl;
-              helm plugin install https://github.com/aslafy-z/helm-git --version 1.3.0;
-              chmod -R 777 $HELM_PLUGINS;
-```
-
-### Updating the chart dependency
-
-To use your modified chart in the downstream project, the following changes should be made to the `Chart.yaml` file that pulls in the modified chart as a dependency:
-
- * Point `repository` to the modified chart on your branch in GitHub
-   * e.g.: `git+https://github.com/boozallen/aissemble/@extensions/extensions-helm/<modified-chart>?ref=<your-branch>`
-   * _**NB:** if the chart being tested is in a nested project under extensions-helm, update the repo path accordingly_
- * Set `version` to `1.0.0`
-
-### Potential pitfalls
-
- * There is an issue with committing Chart.lock files when using an explicit repository vs a repository alias, so Chart.lock files must not be committed.
+* When completing and locally testing a migration of a module to Extensions Helm, the above steps can be taken.
+  * As a precursor to the above steps, it would be helpful to create a simple aiSSEMBLE project to use as a test-bed.
+* It is important to note that because the module's helm charts will not be published to the Helm Repository remote 
+  until a build via GitHub Actions is completed, the `repository` field in the module's `Chart.yaml` file must be updated 
+  to point to the local aiSSEMBLE baseline code. More specifically, in the test-bed project, the `repository` field in
+  `<project-name>-deploy/src/main/resources/apps/<module-name>/Chart.yaml` should have hold a value in the following form:
+`"file://../../../../../../../aissemble/extensions/extensions-helm/aissemble-<module-name>-chart"`
+  * the file path given is relative to the location of the `Chart.yaml` file, so 7 or more `../` prefixes will be 
+    required to reach wherever the local aiSSEMBLE baseline is stored on your local machine
+  * in this example 7 `../` prefixes are added to the relative path, as the test-bed project sits in the same directory 
+    as the local `aissemble` baseline code.
+* Additionally, for local development only, the application's `Chart.yaml` in its corresponding `aissemble-<app-name>-chart` 
+  should set the `version` and `appVersion` field to whatever the current aiSSEMBLE version is; this will allow for 
+  testing of the local deployment when leveraging tilt
+  * If making use of additional aiSSEMBLE charts within your application's dependencies, the dependent subcharts should 
+  have their `version` and `appVersion` updated to the current aiSSEMBLE version as well

extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-hive-metastore-service-chart/README.md

@@ -34,7 +34,6 @@ helm install hive-metastore-service oci://ghcr.io/boozallen/aissemble-hive-metas
 | deployment.volumes                              | The deployment volumes                                                                                                                                  | No                | &emsp; - name: metastore-service-config <br/>&emsp;&emsp;configMap: <br/>&emsp;&emsp;&emsp;name: metastore-service-config <br/>&emsp;&emsp;&emsp;items: <br/>&emsp;&emsp;&emsp;&emsp; - key: metastore-site.xml <br/>&emsp;&emsp;&emsp;&emsp;&emsp;path: metastore-site.xml |
 | service.spec.ports                              | The service spec ports                                                                                                                                  | No                | &emsp; - name: "thrift" <br/>&emsp;&emsp;port: 9083 <br/>&emsp;&emsp;targetPort: 9083                                                                                                                                                                                       |
 | mysql.enabled                                   | Whether to use mysql as the backing database                                                                                                            | No                | true                                                                                                                                                                                                                                                                        |
-| configMap.configStore                           | Whether to use Configuration Store to inject property values                                                                                            | No                | enabled                                                                                                                                                                                                                                                                     |                                                                                                                                                                                                                                                                            
 | configMap.metastoreServiceConfig.baseProperties | Default configuration for the metastore service                                                                                                         | No                | See [values.yaml](./values.yaml)                                                                                                                                                                                                                                            |
 | configMap.metastoreServiceConfig.properties     | Optional configuration for the metastore service. Properties added here will be included in the configuration without overriding the default properties | No                |                                                                                                                                                                                                                                                                             |
 
@@ -49,7 +48,6 @@ configuration options.
 |------------------|---------------------|
 | fullnameOverride | "hive-metastore-db" |
 | auth.database    | "metastore"         |
-| auth.username    | "hive"              |
 
 # Migration from aiSSEMBLE v1 Helm Charts
 
@@ -78,7 +76,7 @@ In the table below, the notation `env[KEY]` refers the `env` list item whose `na
 | configMap.metastoreServiceConfig.configuration.property[metastore.expression.proxy]            | configMap.metastoreServiceConfig.baseProperties[metastore.expression.proxy]            |                                                        |
 | configMap.metastoreServiceConfig.configuration.property[javax.jdo.option.ConnectionDriverName] | configMap.metastoreServiceConfig.baseProperties[javax.jdo.option.ConnectionDriverName] |                                                        |
 | configMap.metastoreServiceConfig.configuration.property[javax.jdo.option.ConnectionURL]        | configMap.metastoreServiceConfig.baseProperties[javax.jdo.option.ConnectionURL]        |                                                        |
-| configMap.metastoreServiceConfig.configuration.property[javax.jdo.option.ConnectionUserName]   | configMap.metastoreServiceConfig.baseProperties[javax.jdo.option.ConnectionUserName]   | Using Configuration store Service to inject this value |
+
 ## Property Removed
 
 The following properties no longer exist.

extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-hive-metastore-service-chart/templates/configmap.yaml

@@ -2,8 +2,6 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: metastore-service-config
-  labels:
-    aissemble-configuration-store: {{ .Values.configMap.configStore}}
 data:
   # Add all the default properties from the local values.yaml to the ConfigMap
   # Then check if there are any downstream properties and add them as well

extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-hive-metastore-service-chart/tests/configmap_test.yaml

@@ -33,11 +33,6 @@ tests:
                   <value>jdbc:mysql://hive-metastore-db:3306/metastore?createDatabaseIfNotExist=true&amp;allowPublicKeyRetrieval=true&amp;useSSL=false</value>
                   <description>JDBC connect string for a JDBC metastore</description>
                 </property>
-                <property>
-                  <name>javax.jdo.option.ConnectionUserName</name>
-                  <value>$getConfigValue(groupName=spark-infrastructure;propertyName=metastore.db.username)</value>
-                  <description>Username to use against metastore database</description>
-                </property>
               </configuration>
   - it: Should override default properties appropriately
     set:
@@ -95,11 +90,6 @@ tests:
                   <value>jdbc:mysql://hive-metastore-db:3306/metastore?createDatabaseIfNotExist=true&amp;allowPublicKeyRetrieval=true&amp;useSSL=false</value>
                   <description>JDBC connect string for a JDBC metastore</description>
                 </property>
-                <property>
-                  <name>javax.jdo.option.ConnectionUserName</name>
-                  <value>$getConfigValue(groupName=spark-infrastructure;propertyName=metastore.db.username)</value>
-                  <description>Username to use against metastore database</description>
-                </property>
                 <property>
                   <name>propertyName1</name>
                   <value>value1</value>
@@ -139,11 +129,6 @@ tests:
                   <value>jdbc:mysql://hive-metastore-db:3306/metastore?createDatabaseIfNotExist=true&amp;allowPublicKeyRetrieval=true&amp;useSSL=false</value>
                   <description>JDBC connect string for a JDBC metastore</description>
                 </property>
-                <property>
-                  <name>javax.jdo.option.ConnectionUserName</name>
-                  <value>$getConfigValue(groupName=spark-infrastructure;propertyName=metastore.db.username)</value>
-                  <description>Username to use against metastore database</description>
-                </property>
                 <property>
                   <name>metastore.thrift.uris</name>
                   <value>thrift://0.0.0.0:8081</value>

extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-hive-metastore-service-chart/values.yaml

@@ -15,16 +15,11 @@ image:
   dockerRepo: "ghcr.io/"
 
 mysql:
-  commonLabels:
-    aissemble-configuration-store: enabled
   enabled: true
   fullnameOverride: "hive-metastore-db"
   auth:
     # The schematool for the metastore service will create the database for us.
     database: "metastore"
-    # Note: Changing these values requires removal of the `hive-metastore-db-0` PVC, or manual modification of the
-    # persisted database.
-    username: $getConfigValue(groupName=spark-infrastructure;propertyName=metastore.db.username)
 
 hive:
   dbType: "mysql"
@@ -63,7 +58,6 @@ service:
 
 # hive-metastore-service Config Map
 configMap:
-  configStore: enabled
   metastoreServiceConfig:
     baseProperties:
       - name: metastore.thrift.uris
@@ -78,7 +72,4 @@ configMap:
       - name: javax.jdo.option.ConnectionURL
         value: jdbc:mysql://hive-metastore-db:3306/metastore?createDatabaseIfNotExist=true&allowPublicKeyRetrieval=true&useSSL=false
         description: JDBC connect string for a JDBC metastore
-      - name: javax.jdo.option.ConnectionUserName
-        value: $getConfigValue(groupName=spark-infrastructure;propertyName=metastore.db.username)
-        description: Username to use against metastore database
     properties: {}

foundation/foundation-archetype/src/main/resources/META-INF/archetype-post-generate.groovy

@@ -13,10 +13,6 @@ import javax.lang.model.SourceVersion
 
 final Logger logger = LoggerFactory.getLogger("com.boozallen.aissemble.foundation.archetype-post-generate")
 
-def dir = new File(new File(request.outputDirectory), request.artifactId)
-file = new File(dir,"deploy.sh")
-file.setExecutable(true, false)
-
 def groupIdRegex = ~'^[a-z][a-z0-9]*(?:\\.[a-z][a-z0-9]*)*$' // lowercase letters, numbers, and periods
 def artifactIdRegex = ~'^[a-z][a-z0-9]*(?:-?[\\da-z]+)*$' // lowercase letters, numbers, and hyphens
 def versionRegex = ~'^(0|[1-9]\\d*)\\.(0|[1-9]\\d*)\\.(0|[1-9]\\d*)(?:-((?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\\+([0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$' // Semantic Versioning

foundation/foundation-archetype/src/main/resources/META-INF/maven/archetype-metadata.xml

@@ -61,7 +61,6 @@
 			<directory/>
 			<includes>
 				<include>Tiltfile</include>
-				<include>deploy.sh</include>
 				<include>.tiltignore</include>
 				<include>devops/**</include>
 				<include>jenkinsPipelineSteps.groovy</include>
@@ -107,20 +106,6 @@
 				</fileSet>
 			</fileSets>
 		</module>
-		<module id="${rootArtifactId}-infrastructure"
-				dir="__rootArtifactId__-infrastructure" name="${rootArtifactId}-infrastructure">
-			<fileSets>
-				<fileSet filtered="true" encoding="UTF-8">
-					<directory/>
-					<includes>
-						<include>*/**</include>
-					</includes>
-					<excludes>
-						<exclude>pom.xml</exclude>
-					</excludes>
-				</fileSet>
-			</fileSets>
-		</module>
 		<module id="${rootArtifactId}-shared"
 				dir="__rootArtifactId__-shared" name="${rootArtifactId}-shared">
 		</module>