Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#484 Feature: integrate Universal Config store in the hive metastore service #494

Merged
merged 1 commit into from
Dec 9, 2024

Conversation

jaebchoi
Copy link
Contributor

@jaebchoi jaebchoi commented Dec 5, 2024

@jaebchoi jaebchoi force-pushed the 484-integrate-config-store-hive branch from 1aaa91c to bfbc5d4 Compare December 6, 2024 15:55
@jaebchoi jaebchoi force-pushed the 484-integrate-config-store-hive branch from bfbc5d4 to e9503aa Compare December 6, 2024 16:25
@jaebchoi jaebchoi force-pushed the 484-integrate-config-store-hive branch 2 times, most recently from b3c234c to 44c9e26 Compare December 6, 2024 16:32
@@ -15,11 +15,16 @@ image:
dockerRepo: "ghcr.io/"

mysql:
commonLabels:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A: Seems like we might want to be more precise and attach labels only to the resources that need it instead of using this commonLables field.

Copy link
Contributor Author

@jaebchoi jaebchoi Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup! I wish I would inject label for specific to kind, but looking at mysql's logic all metadata's label logic is embedded to use only commonLabels. see here for example code snippet for mysql chart

{{- if eq .Values.architecture "replication" }}
apiVersion: {{ include "common.capabilities.statefulset.apiVersion" . }}
kind: StatefulSet
metadata:
  name: {{ include "mysql.secondary.fullname" . }}
  namespace: {{ include "common.names.namespace" . | quote }}
  labels: {{- include "common.labels.standard" ( dict "customLabels" .Values.commonLabels "context" $ ) | nindent 4 }}
    app.kubernetes.io/component: secondary
  {{- if .Values.commonAnnotations }}
  annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
  {{- end }}
spec:

data:
# Add all the default properties from the local values.yaml to the ConfigMap
# Then check if there are any downstream properties and add them as well
metastore-site.xml: |
{{- $basePropDict := dict }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: Just use one dictionary. Write the base keys first, then write the props. That will automatically cause anything duplicate values in properties to take precedence over baseProperties without complex logic.

Copy link
Contributor

@ewilkins-csi ewilkins-csi Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I'm not really sure why we need these dictionaries at all. Can you not just do:
{{- if not hasProperty .Values.configMap.metastoreServiceConfig.properties $baseProperty.name }}

Copy link
Contributor Author

@jaebchoi jaebchoi Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I'm bit new to helm syntax but from what I looked at the doc it has some limitation in creating variable and function so I had to go with dicts but open to idea and see if there's easy way to do it.

is hasProperty custom method? wasn't sure it was doable, even if it was custom methods,, shouldn't hasProperty need dict to accomplish the logic?

Im trying to accomplish 3 use case,

  1. add baseProperty that doesn't exist in properties
  2. add properties that doesn't exist in baseProperties
  3. Add properties that both exist in baseProperties and properties ( take properties as precedence )

it's bit hard to accomplish contain logic in helm because property schema is list of dict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can def cut follow up ticket if this requires refactoring given that this ticket is past DPC1!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added https://jira.boozallencsn.com/browse/AIOPS-3838 as a follow up and will be working this

@jaebchoi jaebchoi force-pushed the 484-integrate-config-store-hive branch 2 times, most recently from f9e332f to 6ffe878 Compare December 6, 2024 19:08
@jaebchoi jaebchoi force-pushed the 484-integrate-config-store-hive branch 2 times, most recently from 8e8ccd7 to 859c8f5 Compare December 6, 2024 22:06
@jaebchoi jaebchoi force-pushed the 484-integrate-config-store-hive branch from b27ac10 to ddbc3ad Compare December 9, 2024 18:00
@jaebchoi jaebchoi merged commit f46877e into dev Dec 9, 2024
3 checks passed
@jaebchoi jaebchoi deleted the 484-integrate-config-store-hive branch December 9, 2024 19:07
ewilkins-csi added a commit that referenced this pull request Dec 19, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
ewilkins-csi added a commit that referenced this pull request Dec 19, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
ewilkins-csi added a commit that referenced this pull request Dec 20, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
ewilkins-csi added a commit that referenced this pull request Dec 20, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
ewilkins-csi added a commit that referenced this pull request Dec 20, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Dec 23, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Dec 24, 2024
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 6, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 7, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 13, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 13, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 14, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 16, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 22, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
jaebchoi pushed a commit that referenced this pull request Jan 24, 2025
To allow for the dev branch to be released, the changes in this commit
were reverted and moved to this feature branch.  The original work was
done in the following tickets:
 - #484 (ddbc3ad) merged in #494
 - #501 (c817c5f) merged in #504
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants