Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support reading configuration from config files #71

Merged
merged 14 commits into from
Nov 22, 2022

Conversation

romanwozniak
Copy link
Contributor

@romanwozniak romanwozniak commented Nov 17, 2022

Motivation:

As of now, all the configuration values are passed to gojek/mlp via environment variables. With the growing complexity of the application, the complexity of its configuration grows proportionally, so this way of passing config data to the application is no longer convenient. This is because it requires some sort of encoding to pass complex struct/object data (i.e. serializing data as JSON-string. Example: charts/mlp/values.yaml#L47) to the application.

This PR does the necessary refactoring to allow passing runtime configuration to the application via YAML config files, while still making it possible to override sensitive config keys via env variables:

mlp --config config-dev.yaml --config config-overrides.yaml

Changes:

Internally, knadh/koanf is used to parse and deserialize config data. Despite some similarity with spf13/viper, knadh/koanf doesn't lowercase config keys and generally feels like a better-maintained alternative to viper.

Breaking Changes:

Streams/Teams configuration was refactored to represent the Stream > Team hierarchy. Previously Stream and Team were two independent slices of strings, so MLP project can be created with any stream/team combination, which breaks the one-to-many relationship between stream and team.
Now, the Team configuration is completely removed from the top-level configuration and instead, the type of Stream configuration is changed to map[string][]string to hold the mapping between streams and the teams belonging to that stream.
Before:

Streams:
  - stream-a
  - stream-b

Teams:
  - team-a
  - team-b

After:

Streams:
  "stream-a":
    - team-a
  "stream-b":
    - team-b

Applications' UI was updated to support this refactoring too.

@romanwozniak romanwozniak marked this pull request as ready for review November 21, 2022 10:30
Copy link
Collaborator

@krithika369 krithika369 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some questions for clarification. LGTM. Thanks!

.github/workflows/ci-pipeline.yml Show resolved Hide resolved
Makefile Show resolved Hide resolved
@romanwozniak romanwozniak merged commit ac5d3eb into caraml-dev:main Nov 22, 2022
@romanwozniak romanwozniak deleted the config-refactoring branch December 3, 2022 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants