Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Changed template resolution logic to support finding multiple templates with the same name.
Changed template rendering to support multiple output documents -- one for each found template.
Changed the build to avoid using fat application jars, enabling multiple adapters to implement templates with the same name.
Details
Occasionally it is useful for two adapters to provide a template with the same name. For example, a Flink adapter and a Spark adapter may both provide
SqlJob.yaml.template
. In such cases, having both adapters in the classpath would result in racy behavior, where nuances in the build would result in only one of the adapters actually being used.Instead, we want adapters to be additive. Having both a Flink and Spark adapter in the classpath should result in both Flink and Spark jobs -- not one or the other. This requires using a normal application distribution -- not a fat jar -- since a fat jar cannot have multiple files with the same name.
This fixes #41 by allowing adapters to just not provide a template. In that case, Resources are rendered to an empty set of YAML files, and the operator does nothing.
Testing Done
There are currently no adapters that provide additive templates as described above, so to test this I added a dummy
SqlJob.yaml.template
to the Kafka adapter in my local build. This meant that both the Kafka adapter and Flink adapter ostensibly implemented SqlJobs. I was able to verify that the operator attempts to create K8s objects corresponding to the output of both templates.