Convert lading configuration from toml to yaml, remove rayon #119
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit makes a breaking change to lading and opens up the 0.6 release series. The major change here is the conversion to the use of yaml in lading's configuration file, driven by the toml crates inability to deal with deserializing enums that have values in them. This means our 'static' format variant is not actually usable in toml-land, as that variant embeds a path in it. I have also taken the opportunity to do a little house cleaning, removing the rayon dependency from file_gen and adjusting the goofy case issues that we've had in configuration for a minute. Now every option will be snake_case and users don't have to deal with the odd Post etc anymore. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
blt
added a commit
to vectordotdev/vector
that referenced
this pull request
Nov 23, 2021
This commit upgrades the soaks to use lading 0.6 -- currently targetting a SHA release -- in preparation for soak testing with static files. The reason an upgrade to lading 0.6 is necessary is outlined in DataDog/lading#119. I have extracted common configuration files upward into `soaks/common/configs`, making the transition easier. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
blt
added a commit
to vectordotdev/vector
that referenced
this pull request
Nov 23, 2021
This commit introduces a new soak to investigate our pipelines in an ongoing basis. The pipeline configuration for vector and the sample data was contributed by work was contributed @vladimir-dd. I have expanded the soak to allow mounting a TESTNAME/data to smuggle static data into the minikube, by which we feed http_gen. Unfortunately this does not work as we cannot use the Virtual Box VM for soak testing and kubernetes/minikube#12301 is open. We will have to figure out some other way to smuggle data into the minikube for this soak to function. This commit depends on #10141 which depends on DataDog/lading#119. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
This was referenced Nov 23, 2021
tobz
approved these changes
Nov 23, 2021
blt
added a commit
to vectordotdev/vector
that referenced
this pull request
Nov 23, 2021
* Upgrade lading This commit upgrades the soaks to use lading 0.6 -- currently targetting a SHA release -- in preparation for soak testing with static files. The reason an upgrade to lading 0.6 is necessary is outlined in DataDog/lading#119. I have extracted common configuration files upward into `soaks/common/configs`, making the transition easier. Signed-off-by: Brian L. Troutwine <brian@troutwine.us> * missed some changes Signed-off-by: Brian L. Troutwine <brian@troutwine.us> * path correction Signed-off-by: Brian L. Troutwine <brian@troutwine.us> * file extensions Signed-off-by: Brian L. Troutwine <brian@troutwine.us> * missed more config changes Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
blt
added a commit
to vectordotdev/vector
that referenced
this pull request
Nov 23, 2021
This commit introduces a new soak to investigate our pipelines in an ongoing basis. The pipeline configuration for vector and the sample data was contributed by work was contributed @vladimir-dd. I have expanded the soak to allow mounting a TESTNAME/data to smuggle static data into the minikube, by which we feed http_gen. Unfortunately this does not work as we cannot use the Virtual Box VM for soak testing and kubernetes/minikube#12301 is open. We will have to figure out some other way to smuggle data into the minikube for this soak to function. This commit depends on #10141 which depends on DataDog/lading#119. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
blt
added a commit
to vectordotdev/vector
that referenced
this pull request
Nov 24, 2021
* Introduce a new http -> pipelines -> blackhole soak This commit introduces a new soak to investigate our pipelines in an ongoing basis. The pipeline configuration for vector and the sample data was contributed by work was contributed @vladimir-dd. I have expanded the soak to allow mounting a TESTNAME/data to smuggle static data into the minikube, by which we feed http_gen. Unfortunately this does not work as we cannot use the Virtual Box VM for soak testing and kubernetes/minikube#12301 is open. We will have to figure out some other way to smuggle data into the minikube for this soak to function. This commit depends on #10141 which depends on DataDog/lading#119. Signed-off-by: Brian L. Troutwine <brian@troutwine.us> * Use configmap hack It turns out that the problem discussed in the last commit is fixed by kubernetes/minikube#13013. This is not present in a minikube release yet, so we can't rely on it. That said, this commit introduces a hack whereby the bootstrap is passed in a configmap. This limits the size of the bootstrap with all the problems that come along with a low-entropy experiment. Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit makes a breaking change to lading and opens up the 0.6 release
series. The major change here is the conversion to the use of yaml in lading's
configuration file, driven by the toml crates inability to deal with
deserializing enums that have values in them. This means our 'static' format
variant is not actually usable in toml-land, as that variant embeds a path in
it.
I have also taken the opportunity to do a little house cleaning, removing the
rayon dependency from file_gen and adjusting the goofy case issues that we've
had in configuration for a minute. Now every option will be snake_case and users
don't have to deal with the odd Post etc anymore.
Signed-off-by: Brian L. Troutwine brian@troutwine.us