Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rollup] Remove builders from MetricConfig #32536

Merged

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Aug 1, 2018

Same motivation as #32507 but for the MetricConfig configuration object.

Related to #29827

@tlrx tlrx added review v7.0.0 >refactoring :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data v6.5.0 labels Aug 1, 2018
@tlrx tlrx requested review from polyfractal and jimczi August 1, 2018 10:06
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validation of the metrics is fragile IMO. It feels wrong to detect a bad metric name in the transport level. @polyfractal I think it's time to use an Enum rather than a String in the metrics config ? Something like:

public enum Metric {
        /**
         * Pick the lowest value.
         */
        MIN(MinAggregationBuilder.NAME),

        /**
         * Pick the highest value.
         */
        MAX(MaxAggregationBuilder.NAME),

        /**
         * Use the average of all values.
         */
        AVG(AvgAggregationBuilder.NAME),

        /**
         * Use the sum of all values.
         */
        SUM(SumAggregationBuilder.NAME),

        /**
         * Use the number of values.
         */
        VALUE_COUNT(ValueCountAggregationBuilder.NAME);

        final String name;

        Metric(String name) {
            this.name = name;
        }
    }

We can add more later but the choice should be constrained to the available ones ?

@tlrx
Copy link
Member Author

tlrx commented Aug 1, 2018

Thanks @jimczi.

The validation of the metrics is fragile IMO. It feels wrong to detect a bad metric name in the transport level.

Indeed, I've been too quick on this. Having an enum is the right solution and I think I saw some todos about this in the rollup codebase. This can be done as a separate change, so maybe I could just reuse the previous code that hard check the metrics names so that this PR is not blocked?

@jimczi
Copy link
Contributor

jimczi commented Aug 1, 2018

sure +1 to keep the previous code and we can update to an enum in a follow up if @polyfractal agrees.

@tlrx
Copy link
Member Author

tlrx commented Aug 1, 2018

Thanks @jimczi. I restored the hard checks for metrics.

@polyfractal
Copy link
Contributor

Yeah, an enum would be cleaner/nicer. As soon as we started hardcoding strings I put in that TODO to stop doing it :)

@@ -125,6 +96,39 @@ public static String getCronString() {
" " + (ESTestCase.randomBoolean() ? "*" : String.valueOf(ESTestCase.randomIntBetween(1970, 2199))); //year
}

public static List<MetricConfig> randomMetricsConfigs(final Random random) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just curious, but why Random as an argument instead of using the test class' random methods (randomBoolean(), etc)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure that ConfigTestHelpers methods were always used in the context of an ESTestCase, and I expect to have to use these methods in the high level rest client too so I found it safer to pass the Random object to the methods similarly to what is done for RandomNumbers or RandomPicks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 cool, thanks for the explanation :)

config.setField("");
e = expectThrows(IllegalArgumentException.class, config::build);
assertThat(e.getMessage(), equalTo("Parameter [field] must be a non-null, non-empty string."));
final String field = randomBoolean() ? "" : null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any advantage in randomizing these? I assumed that since unit tests are fast to execute, we should go ahead and test the obvious paths all the time so that failures aren't flaky.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, Jim made the same remark on another pull request. I removed the randomization.

Copy link
Contributor

@polyfractal polyfractal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@tlrx tlrx force-pushed the remove-builders-from-rollup-metric-group-config branch from 9067600 to a6b710a Compare August 2, 2018 11:19
@tlrx tlrx force-pushed the remove-builders-from-rollup-metric-group-config branch from a6b710a to 645abfb Compare August 2, 2018 18:45
@tlrx tlrx merged commit 937dcfd into elastic:master Aug 3, 2018
@tlrx tlrx deleted the remove-builders-from-rollup-metric-group-config branch August 3, 2018 08:01
@tlrx
Copy link
Member Author

tlrx commented Aug 3, 2018

Thanks @jimczi and @polyfractal

tlrx added a commit that referenced this pull request Aug 3, 2018
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Aug 6, 2018
…pe-detection-with-leading-whitespace

* elastic/master: (34 commits)
  Cross-cluster search: preserve cluster alias in shard failures (elastic#32608)
  Handle AlreadyClosedException when bumping primary term
  [TEST] Allow to run in FIPS JVM (elastic#32607)
  [Test] Add ckb to the list of unsupported languages (elastic#32611)
  SCRIPTING: Move Aggregation Scripts to their own context (elastic#32068)
  Painless: Use LocalMethod Map For Lookup at Runtime (elastic#32599)
  [TEST] Enhance failure message when bulk updates have failures
  [ML] Add ML result classes to protocol library (elastic#32587)
  Suppress LicensingDocumentationIT.testPutLicense in release builds (elastic#32613)
  [Rollup] Update wire version check after backport
  Suppress Wildfly test in FIPS JVMs (elastic#32543)
  [Rollup] Improve ID scheme for rollup documents (elastic#32558)
  ingest: doc: move Dot Expander Processor doc to correct position (elastic#31743)
  [ML] Add some ML config classes to protocol library (elastic#32502)
  [TEST]Split transport verification mode none tests (elastic#32488)
  Core: Move helper date formatters over to java time (elastic#32504)
  [Rollup] Remove builders from DateHistogramGroupConfig (elastic#32555)
  [TEST} unmutes SearchAsyncActionTests and adds debugging info
  [ML] Add Detector config classes to protocol library (elastic#32495)
  [Rollup] Remove builders from MetricConfig (elastic#32536)
  ...
dnhatn added a commit that referenced this pull request Aug 6, 2018
* 6.x:
  [Kerberos] Use canonical host name (#32588)
  Cross-cluster search: preserve cluster alias in shard failures (#32608)
  [TEST] Allow to run in FIPS JVM (#32607)
  Handle AlreadyClosedException when bumping primary term
  [Test] Add ckb to the list of unsupported languages (#32611)
  SCRIPTING: Move Aggregation Scripts to their own context (#32068) (#32629)
  [TEST] Enhance failure message when bulk updates have failures
  [ML] Add ML result classes to protocol library (#32587)
  Suppress LicensingDocumentationIT.testPutLicense in release builds (#32613)
  [Rollup] Improve ID scheme for rollup documents (#32558)
  Mutes failing SQL string function tests due to #32589
  Suppress Wildfly test in FIPS JVMs (#32543)
  Add cluster UUID to Cluster Stats API response (#32206)
  [ML] Add some ML config classes to protocol library (#32502)
  [TEST]Split transport verification mode none tests (#32488)
  [Rollup] Remove builders from DateHistogramGroupConfig (#32555)
  [ML] Add Detector config classes to protocol library (#32495)
  [Rollup] Remove builders from MetricConfig (#32536)
  Fix race between replica reset and primary promotion (#32442)
  HLRC: Move commercial clients from XPackClient (#32596)
  Security: move User to protocol project (#32367)
  Minor fix for javadoc (applicable for java 11). (#32573)
  Painless: Move Some Lookup Logic to PainlessLookup (#32565)
  Core: Minor size reduction for AbstractComponent (#32509)
  INGEST: Enable default pipelines (#32286) (#32591)
  TEST: Avoid merges in testSeqNoAndCheckpoints
  [Rollup] Remove builders from HistoGroupConfig (#32533)
  fixed elements in array of produced terms (#32519)
  Mutes ReindexFailureTests.searchFailure dues to #28053
  Mutes LicensingDocumentationIT due to #32580
  Remove the SATA controller from OpenSUSE box
  [ML] Rename JobProvider to JobResultsProvider (#32551)
dnhatn added a commit that referenced this pull request Aug 6, 2018
* master:
  Cross-cluster search: preserve cluster alias in shard failures (#32608)
  Handle AlreadyClosedException when bumping primary term
  [TEST] Allow to run in FIPS JVM (#32607)
  [Test] Add ckb to the list of unsupported languages (#32611)
  SCRIPTING: Move Aggregation Scripts to their own context (#32068)
  Painless: Use LocalMethod Map For Lookup at Runtime (#32599)
  [TEST] Enhance failure message when bulk updates have failures
  [ML] Add ML result classes to protocol library (#32587)
  Suppress LicensingDocumentationIT.testPutLicense in release builds (#32613)
  [Rollup] Update wire version check after backport
  Suppress Wildfly test in FIPS JVMs (#32543)
  [Rollup] Improve ID scheme for rollup documents (#32558)
  ingest: doc: move Dot Expander Processor doc to correct position (#31743)
  [ML] Add some ML config classes to protocol library (#32502)
  [TEST]Split transport verification mode none tests (#32488)
  Core: Move helper date formatters over to java time (#32504)
  [Rollup] Remove builders from DateHistogramGroupConfig (#32555)
  [TEST} unmutes SearchAsyncActionTests and adds debugging info
  [ML] Add Detector config classes to protocol library (#32495)
  [Rollup] Remove builders from MetricConfig (#32536)
  Tests: Add rolling upgrade tests for watcher (#32428)
  Fix race between replica reset and primary promotion (#32442)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>refactoring :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data v6.5.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants