Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Data Frame Analytics: adds functional tests for runtime fields support #96262

Merged
merged 2 commits into from
Apr 6, 2021
+133 −1
Merged

[ML] Data Frame Analytics: adds functional tests for runtime fields support #96262

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a7f7905
add basic runtime mapping functional tests to analyics
alvarezmelissa87 Apr 5, 2021
68f8857
confirm runtime mapping added correctly.
alvarezmelissa87 Apr 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ...frame_analytics/pages/analytics_creation/components/runtime_mappings/runtime_mappings.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ const COPY_TO_CLIPBOARD_RUNTIME_MAPPINGS = i18n.translate(

const { useXJsonMode } = XJson;
const xJsonMode = new XJsonMode();
export type XJsonModeType = ReturnType<typeof XJsonMode>;

interface Props {
actions: CreateAnalyticsFormProps['actions'];
Expand Down
3 changes: 2 additions & 1 deletion ...nalytics/pages/analytics_creation/components/runtime_mappings/runtime_mappings_editor.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@ import React, { memo, FC } from 'react';
import { EuiCodeEditor } from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { isRuntimeMappings } from '../../../../../../../common/util/runtime_field_utils';
import { XJsonModeType } from './runtime_mappings';

interface Props {
convertToJson: (data: string) => string;
setAdvancedRuntimeMappingsConfig: React.Dispatch<string>;
setIsRuntimeMappingsEditorApplyButtonEnabled: React.Dispatch<React.SetStateAction<boolean>>;
advancedEditorRuntimeMappingsLastApplied: string | undefined;
advancedRuntimeMappingsConfig: string;
xJsonMode: any;
xJsonMode: XJsonModeType;
}

export const RuntimeMappingsEditor: FC<Props> = memo(
Expand Down
22 changes: 22 additions & 0 deletions x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ export default function ({ getService }: FtrProviderContext) {
get destinationIndex(): string {
return `user-${this.jobId}`;
},
runtimeFields: {
uppercase_y: {
type: 'keyword',
script: 'emit(params._source.y.toUpperCase())',
},
},
dependentVariable: 'y',
trainingPercent: 20,
modelMemory: '60mb',
Expand Down Expand Up @@ -92,6 +98,22 @@ export default function ({ getService }: FtrProviderContext) {
await ml.dataFrameAnalyticsCreation.assertJobTypeSelectExists();
await ml.dataFrameAnalyticsCreation.selectJobType(testData.jobType);

await ml.testExecution.logTestStep('displays the runtime mappings editor switch');
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingSwitchExists();

await ml.testExecution.logTestStep('enables the runtime mappings editor');
await ml.dataFrameAnalyticsCreation.toggleRuntimeMappingsEditorSwitch(true);
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingsEditorContent(['']);

await ml.testExecution.logTestStep('sets runtime mappings');
await ml.dataFrameAnalyticsCreation.setRuntimeMappingsEditorContent(
JSON.stringify(testData.runtimeFields)
);
await ml.dataFrameAnalyticsCreation.applyRuntimeMappings();
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingsEditorContent([
'{"uppercase_y":{"type":"keyword","script":"emit(params._source.y.toUpperCase())"}}',
]);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate that uppercase_y is showing up correctly in the wizard after this button is applied?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked in 68f8857

await ml.testExecution.logTestStep('inputs the dependent variable');
await ml.dataFrameAnalyticsCreation.assertDependentVariableInputExists();
await ml.dataFrameAnalyticsCreation.selectDependentVariable(testData.dependentVariable);
Expand Down
22 changes: 22 additions & 0 deletions x-pack/test/functional/apps/ml/data_frame_analytics/outlier_detection_creation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ export default function ({ getService }: FtrProviderContext) {
get destinationIndex(): string {
return `user-${this.jobId}`;
},
runtimeFields: {
lowercase_central_air: {
type: 'keyword',
script: 'emit(params._source.CentralAir.toLowerCase())',
},
},
modelMemory: '5mb',
createIndexPattern: true,
expected: {
Expand Down Expand Up @@ -106,6 +112,22 @@ export default function ({ getService }: FtrProviderContext) {
await ml.dataFrameAnalyticsCreation.assertJobTypeSelectExists();
await ml.dataFrameAnalyticsCreation.selectJobType(testData.jobType);

await ml.testExecution.logTestStep('displays the runtime mappings editor switch');
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingSwitchExists();

await ml.testExecution.logTestStep('enables the runtime mappings editor');
await ml.dataFrameAnalyticsCreation.toggleRuntimeMappingsEditorSwitch(true);
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingsEditorContent(['']);

await ml.testExecution.logTestStep('sets runtime mappings');
await ml.dataFrameAnalyticsCreation.setRuntimeMappingsEditorContent(
JSON.stringify(testData.runtimeFields)
);
await ml.dataFrameAnalyticsCreation.applyRuntimeMappings();
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingsEditorContent([
'{"lowercase_central_air":{"type":"keyword","script":"emit(params._source.CentralAir.toLowerCase())"}}',
]);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate that lowercase_central_air is showing up correctly in the wizard after this button is applied?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably have a check either in this test or another dedicated test if for example, the histogram charts for runtime fields like lowercase_central_air is showing up correctly once the mappings is applied.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking in 68f8857
Agreed - this is a basic runtime fields tests. I'll add in more advanced ones in a follow up

await ml.testExecution.logTestStep('does not display the dependent variable input');
await ml.dataFrameAnalyticsCreation.assertDependentVariableInputMissing();

Expand Down
22 changes: 22 additions & 0 deletions x-pack/test/functional/apps/ml/data_frame_analytics/regression_creation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ export default function ({ getService }: FtrProviderContext) {
get destinationIndex(): string {
return `user-${this.jobId}`;
},
runtimeFields: {
uppercase_stab: {
type: 'keyword',
script: 'emit(params._source.stabf.toUpperCase())',
},
},
dependentVariable: 'stab',
trainingPercent: 20,
modelMemory: '20mb',
Expand Down Expand Up @@ -85,6 +91,22 @@ export default function ({ getService }: FtrProviderContext) {
await ml.dataFrameAnalyticsCreation.assertJobTypeSelectExists();
await ml.dataFrameAnalyticsCreation.selectJobType(testData.jobType);

await ml.testExecution.logTestStep('displays the runtime mappings editor switch');
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingSwitchExists();

await ml.testExecution.logTestStep('enables the runtime mappings editor');
await ml.dataFrameAnalyticsCreation.toggleRuntimeMappingsEditorSwitch(true);
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingsEditorContent(['']);

await ml.testExecution.logTestStep('sets runtime mappings');
await ml.dataFrameAnalyticsCreation.setRuntimeMappingsEditorContent(
JSON.stringify(testData.runtimeFields)
);
await ml.dataFrameAnalyticsCreation.applyRuntimeMappings();
await ml.dataFrameAnalyticsCreation.assertRuntimeMappingsEditorContent([
'{"uppercase_stab":{"type":"keyword","script":"emit(params._source.stabf.toUpperCase())"}}',
]);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate that uppercase_stab is showing up correctly in the wizard after this button is applied?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call - checked in 68f8857

await ml.testExecution.logTestStep('inputs the dependent variable');
await ml.dataFrameAnalyticsCreation.assertDependentVariableInputExists();
await ml.dataFrameAnalyticsCreation.selectDependentVariable(testData.dependentVariable);
Expand Down
64 changes: 64 additions & 0 deletions x-pack/test/functional/services/ml/data_frame_analytics_creation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export function MachineLearningDataFrameAnalyticsCreationProvider(
const testSubjects = getService('testSubjects');
const comboBox = getService('comboBox');
const retry = getService('retry');
const aceEditor = getService('aceEditor');

return {
async assertJobTypeSelectExists() {
Expand Down Expand Up @@ -236,6 +237,69 @@ export function MachineLearningDataFrameAnalyticsCreationProvider(
);
},

async assertRuntimeMappingSwitchExists() {
await testSubjects.existOrFail('mlDataFrameAnalyticsRuntimeMappingsEditorSwitch');
},

async assertRuntimeMappingEditorExists() {
await testSubjects.existOrFail('mlDataFrameAnalyticsAdvancedRuntimeMappingsEditor');
},

async assertRuntimeMappingsEditorSwitchCheckState(expectedCheckState: boolean) {
const actualCheckState = await this.getRuntimeMappingsEditorSwitchCheckedState();
expect(actualCheckState).to.eql(
expectedCheckState,
`Advanced runtime mappings editor switch check state should be '${expectedCheckState}' (got '${actualCheckState}')`
);
},

async getRuntimeMappingsEditorSwitchCheckedState(): Promise<boolean> {
const subj = 'mlDataFrameAnalyticsRuntimeMappingsEditorSwitch';
const isSelected = await testSubjects.getAttribute(subj, 'aria-checked');
return isSelected === 'true';
},

async toggleRuntimeMappingsEditorSwitch(toggle: boolean) {
const subj = 'mlDataFrameAnalyticsRuntimeMappingsEditorSwitch';
if ((await this.getRuntimeMappingsEditorSwitchCheckedState()) !== toggle) {
await retry.tryForTime(5 * 1000, async () => {
await testSubjects.clickWhenNotDisabled(subj);
await this.assertRuntimeMappingsEditorSwitchCheckState(toggle);
});
}
},

async setRuntimeMappingsEditorContent(input: string) {
await aceEditor.setValue('mlDataFrameAnalyticsAdvancedRuntimeMappingsEditor', input);
},

async assertRuntimeMappingsEditorContent(expectedContent: string[]) {
await this.assertRuntimeMappingEditorExists();

const runtimeMappingsEditorString = await aceEditor.getValue(
'mlDataFrameAnalyticsAdvancedRuntimeMappingsEditor'
);
// Not all lines may be visible in the editor and thus aceEditor may not return all lines.
// This means we might not get back valid JSON so we only test against the first few lines
// and see if the string matches.
const splicedAdvancedEditorValue = runtimeMappingsEditorString.split('\n').splice(0, 3);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't have to be a part of this PR, but since this piece is now used multiple times across different parts of ml/transform, I wonder if it would be beneficial for us to move this chunk into MachineLearningCommonUIProvider.

expect(splicedAdvancedEditorValue).to.eql(
expectedContent,
`Expected the first editor lines to be '${expectedContent}' (got '${splicedAdvancedEditorValue}')`
);
},

async applyRuntimeMappings() {
const subj = 'mlDataFrameAnalyticsRuntimeMappingsApplyButton';
await testSubjects.existOrFail(subj);
await testSubjects.clickWhenNotDisabled(subj);
const isEnabled = await testSubjects.isEnabled(subj);
expect(isEnabled).to.eql(
false,
`Expected runtime mappings 'Apply changes' button to be disabled, got enabled.`
);
},

async assertDependentVariableSelection(expectedSelection: string[]) {
await this.waitForDependentVariableInputLoaded();
const actualSelection = await comboBox.getComboBoxSelectedOptions(
Expand Down