[security solutions][lists] Adds end to end tests part 1 #74473

FrankHassanabad · 2020-08-06T04:21:10Z

Summary

Adds initial set of end to end tests for lists

You can run all of these with the command from kibana root:

node scripts/functional_tests --config x-pack/test/lists_api_integration/security_and_spaces/config.ts

Fixes a few minor bugs found such as...

Validation for importing lists was not checking if the indexes were created first
Some wording for the error messages had duplicate words within them

This is adds tests for:

create_list_items
create_list
delete_list_items
export_list_items
find_list_items
find_lists
import_list_items
read_list_items
read_lists
update_list_items
update_lists

This does not yet add tests for:

Exception lists
some value list features such as serializer and deseralizer

Checklist

Unit or functional tests were updated or added to match the most common scenarios

madirey · 2020-08-06T18:53:04Z

x-pack/test/lists_api_integration/security_and_spaces/tests/read_lists.ts

+
+      it('should be able to read a single list using id', async () => {
+        // create a simple list to read
+        await supertest


Is the preference to do this, rather than using fixtures? Ideally we'd keep the scope of this test small and just test reads (this is, in effect, testing both create and read), but on the other hand... maintenance of fixtures can be "not fun." Just curious if there's ever been any discussion as to what the standard should be ... I see a good mix of both across the different plugins. FWIW, we used a bit of a mix for the endpoint artifact packaging code, but any setup calls were done in before*.

There hasn't been much discussion, I haven't used fixtures before other than other people's.

So far this has been simple to maintain and when things blow up it hasn't been hard to trace it down to the right spot. I'm not against learning fixtures though. Seems like it might be more maintenance like the issues with loading saved objects from esArchiver has been a bit of a pain point for cypress on the front end.

Overall, I have been maintaining the detection engine tests using mostly REST setup, tear downs, and the only maintenance has been two suite skips due to networking issues we solved by using retry loops (which would still have happened with fixtures or dummy data).

I agree it can make maintenance a little more painful. Carry on. :)

…s, and adds some missing unit tests

yctercero · 2020-08-10T14:04:51Z

x-pack/plugins/lists/common/schemas/request/update_list_schema.mock.ts

+});
+
+/**
+ * Useful for end to end tests and other mechanisms which want to fill in the values


This comment might be useful in the mocks added in create_list_schema.mock.ts too?

Yeah, let me change and add it there.

yctercero · 2020-08-10T14:08:01Z

x-pack/test/lists_api_integration/common/config.ts

+}
+
+export function createTestConfig(name: string, options: CreateTestConfigOptions) {
+  const { license = 'trial', disabledPlugins = [], ssl = false } = options;


Since these are defaulted in x-pack/test/lists_api_integration/security_and_spaces/config.ts does it need to be defaulted here too?

No, from a technical standpoint it does not, it just follows the other patterns where a default is given and then everything still overrides it. I will probably keep it for the sake of the other examples in the other plugins doing it the same way but fwiw.

Later if we introduce tests only for basic license rather than trial which gives us full platinum access then that config.ts would look like this:

export default createTestConfig('security_and_spaces', { license: 'basic' });

yctercero · 2020-08-10T14:13:18Z

x-pack/test/lists_api_integration/utils.ts

+
+/**
+ * Creates the lists index for use inside of beforeEach blocks of tests
+ * This will retry 20 times before giving up and hopefully still not interfere with other tests


Just curious - did we find that it took around 20 attempts to ensure the test hit at least once, or is this just more an arbitrary number?

Arbitrary number, figured I would increase that default if we hit the errors again which we haven't luckily.

yctercero

LGTM - thanks for setting up all these tests. Just left super nit comments, nothing related to functionality.

…ng able to import when there is not indexes created first

## Summary Adds initial set of end to end tests for lists You can run all of these with the command from kibana root: ```ts node scripts/functional_tests --config x-pack/test/lists_api_integration/security_and_spaces/config.ts ``` Fixes a few minor bugs found such as... * Validation for importing lists was not checking if the indexes were created first * Some wording for the error messages had duplicate words within them ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

* master: (106 commits) [Functional Tests] Adds a wait time between setting the index pattern and the time field on TSVB (elastic#74736) [Lens] Add styling options for x and y axes on the settings popover (elastic#71829) [Maps] add initial location option that fits to data bounds (elastic#74583) theme function (elastic#73451) [data.ui.query] Write filters to query log from default editor. (elastic#74474) [data.search.SearchSource] Move some SearchSource dependencies to the server. (elastic#74607) [Canvas][tech-debt] Convert renderers (elastic#74134) [security solutions][lists] Adds end to end tests (elastic#74473) pluralized for occurrences vs occurrence (elastic#74564) Update links that pointed to CONTRIBUTING.md (elastic#74757) [Ingest pipelines] Implement tabs in processor flyout (elastic#74469) [Event log] Use Alerts client & Actions client when fetching these types of SOs (elastic#73257) Bump chalk to 4.1.0 (elastic#73397) Index pattern field list - transition away from extending array - introduce and use getAll() (elastic#74718) [SECURITY] Bugs css/inspect (elastic#74711) [telemetry] update README to downplay ui_metrics (elastic#74635) Fixed grammar (elastic#74725) [Telemetry][API Integration] size_in_bytes to be a number (elastic#74664) [ILM] Convert node details flyout to TS (elastic#73707) [Ingest Node Pipelines] Sentence-case processor names (elastic#74645) ...

kibanamachine · 2020-08-11T17:21:11Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 940d582

Build metrics

✅ unchanged

History

💚 Build #67421 succeeded 940d582
💚 Build #67382 succeeded f91765b
💚 Build #67339 succeeded c8aec74
💚 Build #67079 succeeded ec64d17
💚 Build #66821 succeeded 55bc2ad

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

## Summary Adds initial set of end to end tests for lists You can run all of these with the command from kibana root: ```ts node scripts/functional_tests --config x-pack/test/lists_api_integration/security_and_spaces/config.ts ``` Fixes a few minor bugs found such as... * Validation for importing lists was not checking if the indexes were created first * Some wording for the error messages had duplicate words within them ### Checklist - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

…nes/processor-forms-a-d * 'master' of github.com:elastic/kibana: (25 commits) [ML] Removing full lodash library imports (elastic#74742) [Search] Server strategy example (elastic#71679) [Reporting] Fix and test for Listing of Reports (elastic#74453) [maps] fix drawing shapes (elastic#74689) [Resolver] Improve simulator. Add more click-through tests and panel tests. (elastic#74601) Deprecate schema-less specs in Vega (elastic#73805) [Security Solution] Rename Administration > Hosts subtab to Endpoints (elastic#74287) Timelion deprecation doc (elastic#74508) [Functional Tests] Adds a wait time between setting the index pattern and the time field on TSVB (elastic#74736) [Lens] Add styling options for x and y axes on the settings popover (elastic#71829) [Maps] add initial location option that fits to data bounds (elastic#74583) theme function (elastic#73451) [data.ui.query] Write filters to query log from default editor. (elastic#74474) [data.search.SearchSource] Move some SearchSource dependencies to the server. (elastic#74607) [Canvas][tech-debt] Convert renderers (elastic#74134) [security solutions][lists] Adds end to end tests (elastic#74473) pluralized for occurrences vs occurrence (elastic#74564) Update links that pointed to CONTRIBUTING.md (elastic#74757) [Ingest pipelines] Implement tabs in processor flyout (elastic#74469) [Event log] Use Alerts client & Actions client when fetching these types of SOs (elastic#73257) ... # Conflicts: # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/field_components/text_editor.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/manage_processor_form.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/append.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/bytes.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/circle.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/common_fields/field_name_field.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/common_fields/ignore_missing_field.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/convert.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/csv.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/date.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/date_index_name.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/dissect.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/dot_expander.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/drop.tsx # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/index.ts # x-pack/plugins/ingest_pipelines/public/application/components/pipeline_processors_editor/components/manage_processor_form/processors/shared.ts

FrankHassanabad added 2 commits August 3, 2020 11:15

Initial addition of end to end tests for lists

1d645f5

Initial end to end test setup

8a92670

FrankHassanabad requested review from a team as code owners August 6, 2020 04:21

Merge branch 'master' into initial-lists-endtoend

216220a

FrankHassanabad self-assigned this Aug 6, 2020

FrankHassanabad added release_note:skip Skip the PR/issue when compiling release notes v8.0.0 v7.10.0 labels Aug 6, 2020

FrankHassanabad added 3 commits August 5, 2020 22:35

Adds more basics of the tests for lists

7d1d940

Fixes lint issue and adds a base read_lists end to end test

55bc2ad

Merge branch 'master' into initial-lists-endtoend

cc685d0

madirey reviewed Aug 6, 2020

View reviewed changes

Adds more end to end tests, fixes some message bugs in the REST route…

ec64d17

…s, and adds some missing unit tests

yctercero reviewed Aug 10, 2020

View reviewed changes

yctercero approved these changes Aug 10, 2020

View reviewed changes

FrankHassanabad added 9 commits August 10, 2020 11:55

Added more end to end tests with list items

52920f5

Fixed one bug with some refactoring and failed test

6ca0244

Merge branch 'master' into initial-lists-endtoend

c8aec74

Added more end to end tests for lists

055b219

Merge branch 'master' into initial-lists-endtoend

f91765b

Updated tests to have import and fixed a few bugs with the import bei…

acc8a42

…ng able to import when there is not indexes created first

Merge branch 'master' into initial-lists-endtoend

11c0ebb

Added find_list_items unit tests

3e0e4bf

Added export_list_items end to end tests

940d582

FrankHassanabad merged commit 461d684 into elastic:master Aug 11, 2020

FrankHassanabad deleted the initial-lists-endtoend branch August 11, 2020 15:25

FrankHassanabad changed the title ~~[security solutions][lists] Adds end to end tests~~ [security solutions][lists] Adds end to end tests part 1 Aug 11, 2020

FrankHassanabad mentioned this pull request Aug 11, 2020

[7.x] [security solutions][lists] Adds end to end tests (#74473) #74768

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[security solutions][lists] Adds end to end tests part 1 #74473

[security solutions][lists] Adds end to end tests part 1 #74473

FrankHassanabad commented Aug 6, 2020 •

edited

Loading

madirey Aug 6, 2020

FrankHassanabad Aug 6, 2020 •

edited

Loading

madirey Aug 10, 2020

yctercero Aug 10, 2020

FrankHassanabad Aug 10, 2020

yctercero Aug 10, 2020

FrankHassanabad Aug 10, 2020

yctercero Aug 10, 2020

FrankHassanabad Aug 10, 2020

yctercero left a comment

kibanamachine commented Aug 11, 2020

[security solutions][lists] Adds end to end tests part 1 #74473

[security solutions][lists] Adds end to end tests part 1 #74473

Conversation

FrankHassanabad commented Aug 6, 2020 • edited Loading

Summary

Checklist

Choose a reason for hiding this comment

FrankHassanabad Aug 6, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yctercero left a comment

Choose a reason for hiding this comment

kibanamachine commented Aug 11, 2020

💚 Build Succeeded

Build metrics

History

FrankHassanabad commented Aug 6, 2020 •

edited

Loading

FrankHassanabad Aug 6, 2020 •

edited

Loading