Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Multi-line string fields diff algorithm test plan #188323

Merged
merged 4 commits into from
Jul 23, 2024
+68 −24
Merged

[Security Solution] Multi-line string fields diff algorithm test plan #188323

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2057ae8
writes test plan
dplumlee Jul 15, 2024
cac1242
add missing version test cases
dplumlee Jul 17, 2024
b2c7a55
Merge branch 'main' into multi-line-string-diff-test-plan
banderror Jul 19, 2024
c21678c
Merge branch 'main' into multi-line-string-diff-test-plan
jpdjere Jul 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 68 additions & 24 deletions ...sting/test_plans/detection_response/prebuilt_rules/upgrade_review_algorithms.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Status: `in progress`.
- [**Scenario: `ABB` - Rule field is any type**](#scenario-abb---rule-field-is-any-type)
- [Rule field has an update and a custom value that are NOT the same - `ABC`](#rule-field-has-an-update-and-a-custom-value-that-are-not-the-same---abc)
- [**Scenario: `ABC` - Rule field is a number or single line string**](#scenario-abc---rule-field-is-a-number-or-single-line-string)
- [**Scenario: `ABC` - Rule field is a mergable multi line string**](#scenario-abc---rule-field-is-a-mergable-multi-line-string)
- [**Scenario: `ABC` - Rule field is a non-mergable multi line string**](#scenario-abc---rule-field-is-a-non-mergable-multi-line-string)
- [**Scenario: `ABC` - Rule field is an array of scalar values**](#scenario-abc---rule-field-is-an-array-of-scalar-values)
- [Rule field has an update and a custom value that are the same and the rule base version doesn't exist - `-AA`](#rule-field-has-an-update-and-a-custom-value-that-are-the-same-and-the-rule-base-version-doesnt-exist----aa)
- [**Scenario: `-AA` - Rule field is any type**](#scenario--aa---rule-field-is-any-type)
Expand Down Expand Up @@ -61,7 +63,7 @@ Status: `in progress`.

#### **Scenario: `AAA` - Rule field is any type**

**Automation**: 3 integration tests with mock rules + a set of unit tests for each algorithm
**Automation**: 4 integration tests with mock rules + a set of unit tests for each algorithm

```Gherkin
Given <field_name> field is not customized by the user (current version == base version)
Expand All @@ -71,10 +73,11 @@ And <field_name> field should not be returned from the `upgrade/_review` API end
And <field_name> field should not be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "A" | "A" | "A" |
| number | risk_score | 1 | 1 | 1 | 1 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "three", "two"] | ["three", "one", "two"] | ["one", "three", "two"] |
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "A" | "A" | "A" |
| multi line string | description | "My description.\nThis is a second line." | "My description.\nThis is a second line." | "My description.\nThis is a second line." | "My description.\nThis is a second line." |
| number | risk_score | 1 | 1 | 1 | 1 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "three", "two"] | ["three", "one", "two"] | ["one", "three", "two"] |
```

### Rule field doesn't have an update but has a custom value - `ABA`
Expand All @@ -91,10 +94,11 @@ And <field_name> field should be returned from the `upgrade/_review` API endpoin
And <field_name> field should be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "B" | "A" | "B" |
| number | risk_score | 1 | 2 | 1 | 2 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "two", "four"] | ["one", "two", "three"] | ["one", "two", "four"] |
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "B" | "A" | "B" |
| multi line string | description | "My description.\nThis is a second line." | "My GREAT description.\nThis is a second line." | "My description.\nThis is a second line." | "My GREAT description.\nThis is a second line." |
| number | risk_score | 1 | 2 | 1 | 2 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "two", "four"] | ["one", "two", "three"] | ["one", "two", "four"] |
```

### Rule field has an update and doesn't have a custom value - `AAB`
Expand All @@ -111,10 +115,11 @@ And <field_name> field should be returned from the `upgrade/_review` API endpoin
And <field_name> field should be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "A" | "B" | "B" |
| number | risk_score | 1 | 1 | 2 | 2 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "two", "three"] | ["one", "two", "four"] | ["one", "two", "four"] |
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "A" | "B" | "B" |
| multi line string | description | "My description.\nThis is a second line." | "My description.\nThis is a second line." | "My GREAT description.\nThis is a second line." | "My GREAT description.\nThis is a second line." |
| number | risk_score | 1 | 1 | 2 | 2 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "two", "three"] | ["one", "two", "four"] | ["one", "two", "four"] |
```

### Rule field has an update and a custom value that are the same - `ABB`
Expand All @@ -132,10 +137,11 @@ And <field_name> field should be returned from the `upgrade/_review` API endpoin
And <field_name> field should be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "B" | "B" | "B" |
| number | risk_score | 1 | 2 | 2 | 2 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "two", "four"] | ["one", "two", "four"] | ["one", "two", "four"] |
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | "A" | "B" | "B" | "B" |
| multi line string | description | "My description.\nThis is a second line." | "My GREAT description.\nThis is a second line." | "My GREAT description.\nThis is a second line." | "My GREAT description.\nThis is a second line." |
| number | risk_score | 1 | 2 | 2 | 2 |
| array of scalars | tags | ["one", "two", "three"] | ["one", "two", "four"] | ["one", "two", "four"] | ["one", "two", "four"] |
```

### Rule field has an update and a custom value that are NOT the same - `ABC`
Expand All @@ -158,6 +164,42 @@ Examples:
| number | risk_score | 1 | 2 | 3 | 2 |
```

#### **Scenario: `ABC` - Rule field is a mergable multi line string**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithms

```Gherkin
Given <field_name> field is customized by the user (current version != base version)
And <field_name> field is updated by Elastic in this upgrade (target version != base version)
And customized <field_name> field is different than the Elastic update in this upgrade (current version != target version)
And the 3-way diff of <field_name> fields are determined to be mergable
Then for <field_name> field the diff algorithm should output a merged version as the merged one with a solvable conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| multi line string | description | "My description.\nThis is a second line." | "My GREAT description.\nThis is a second line." | "My description.\nThis is a second line, now longer." | "My GREAT description.\nThis is a second line, now longer." |
```

#### **Scenario: `ABC` - Rule field is a non-mergable multi line string**

**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithms

```Gherkin
Given <field_name> field is customized by the user (current version != base version)
And <field_name> field is updated by Elastic in this upgrade (target version != base version)
And customized <field_name> field is different than the Elastic update in this upgrade (current version != target version)
And the 3-way diff of <field_name> fields are determined to be unmergable
Then for <field_name> field the diff algorithm should output the current version as the merged one with a non-solvable conflict
And <field_name> field should be returned from the `upgrade/_review` API endpoint
And <field_name> field should be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| multi line string | description | "My description.\nThis is a second line." | "My GREAT description.\nThis is a third line." | "My EXCELLENT description.\nThis is a fourth." | "My GREAT description.\nThis is a third line." |
```

#### **Scenario: `ABC` - Rule field is an array of scalar values**

**Automation**: 5 integration tests with mock rules + a set of unit tests for the algorithm
Expand Down Expand Up @@ -198,10 +240,11 @@ And <field_name> field should not be returned from the `upgrade/_review` API end
And <field_name> field should not be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | N/A | "A" | "A" | "A" |
| number | risk_score | N/A | 1 | 1 | 1 |
| array of scalars | tags | N/A | ["one", "three", "two"] | ["three", "one", "two"] | ["one", "three", "two"] |
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | N/A | "A" | "A" | "A" |
| multi line string | description | N/A | "My description.\nThis is a second line." | "My description.\nThis is a second line." | "My description.\nThis is a second line." |
| number | risk_score | N/A | 1 | 1 | 1 |
| array of scalars | tags | N/A | ["one", "three", "two"] | ["three", "one", "two"] | ["one", "three", "two"] |
```

### Rule field has an update and a custom value that are NOT the same and the rule base version doesn't exist - `-BC`
Expand All @@ -219,9 +262,10 @@ And <field_name> field should be returned from the `upgrade/_review` API endpoin
And <field_name> field should be shown in the upgrade preview UI

Examples:
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | N/A | "B" | "C" | "C" |
| number | risk_score | N/A | 2 | 3 | 3 |
| algorithm | field_name | base_version | current_version | target_version | merged_version |
| single line string | name | N/A | "B" | "C" | "C" |
| multi line string | description | N/A | "My description.\nThis is a second line." | "My GREAT description.\nThis is a second line." | "My GREAT description.\nThis is a second line." |
| number | risk_score | N/A | 2 | 3 | 3 |
```

#### **Scenario: `-BC` - Rule field is an array of scalar values**
Expand Down