Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Exceptions] - Exception Modal Part I #70639

Merged
merged 39 commits into from
Jul 8, 2020

Conversation

peluja1012
Copy link
Contributor

@peluja1012 peluja1012 commented Jul 2, 2020

Summary

This PR adds modals for Adding and Editing exception items. These modals will be used in various pages in the app to enable the user to add Exceptions to their rules.

  • When an endpoint exception is added, the modal is pre-populated with exception items most common to the Endpoint use case
  • When adding an exception item, the items are enriched with comments before they're sent to the server. Endpoint exceptions are also enriched with an operating system _tag.
  • When a user edits an item, they can edit the item entries but also have the option to add new items. Handling of deletion of an item while editing will be address in an upcoming PR
  • The modals have checkboxes for "Closing the alert" and for "Bulk close alerts matching exception attributes". When editing or adding exceptions from the exceptions viewer, users will only able to bulk close alerts given that there is no "alert" context.
  • When creating an exception for the first time, a new exception list will be created and a new reference to this list will be added to the corresponding rule.

Coming in Part II

  • Handle case when user deletes an item in the Edit Modal.
  • Implement bulk close. Currently the checkbox doesn't trigger a bulk close action on the server.
  • Filter out fields not included in the list of Endpoint "exceptionable" fields when the user tries to add an Endpoint Exception
  • Allow the user to add Exceptions from the Timeline
  • Disable "Add Endpoint Exception" option for non-Endpoint alerts
  • Add tests for helper functions
  • Add tests for react components if necessary
  • Look into using lists plugin hooks like the persistExceptionItem hook

Usage Examples

Alerts page

image

Rule Details External Alerts tab

image

Rule Details Exceptions tab

image

Adding an exception from the exceptions viewer

image

Adding an exception from an alert event

image

Editing an exception

image

Exception comments

image

Testing

To turn on lists plugin - in kibana.dev.yml

# Enable lists feature
xpack.lists.enabled: true
xpack.lists.listIndex: '.lists-yara'
xpack.lists.listItemIndex: '.items-yara'
  • Navigate to the Alerts page or Rule Details page and select Add Exception or Add Endpoint Exception from the options in the alerts Timeline
  • In the Rule Details page, click on the Exceptions tab, then select Add to endpoint list or Add to detection list from the options in the Exception Viewer header.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@@ -13,6 +13,12 @@ export { useFindLists } from './lists/hooks/use_find_lists';
export { useImportList } from './lists/hooks/use_import_list';
export { useDeleteList } from './lists/hooks/use_delete_list';
export { useExportList } from './lists/hooks/use_export_list';
export {
addExceptionListItem,
updateExceptionListItem,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we need to export these as they are all available in useApi

@@ -190,6 +205,21 @@ export const AlertsTableComponent: React.FC<AlertsTableComponentProps> = ({
[dispatchToaster]
);

const openAddExceptionModalCallback = useCallback(
({ ruleName, ruleId, exceptionListType, alertData }: AddExceptionOnClick) => {
if (alertData !== null && alertData !== undefined) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use != here to check for both null and undefined

setAddExceptionModalState(addExceptionModalInitialState);
}, [setShouldShowAddExceptionModal, setAddExceptionModalState]);

const onAddExceptionCancel = useCallback(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could onAddExceptionCancel and onAddExceptionConfirm be combined into one function like onCloseAddExceptionModal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will look into this in the follow up PR

start={from}
utilityBar={utilityBarCallback}
/>
{shouldShowAddExceptionModal === true && addExceptionModalState.alertData !== null && (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here as before, using != null will catch both undefined and null

@@ -0,0 +1,139 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should just be able to use the usePersistExceptionItem hook from the lists plugin. It determines when to use the post vs put within it. Maybe you can update that existing hook to accept a list of items instead of just 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will look into this in the follow up PR. This hook will eventually need to handle bulk close as well as current alert close also.

@@ -258,6 +258,21 @@ export const RuleDetailsPageComponent: FC<PropsFromRedux> = ({
return null;
}

// TODO: remove this once namespace is finalized as camecased or snakecase
const exceptionListsMeta = useMemo(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working to get that PR in! :) Hope to have it cleaned up EOD.

<EuiFlexGroup gutterSize={'none'}>
<EuiFlexItem grow={false}>
<MyAvatar
name={currentUser !== null ? currentUser.username.toUpperCase() ?? '' : ''} // TODO: Still need to standardize this
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where there wouldn't be a username accessible here? If there isn't one does that mean they don't have certain access rights and would we at that point not let them add an exception at all?

export interface AddExceptionOnClick {
ruleName: string;
ruleId: string;
ruleExceptionLists?: ExceptionListSchema[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be explicit here like ruleExceptionLists: ExceptionListSchema[] | undefined; Because we're using it as a tell for whether the user is creating a new list vs. editing right?

onConfirm();
}, [dispatchToaster, onConfirm]);

const [{ isLoading: addExceptionIsLoading }, addOrUpdateExceptionItems] = useAddOrUpdateException(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned it above, but I think you can just use the existing persist_exception_item.ts hook in the lists plugin.

[setShouldBulkCloseAlert]
);

const enrichExceptionItems = useCallback(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might move some of these functions out into a helpers.ts file so you can unit test them easily.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll look into moving the entire enrichExceptionItems function to the helpers file in the follow up PR.

<EuiText>{i18n.EXCEPTION_BUILDER_INFO}</EuiText>
<EuiSpacer />
<ExceptionBuilder
exceptionListItems={exceptionItemsToAdd}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I might avoid passing this in, since the ExceptionBuilder manages its own state, it just takes an initial list of exception list items to start with. Might be easier to discuss this one offline to see if I missed something.

@peluja1012 peluja1012 marked this pull request as ready for review July 3, 2020 00:07
@peluja1012 peluja1012 requested review from a team as code owners July 3, 2020 00:07
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

return Promise.resolve(newExceptionList);
} catch (error) {
// TODO: properly handle 409 conflict if exception list already exists
return Promise.reject(error);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to handle conflict errors in Part || if the exception list already exists due to stale rule references or race conditions.

_tags: undefined,
tags: undefined,
// TODO: Make this a constant that's shared throughout the app
list_id: exceptionListType === 'endpoint' ? 'endpoint_list' : undefined,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Part II, we'll create a constant and share it with the endpoint exceptions artifact builder code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need to pass a list_id here. It will be generated by the server.

@peluja1012 peluja1012 changed the title [SIEM][Exceptions] - Exception Modal Part I [Security Solution][Exceptions] - Exception Modal Part I Jul 3, 2020
@@ -90,6 +101,10 @@ export const AlertsTableComponent: React.FC<AlertsTableComponentProps> = ({

const [showClearSelectionAction, setShowClearSelectionAction] = useState(false);
const [filterGroup, setFilterGroup] = useState<Status>(FILTER_OPEN);
const [shouldShowAddExceptionModal, setShouldShowAddExceptionModal] = useState(false);
const [addExceptionModalState, setAddExceptionModalState] = useState(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I might add the typing here like - const [addExceptionModalState, setAddExceptionModalState] = useState<AddExceptionOnClick>(addExceptionModalInitialState);

const setIsModalOpen = useCallback(
(isOpen: boolean): void => {
const setCurrentModal = useCallback(
(modalName: string | null): void => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Can we make this an enum for the different modal types?

/>
)}

{currentModal === 'addModal' && exceptionListTypeToEdit !== null && (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I would use the != null to catch null and undefined.

ruleId,
exceptionListType,
onError,
onSuccess,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need the onSuccess or can that be inferred from the return value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need it. Removed.

): Promise<ExceptionListSchema> {
const newExceptionList = await createExceptionList(ruleResponse);

let newExceptionListReferences: ListArray;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think one pattern I've noticed is that we try to avoid using lets and casting. I think here it could be something like:

      const newExceptonListReference = {
        id: newExceptionList.id,
        namespace_type: newExceptionList.namespace_type,
      };
      const newExceptionListReferences: ListArray = [...(ruleResponse.exceptions_list ?? []), newExceptionListReference];

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Updated.

@@ -161,6 +167,10 @@ export const getOperatingSystems = (tags: string[]): string => {
return osMatches;
};

export const getOsTagValues = (tags: string[]): string[] => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this functionally the same as getOperatingSystems?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so. @dplumlee is going to look into dropping the additional formatting from getOperatingSystems in the next PR.

@@ -184,7 +194,7 @@ export const getDescriptionListContent = (
const details = [
{
title: i18n.OPERATING_SYSTEM,
value: getOperatingSystems(exceptionItem._tags),
value: getOperatingSystems(exceptionItem._tags ?? []),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In kibana/x-pack/plugins/lists/common/schemas/request/create_exception_list_item_schema.ts it looks like _tags gets defaulted to an empty array on creation, so we should be good to remove the ?? [].

return undefined;
};

export const entryHasListType = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use the helper export const isListType = (item: BuilderEntry): item is EmptyListEntry => item.type === OperatorTypeEnum.LIST; here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be addressed in follow up PR

if (item != null && item.value != null) {
return item.value;
}
return undefined;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you return an empty array here and then you don't have to do checks like the following and the return type is just string[]: getMappedNonEcsValue({ data: alertData, fieldName: 'file.hash.sha1' }) ?? []

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be addressed in follow up PR

Copy link
Contributor

@yctercero yctercero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Played around with the modal together while debugging a separate issue. Excited to see it all come together! LGTM - thanks for creating a ticket with the list of things to follow up on so they don't slip through the cracks. Some things I would make sure to follow up on is using != null, that'll help avoid missing catching an undefined or null if we try to explicitly check for each separately. Also, following up on the need for list_id on creation of exception lists. Thanks so much!

</ModalHeader>

{fetchOrCreateListError === true && (
<EuiCallOut title={'Error'} color="danger" iconType="alert">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the title needs to be in i18n?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Done.

import { getFormattedComments } from './helpers';

interface AddExceptionCommentsProps {
exceptionItemComments?: Comments[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think we can get rid of the ? here? Comments defaults to an empty array, so it should appear. Or maybe make it explicit Comments[] | undefined if in a not yet created exception item it's not yet defined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking was that this prop would be optional because we use this commend in the Add Modal also where there are no pre-existing comments.

<EuiSpacer />
<ExceptionBuilder
exceptionListItems={initialExceptionItems}
listType={exceptionListType as 'endpoint' | 'detection'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this needs to be casted here. The schema checks should validate that it's either endpoint | detection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, done.

@@ -43,7 +43,7 @@ interface OnChangeProps {
}

interface ExceptionBuilderProps {
exceptionListItems: ExceptionListItemSchema[];
exceptionListItems: ExceptionsBuilderExceptionItem[];
listType: 'detection' | 'endpoint';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I missed this in my PR, but here we can update to use the enum from the lists plugin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually why I was having problems with listType in the modal. I'll update the builder to use the enum.

<ModalBodySection className="builder-section">
<ExceptionBuilder
exceptionListItems={[exceptionItem]}
listType={exceptionListType as 'endpoint' | 'detection'}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as comment above, I think I missed updating the prop to be ExceptionListTypeEnum (from lists plugin). That should remove any need to cast.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -292,7 +309,7 @@ export const getNewExceptionItem = ({
namespaceType,
ruleName,
}: {
listType: 'detection' | 'endpoint';
listType: ExceptionListType;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this in my PR. I would update to use ExceptionListTypeEnum from lists common schema.

...endpointExceptionList,
name: 'new endpoint exception list',
};
const exceptionsListReferences: ListArray = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use the getListMock from x-pack/plugins/security_solution/common/detection_engine/schemas/types/lists.mock.ts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Done.

_tags: undefined,
tags: undefined,
// TODO: Make this a constant that's shared throughout the app
list_id: exceptionListType === 'endpoint' ? 'endpoint_list' : undefined,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need to pass a list_id here. It will be generated by the server.

}

async function fetchRuleExceptionLists(ruleResponse: Rule): Promise<ExceptionListSchema[]> {
const exceptionListReferences = ruleResponse.exceptions_list as ListArray;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the response gets validated, we shouldn't need to do any casting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, this is not needed. It's leftover from when the types weren't finalized yet.

let exceptionListToUse: ExceptionListSchema;
const matchingList = exceptionLists.find((list) => {
if (exceptionListType === 'endpoint') {
return list.type === exceptionListType && list.list_id === 'endpoint_list';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might check with @FrankHassanabad , but almost certain that we shouldn't create a list_id in the client. We should let the server create it and then reference it here using the response from the creation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to think about the right behavior is here when there is no exception list present. We can iterate on it.

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

id value diff baseline
securitySolution 810 +1 809

History

  • 💛 Build #59567 was flaky 1fd2a483c8659504dd113b6617d4c23047acca10
  • 💔 Build #59545 failed 052a80e5b7f4d8b4dde5f0ba9f81e09f99292af3
  • 💔 Build #59074 failed 169cd100a3a461664d85398b8099437060a3d463
  • 💔 Build #59029 failed 08a2f68e8ab050d2ee7b50f372b8d4ac7d8b9094
  • 💔 Build #58748 failed 931bc457e2390583645e1f59d7a2028d88fbef3a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@peluja1012 peluja1012 merged commit 8facae7 into elastic:master Jul 8, 2020
peluja1012 added a commit that referenced this pull request Jul 8, 2020
…1035)

* adds 2 menu items to alert page, progress on exception modal

* adds enriching

* remove unused useExceptionList()

* implements some types

* move add exception modal files

* Exception builder changes to support latest schema

* Changes to lists plugin schemas and fix api bug

Needed to make the schemas more forgiving. Before this change they required name,
description, etc for creation and update.

The update item API was using the wrong url.

* Adding and editing exceptions working

- Modifies add_exception_modal component
- Creates edit_exception_modal component
- Creates shared comments component
- Creates use_add_exception api hook for adding or editing exceptions
- Updates viewer code to support adding and editing exceptions
- Updates alerts table code to use updated version of add_exception_modal

* fixes duplicate types

* updates os tag input

* fixes comment style

* removes checkbox programatically

* grahpql updates to expose exceptions_list

* Add fetch_or_create_exception_list hook

* fixes data population

* refactor use_add_exception hook, add tests

* fix rebase issues, pending updates to edit modal

* fix edit modal and default endpoint exceptions

* adds second checkbox

* adds signal index stuff

* switches boolean logic

* fix some type errors

* remove unnecesary code

* fixes checkbox logic in edit modal

* fixes recursive prop passing

* addresses comments/fixes types

* Revert schema type changes

* type fixes

* fixes regular exception modal

* fix more type errors, remove console log

* fix tests

* move add exception hook, lint

* close alert checkbox closes alert

* address PR comments

* add type to patch rule call, fix ts errors

* fix lint

* fix merge problems after conflict

* Address PR comments

* undo graphql type change

Co-authored-by: Davis Plumlee <davis.plumlee@elastic.co>

Co-authored-by: Davis Plumlee <davis.plumlee@elastic.co>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 8, 2020
* master: (36 commits)
  fixed api url in example plugin (elastic#70934)
  [data.search.aggs]: Remove remaining client dependencies (elastic#70251)
  [Security Solution][Endpoint] Fix base64 download bug and adopt new user artifact/manifest format (elastic#70998)
  [Security Solution][Exceptions] - Exception Modal Part I (elastic#70639)
  [SIEM][Detection Engine][Lists] Adds additional data types to value based lists
  [SIEM][Detection Engine][Lists] Removes feature flag for lists
  [APM] Show license callout in ML settings (elastic#70959)
  Migrate service settings test to jest (elastic#70992)
  [APM] Add cloud attributes to data telemetry (elastic#71008)
  Fix breadcrumb on panels for visibility / round corners (elastic#71010)
  Improve search typescript (elastic#69333)
  [savedObjects field count] run in baseline job (elastic#70999)
  [Security Solution] [Timeline] Timeline manager tweaks (elastic#69988)
  [Endpoint] Support redirect from Policy Details to Ingest when user initiates Edit Policy from Datasource Edit page (elastic#70874)
  [APM] Add API tests (elastic#70740)
  [Security Solution][Exceptions] - Tie server and client code together (elastic#70918)
  [Audit Logging] Add AuditTrail service (elastic#69278)
  [Usage Collection] Ensure no type duplicates (elastic#70946)
  [Security Solution] [Timeline] Bugfix for timeline row actions disappear sometimes (elastic#70958)
  [CI] Add pipeline task queue framework and merge workers into one (elastic#64011)
  ...
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:enhancement Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants