From b1eaa7ed18413891866b3b16f9b97b313404522f Mon Sep 17 00:00:00 2001 From: Hanbeom kim Date: Fri, 27 Sep 2024 15:28:48 +0900 Subject: [PATCH] Add packet attribute triage function - Add triage functionality for scoring with attributes of each raw event. - Introduced a new crate attrievent. The kinds of raw events and their attributes change as our software evolves. The purpose of attrievent is to provide a comprehensive list of attributes for both review and the UI simultaneously. - Add the `to_attr_value` to the `Match` trait. - Implement `score_by_attr` under `Match` trait. - Modify the `ValueKind` enum to support different types of input. - Remove the `tor` module file. The structures (`HttpEventFields`, `TorConnection`) and implementations within that module have been moved to `crate::event::http`. - Fix HTTP detection events to consistently use `referrer` instead of `referrer` and `referer` interchangeably. - Change the type of fields in the detection event structure for some raw event. - `post_body`: `Vec` to `String`. - `chaddr`: `Vec` to `String`. - `class_id`: `Vec` to `String`. - `client_id`: `Vec` to `String`. Close: #354 --- CHANGELOG.md | 48 ++- Cargo.toml | 1 + src/event.rs | 23 +- src/event/bootp.rs | 43 ++- src/event/common.rs | 455 +++++++++++++++++++++++++- src/event/conn.rs | 78 ++++- src/event/dcerpc.rs | 12 +- src/event/dhcp.rs | 67 +++- src/event/dns.rs | 52 ++- src/event/ftp.rs | 57 +++- src/event/http.rs | 366 ++++++++++++++++++--- src/event/kerberos.rs | 35 +- src/event/ldap.rs | 51 ++- src/event/log.rs | 12 +- src/event/mqtt.rs | 38 ++- src/event/network.rs | 29 +- src/event/nfs.rs | 28 +- src/event/ntlm.rs | 31 +- src/event/rdp.rs | 41 ++- src/event/smb.rs | 37 ++- src/event/smtp.rs | 33 +- src/event/ssh.rs | 41 ++- src/event/sysmon.rs | 31 +- src/event/tls.rs | 69 +++- src/event/tor.rs | 241 -------------- src/migration.rs | 4 +- src/migration/migration_structures.rs | 16 +- src/tables/triage_policy.rs | 7 +- 28 files changed, 1509 insertions(+), 437 deletions(-) delete mode 100644 src/event/tor.rs diff --git a/CHANGELOG.md b/CHANGELOG.md index 8383618..8cb3d86 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,39 @@ file is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Added + +- Added triage functionality for scoring with attributes of each raw event. + - Introduced a new crate attrievent. The kinds of raw events and their + attributes change as our software evolves. The purpose of attrievent is to + provide a comprehensive list of attributes for both review and the UI + simultaneously. + - Added a new enum type `AttrValue`. This type is used to convert the + attribute value of each raw event to its corresponding type to perform + comparison operations. + - Added the `to_attr_value` to the `Match` trait to generate an `AttrValue` + from the field in all detection event. + - Implemented `score_by_attr` under `Match` trait. + +### Changed + +- Removed the `tor` module file. The structures (`HttpEventFields`, + `TorConnection`) and implementations within that module have been moved to + `crate::event::http`. +- Fixed HTTP detection events to consistently use `referrer` instead of + `referrer` and `referer` interchangeably. +- Modified the `ValueKind` enum to support different types of input for packet + attribute triage. +- Changed the type of fields in the detection event structure for some raw + event. This change allows users to see meaningful values directly without + having to do any special conversion for that field. + - `post_body`(HTTP): `Vec` to `String`. + - `chaddr`(BOOTP): `Vec` to `String`. + - `class_id`(DHCP): `Vec` to `String`. + - `client_id`(DHCP): `Vec` to `String`. + ## [0.33.0] - 2024-12-12 ### Changed @@ -327,7 +360,7 @@ Versioning](https://semver.org/spec/v2.0.0.html). - Changed the return type of `Store::network_map` to `IndexedTable` to enhance security by preventing direct exposure of internal structure. - Replaced `IndexedMap::get_by_id` function with `Indexed::get_by_id`, providing - a more structured and type-safe result. + a more structured and type-safe result. - Previously, the function returned a binary representation of the key-value pair: `Result<(Option>, Option>)>`. - Now, it returns `Result>`, where T is the entry type. @@ -343,8 +376,8 @@ Versioning](https://semver.org/spec/v2.0.0.html). more straightforward and human-readable format compared to the raw binary format exposed by `IndexSet`. - Replaced the `IndexedTable::get`, `IndexedTable::get` and - `IndexedTable::get` method with the more general function - `IndexedTable::get_by_id`. This change enhances flexibility by allowing + `IndexedTable::get` method with the more general function + `IndexedTable::get_by_id`. This change enhances flexibility by allowing retrieval based on any type R rather than being limited to a specific category. Existing code using get for categories should be updated to use get_by_id with the appropriate type. @@ -386,7 +419,7 @@ Versioning](https://semver.org/spec/v2.0.0.html). - Added new functions to facilitate insert, remove, and update operations, ensuring a more controlled and secure template management. - Introduced `Structured`, `Unstructured`, `StructuredClusteringAlgorithm` and - `UnstructuredClusteringAlgorithm` to describe data stored in `Table