feat(dump) - extract, decode and write MFT streams

[FranticTyping]

Overview

Our current MFT support doesn't look at data streams stored in MFT entries. These data streams are a goldmine of information for Incident Responders.

This PR aims to introduce three key features:

1. Chainsaw is able to extract MFT data streams, decode them and include them in the YAML and JSON outputs.
2. Chainsaw is able to extract MFT data streams, decode them and write them to disk
3. Chainsaw is able to apply detection logic to data streams

This PR should address issues: #190 #191

Examples

Data Streams in YAML Output

By default, Chainsaw will now extract data streams and include them in the default output. Stream names and values are shown, with values being in a hexstring format:

➜  release ./chainsaw dump ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT
 ██████╗██╗  ██╗ █████╗ ██╗███╗   ██╗███████╗ █████╗ ██╗    ██╗
██╔════╝██║  ██║██╔══██╗██║████╗  ██║██╔════╝██╔══██╗██║    ██║
██║     ███████║███████║██║██╔██╗ ██║███████╗███████║██║ █╗ ██║
██║     ██╔══██║██╔══██║██║██║╚██╗██║╚════██║██╔══██║██║███╗██║
╚██████╗██║  ██║██║  ██║██║██║ ╚████║███████║██║  ██║╚███╔███╔╝
 ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ ╚══╝╚══╝
    By WithSecure Countercept (@FranticTyping, @AlexKornitzer)

[+] Dumping the contents of forensic artefacts from: ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT(extensions: *)
[+] Loaded 1 forensic artefacts (274.2 MB)
<snip>

Signature: FILE
EntryId: 110351
Sequence: 1
BaseEntryId: 0
BaseEntrySequence: 0
HardLinkCount: 2
Flags: ALLOCATED
UsedEntrySize: 1000
TotalEntrySize: 1024
FileSize: 418
IsADirectory: false
IsDeleted: false
HasAlternateDataStreams: true
StandardInfoFlags: FILE_ATTRIBUTE_ARCHIVE
StandardInfoLastModified: 2022-03-10T19:49:34.904995Z
StandardInfoLastAccess: 2022-03-10T19:49:35.460937Z
StandardInfoCreated: 2018-04-09T17:46:18Z
FileNameFlags: FILE_ATTRIBUTE_ARCHIVE
FileNameLastModified: 2022-03-10T19:49:34.892970Z
FileNameLastAccess: 2022-03-10T19:49:34.892970Z
FileNameCreated: 2022-03-10T19:49:34.892970Z
FullPath: Users/TestUser/Downloads/APTSimulator_pw_apt/dist/APTSimulator/test-sets/persistence/web-shells.bat
DataStreams:
- stream_name: ''
  stream_number: 0
  stream_data: 404543484f204f46460a0a4543484f203d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d0a4543484f205745425348454c4c530a4543484f2044726f7070696e6720776562207368656c6c20696e206e657720575757206469726563746f72790a70696e67202d6e2035203132372e302e302e31203e204e554c0a0a4d4b444952202225575757524f4f5425220a22255a495025222065202d70255041535325202225544f4f4c415243482522202d616f61202d6f2225575757524f4f54252220746f6f6c7365745c622e6a7370203e204e554c0a22255a495025222065202d70255041535325202225544f4f4c415243482522202d616f61202d6f2225575757524f4f54252220746f6f6c7365745c74657374732e6a7370203e204e554c0a22255a495025222065202d70255041535325202225544f4f4c415243482522202d616f61202d6f2225575757524f4f54252220746f6f6c7365745c7368656c6c2e676966203e204e554c0a
- stream_name: Zone.Identifier
  stream_number: 1
  stream_data: 00005b5a6f6e655472616e736665725d0d0a5a6f6e6549643d330d0a526566657272657255726c3d433a5c55736572735c54657374557365725c446f776e6c6f6164735c41505453696d756c61746f725f70775f6170742e7a6970

Data Streams in YAML Output with Decoding

The chainsaw dump module now has the --decode-data-streams flag which will ask chainsaw to attempt to decode data streams before outputting them. This is super useful for quickly identifying Zone.Identifiers and the contents of files (if resident)

➜  release ./chainsaw dump ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT --decode-data-streams
 ██████╗██╗  ██╗ █████╗ ██╗███╗   ██╗███████╗ █████╗ ██╗    ██╗
██╔════╝██║  ██║██╔══██╗██║████╗  ██║██╔════╝██╔══██╗██║    ██║
██║     ███████║███████║██║██╔██╗ ██║███████╗███████║██║ █╗ ██║
██║     ██╔══██║██╔══██║██║██║╚██╗██║╚════██║██╔══██║██║███╗██║
╚██████╗██║  ██║██║  ██║██║██║ ╚████║███████║██║  ██║╚███╔███╔╝
 ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ ╚══╝╚══╝
    By WithSecure Countercept (@FranticTyping, @AlexKornitzer)

[+] Dumping the contents of forensic artefacts from: ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT (extensions: *)
[+] Loaded 1 forensic artefacts (274.2 MB)

<snip>
EntryId: 110351
Sequence: 1
BaseEntryId: 0
BaseEntrySequence: 0
HardLinkCount: 2
Flags: ALLOCATED
UsedEntrySize: 1000
TotalEntrySize: 1024
FileSize: 418
IsADirectory: false
IsDeleted: false
HasAlternateDataStreams: true
StandardInfoFlags: FILE_ATTRIBUTE_ARCHIVE
StandardInfoLastModified: 2022-03-10T19:49:34.904995Z
StandardInfoLastAccess: 2022-03-10T19:49:35.460937Z
StandardInfoCreated: 2018-04-09T17:46:18Z
FileNameFlags: FILE_ATTRIBUTE_ARCHIVE
FileNameLastModified: 2022-03-10T19:49:34.892970Z
FileNameLastAccess: 2022-03-10T19:49:34.892970Z
FileNameCreated: 2022-03-10T19:49:34.892970Z
FullPath: Users/TestUser/Downloads/APTSimulator_pw_apt/dist/APTSimulator/test-sets/persistence/web-shells.bat
DataStreams:
- stream_name: ''
  stream_number: 0
  stream_data: |
    @ECHO OFF

    ECHO ===========================================================================
    ECHO WEBSHELLS
    ECHO Dropping web shell in new WWW directory
    ping -n 5 127.0.0.1 > NUL

    MKDIR "%WWWROOT%"
    "%ZIP%" e -p%PASS% "%TOOLARCH%" -aoa -o"%WWWROOT%" toolset\b.jsp > NUL
    "%ZIP%" e -p%PASS% "%TOOLARCH%" -aoa -o"%WWWROOT%" toolset\tests.jsp > NUL
    "%ZIP%" e -p%PASS% "%TOOLARCH%" -aoa -o"%WWWROOT%" toolset\shell.gif > NUL
- stream_name: Zone.Identifier
  stream_number: 1
  stream_data: "\0\0[ZoneTransfer]\r\nZoneId=3\r\nReferrerUrl=C:\\Users\\TestUser\\Downloads\\APTSimulator_pw_apt.zip"

Writing Data Streams to Disk

Chainsaw now has the option to write extracted and decoded data streams to disk via the --data-streams-directory option

➜ ./chainsaw dump ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT --data-streams-directory ./datastreams

 ██████╗██╗  ██╗ █████╗ ██╗███╗   ██╗███████╗ █████╗ ██╗    ██╗
██╔════╝██║  ██║██╔══██╗██║████╗  ██║██╔════╝██╔══██╗██║    ██║
██║     ███████║███████║██║██╔██╗ ██║███████╗███████║██║ █╗ ██║
██║     ██╔══██║██╔══██║██║██║╚██╗██║╚════██║██╔══██║██║███╗██║
╚██████╗██║  ██║██║  ██║██║██║ ╚████║███████║██║  ██║╚███╔███╔╝
 ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ ╚══╝╚══╝
    By WithSecure Countercept (@FranticTyping, @AlexKornitzer)

[+] Dumping the contents of forensic artefacts from: ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT (extensions: *)
[+] Loaded 1 forensic artefacts (274.2 MB)
<snip>

➜  ls ./datastreams | tail -n 3
[Unknown]__edb7a20a758b_0_.disabled
[Unknown]__f7776040c557_1_.disabled
inetpub_wwwroot_tests.jsp__ed4158d930f8_0_.disabled

Hunting on the content of DataStreams

Now that chainsaw extracts MFT datastreams we can write powerful detection rules on them:

➜  cat /tmp/mft_rule/mimikatz_datastream_ref.yml
---
title: Mimikatz DataStream Reference
group: MFT
description: Look for mimikatz strings in data stream content
authors:
  - FranticTyping

kind: mft
level: critical
status: stable
timestamp: StandardInfoCreated


fields:
  - name: FileNamePath
    to: FullPath
  - name: FileSize
    to: FileSize
  - name: IsADirectory
    to: IsADirectory
  - name: IsDeleted
    to: IsDeleted
  - name: HasAlternateDataStreams
    to: HasAlternateDataStreams
  - name: DataStreams
    to: DataStreams

filter:
  condition: mimikatz

  mimikatz:
    DataStreams:
      - stream_data: i*mimi*

Leading to:

./chainsaw hunt /tmp/mft_rule ~/Artfacts/MFT/Win10/APTSimulatorVM/\$MFT

 ██████╗██╗  ██╗ █████╗ ██╗███╗   ██╗███████╗ █████╗ ██╗    ██╗
██╔════╝██║  ██║██╔══██╗██║████╗  ██║██╔════╝██╔══██╗██║    ██║
██║     ███████║███████║██║██╔██╗ ██║███████╗███████║██║ █╗ ██║
██║     ██╔══██║██╔══██║██║██║╚██╗██║╚════██║██╔══██║██║███╗██║
╚██████╗██║  ██║██║  ██║██║██║ ╚████║███████║██║  ██║╚███╔███╔╝
 ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ ╚══╝╚══╝
    By WithSecure Countercept (@FranticTyping, @AlexKornitzer)

[+] Loading detection rules from: /tmp/mft_rule
[+] Loading forensic artefacts from: ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT (extensions: .$MFT, .mft, .bin)
[+] Loaded 1 forensic artefacts (274.2 MB)
[+] Current Artefact: ~/Artefacts/MFT/Win10/APTSimulatorVM/\$MFT
[+] Hunting [========================================] 1/1   [00:00:02]
[+] Group: MFT
┌───────────────────────────┬─────────────────────────────────┬────────────────────────────────────────────────────┬──────────┬──────────────┬───────────┬─────────────────────────┬────────────────────────────────────────────────────┐
│         timestamp         │           detections            │                    FileNamePath                    │ FileSize │ IsADirectory │ IsDeleted │ HasAlternateDataStreams │                    DataStreams                     │
├───────────────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────┼──────────┼──────────────┼───────────┼─────────────────────────┼────────────────────────────────────────────────────┤
│ 2018-03-02 03:31:02+00:00 │ ‣ Mimikatz DataStream Reference │ Users/TestUser/Downloads/APTSimulator_pw_apt/APTSi │ 242      │ false        │ false     │ true                    │ - stream_name: ''                                  │
│                           │                                 │ mulator/test-sets/persistence/at-jobs.bat          │          │              │           │                         │  stream_data: |                                    │
│                           │                                 │                                                    │          │              │           │                         │   @ECHO OFF                                        │
│                           │                                 │                                                    │          │              │           │                         │                                                    │
│                           │                                 │                                                    │          │              │           │                         │   ECHO =========================================== │
│                           │                                 │                                                    │          │              │           │                         │ ================================                   │
│                           │                                 │                                                    │          │              │           │                         │   ECHO At Job Creation                             │
│                           │                                 │                                                    │          │              │           │                         │   ECHO Registering mimikatz in At job              │
│                           │                                 │                                                    │          │              │           │                         │   ping -n 5 127.0.0.1 > NUL                        │
│                           │                                 │                                                    │          │              │           │                         │                                                    │
│                           │                                 │                                                    │          │              │           │                         │   at 13:00 "C:\TMP\mim.exe sekurlsa::LogonPassword │
│                           │                                 │                                                    │          │              │           │                         │ s > C:\TMP\o.txt"                                  │
│                           │                                 │                                                    │          │              │           │                         │  stream_number: 0                                  │
│                           │                                 │                                                    │          │              │           │                         │ - stream_name: Zone.Identifier                     │
│                           │                                 │                                                    │          │              │           │                         │  stream_data: "\0\0[ZoneTransfer]\r\nZoneId=3\r\nR │
│                           │                                 │                                                    │          │              │           │                         │ eferrerUrl=C:\\Users\\TestUser\\Downloads\\APTSimu │
│                           │                                 │                                                    │          │              │           │                         │ lator_pw_apt.zip"                                  │
│                           │                                 │                                                    │          │              │           │                         │  stream_number: 1                                  │
├───────────────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────┼──────────┼──────────────┼───────────┼─────────────────────────┼────────────────────────────────────────────────────┤
│ 2018-03-02 03:31:14+00:00 │ ‣ Mimikatz DataStream Reference │ Users/TestUser/Downloads/APTSimulator_pw_apt/APTSi │ 297      │ false        │ false     │ true                    │ - stream_name: ''                                  │
│                           │                                 │ mulator/test-sets/persistence/schtasks.bat         │          │              │           │                         │  stream_data: |                                    │
│                           │                                 │                                                    │          │              │           │                         │   @ECHO OFF                                        │
│                           │                                 │                                                    │          │              │           │                         │                                                    │
│                           │                                 │                                                    │          │              │           │                         │   ECHO =========================================== │
│                           │                                 │                                                    │          │              │           │                         │ ================================                   │
│                           │                                 │                                                    │          │              │           │                         │   ECHO Schtasks Creation                           │
│                           │                                 │                                                    │          │              │           │                         │   ECHO Registering mimikatz in scheduled task      │
│                           │                                 │                                                    │          │              │           │                         │   ping -n 5 127.0.0.1 > NUL                        │
│                           │                                 │                                                    │          │              │           │                         │                                                    │
│                           │                                 │                                                    │          │              │           │                         │   schtasks /create /f /sc minute /mo 5 /tn GameOve │
│                           │                                 │                                                    │          │              │           │                         │ r /tr "C:\TMP\mim.exe sekurlsa::LogonPasswords > C │
│                           │                                 │                                                    │          │              │           │                         │ :\TMP\o.txt"                                       │
│                           │                                 │                                                    │          │              │           │                         │  stream_number: 0                                  │
│                           │                                 │                                                    │          │              │           │                         │ - stream_name: Zone.Identifier                     │
│                           │                                 │                                                    │          │              │           │                         │  stream_data: "\0\0[ZoneTransfer]\r\nZoneId=3\r\nR │
│                           │                                 │                                                    │          │              │           │                         │ eferrerUrl=C:\\Users\\TestUser\\Downloads\\APTSimu │
│                           │                                 │                                                    │          │              │           │                         │ lator_pw_apt.zip"                                  │
│                           │                                 │                                                    │          │              │           │                         │  stream_number: 1                                  │
├───────────────────────────┼─────────────────────────────────┼────────────────────────────────────────────────────┼──────────┼──────────────┼───────────┼─────────────────────────┼────────────────────────────────────────────────────┤

[alexkornitzer]

Added some feedback, which we can discuss however :)

[alexkornitzer]

Also don't forget to run cargo clippy.