feat: stack-users/stack-ip-addresses #131

fukusuket · 2024-03-03T00:38:49Z

What Changed

Closed: stack-users command #130
Closed: stack-ip-addresses command #129
Related: Refactoring duplicate code #99
- The JSONL For loop in the stack command is almost the same, so move the duplicate code to stackUtil processJSONL
Added integration-test case

Test

Environment

OS: macOS Sonoma version 14.2.1
Hayabusa v2.13.0
Nim: 2.0.2
nimble: 0.14.2

In the integration test below, I confirmed that the all commands succeeded.

https://github.com/Yamato-Security/takajo/actions/runs/8126728721

I would appreciate it if you could check it out when you have time🙏

refactor: remove duplicated jsonl process code

fukusuket · 2024-03-03T00:50:21Z

stack-users

% ./takajo stack-users -t timeline.jsonl -s -l critical -q
Started the Stack Users command

This command will stack the User field as well as show alert information.

Counting total lines. Please wait.
Total lines: 32,365

Scanning the Hayabusa timeline. Please wait.

100%|█████████████████████████| 32365/32365 [ 0.4s< 0.0s, 117.62k/sec]

┌───────┬────────────────────────────┬───────────┬────────────────────────────────────────────────────────────────────────────────────────────┐
│ Count │ User                       │ Levels    │ Alerts                                                                                     │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 13    │ MSEDGEWIN10\IEUser         │ crit (13) │ Antivirus Exploitation Framework Detection (1) | Defender Alert (Severe) (4) | Hacktool    │
│       │                            │           │ Execution - Imphash (1) | Qakbot Rundll32 Fake DLL Extension Execution (1) | TrustedPath   │
│       │                            │           │ UAC Bypass Pattern (2) | WannaCry Ransomware Activity (4)                                  │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 5     │ admmig                     │ crit (5)  │ Active Directory Replication from Non Machine Account (3) | Potential SystemNightmare      │
│       │                            │           │ Exploitation Attempt (2)                                                                   │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 4     │ OFFSEC\admmig              │ crit (4)  │ Antivirus Password Dumper Detection (3) | Sticky Key Like Backdoor Execution (1)           │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 3     │ Administrator              │ crit (3)  │ Active Directory Replication from Non Machine Account (3)                                  │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1     │ IEUser                     │ crit (1)  │ Potential Credential Dumping Via LSASS Process Clone (1)                                   │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1     │ SRVDEFENDER01$             │ crit (1)  │ Sticky Key Like Backdoor Execution (1)                                                     │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1     │ hack1                      │ crit (1)  │ Potential SMB Relay Attack Tool Execution (1)                                              │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1     │ NT AUTHORITY\LOCAL SERVICE │ crit (1)  │ Hacktool Execution - Imphash (1)                                                           │
├───────┼────────────────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1     │ NT AUTHORITY\SYSTEM        │ crit (1)  │ Potential Credential Dumping Via LSASS Process Clone (1)                                   │
└───────┴────────────────────────────┴───────────┴────────────────────────────────────────────────────────────────────────────────────────────┘


Elapsed time: 0 hours, 0 minutes, 0 seconds

fukusuket · 2024-03-03T00:51:09Z

stack-ip-addresses

% ./takajo stack-ip-addresses -t timeline.jsonl -a -l high -q
Started the Stack IpAddresses command

This command will stack the IpAddress field as well as show alert information.

Counting total lines. Please wait.
Total lines: 32,365

Scanning the Hayabusa timeline. Please wait.

100%|█████████████████████████| 32365/32365 [ 0.4s< 0.0s, 121.06k/sec]

┌───────┬───────────────┬───────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Count │ IpAddress     │ Levels    │ Alerts                                                                                                  │
├───────┼───────────────┼───────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 11    │ 10.0.2.15     │ high (11) │ Suspicious Outbound Kerberos Connection - Security (11)                                                 │
├───────┼───────────────┼───────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 10    │ 127.0.0.1     │ high (10) │ Network Connection Initiated Via Notepad.EXE (6) | Outbound RDP Connections Over Non-Standard Tools (2) │
│       │               │           │ | RDP over Reverse SSH Tunnel WFP (2)                                                                   │
├───────┼───────────────┼───────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 4     │ 127.0.0.2     │ high (4)  │ Outbound RDP Connections Over Non-Standard Tools (2) | RDP over Reverse SSH Tunnel WFP (2)              │
├───────┼───────────────┼───────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 2     │ 151.101.0.133 │ high (2)  │ Connection Initiated Via Certutil.EXE (2)                                                               │
├───────┼───────────────┼───────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1     │ 10.0.2.18     │ high (1)  │ Network Connection Initiated Via Notepad.EXE (1)                                                        │
└───────┴───────────────┴───────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────┘


Elapsed time: 0 hours, 0 minutes, 0 seconds

fukusuket · 2024-03-03T00:51:38Z

help

% ./takajo
╔════╦═══╦╗╔═╦═══╗ ╔╦═══╗
║╔╗╔╗║╔═╗║║║╔╣╔═╗║ ║║╔═╗║
╚╝║║╚╣║ ║║╚╝╝║║ ║║ ║║║ ║║
  ║║ ║╚═╝║╔╗╖║╚═╝╠╗║║║ ║║
 ╔╝╚╗║╔═╗║║║╚╣╔═╗║╚╝║╚═╝║
 ╚══╝╚╝ ╚╩╝╚═╩╝ ╚╩══╩═══╝
  by Yamato Security

Version: 2.5.0 Dev Build
Usage: takajo.exe <COMMAND>

Commands:
  help                           print comprehensive or per-cmd help
  extract-scriptblocks           extract and reassemble PowerShell EID 4104 script block logs
  list-domains                   create a list of unique domains to be used with vt-domain-lookup
  list-hashes                    create a list of process hashes to be used with vt-hash-lookup
  list-ip-addresses              create a list of unique target and/or source IP addresses to be used with vt-ip-lookup
  list-undetected-evtx           create a list of undetected evtx files
  list-unused-rules              create a list of unused sigma rules
  split-csv-timeline             split up a large CSV file into smaller ones based on the computer name
  split-json-timeline            split up a large JSONL timeline into smaller ones based on the computer name
  stack-computers                stack computers
  stack-cmdlines                 stack executed command lines
  stack-dns                      stack DNS queries and responses
  stack-ip-addresses             stack ipaddresses
  stack-logons                   stack logons by target user, target computer, source IP address and source computer
  stack-processes                stack executed processes
  stack-services                 stack service names and paths
  stack-tasks                    stack new scheduled tasks
  stack-users                    stack users
  sysmon-process-tree            output the process tree of a certain process
  timeline-logon                 create a CSV timeline of logon events
  timeline-partition-diagnostic  create a CSV timeline of partition diagnostic events
  timeline-suspicious-processes  create a CSV timeline of suspicious processes
  timeline-tasks                 create a CSV timeline of scheduled tasks
  ttp-summary                    summarize tactics and techniques found in each computer
  ttp-visualize                  extract TTPs and create a JSON file to visualize in MITRE ATT&CK Navigator
  ttp-visualize-sigma            extract TTPs from Sigma and create a JSON file to visualize in MITRE ATT&CK Navigator
  vt-domain-lookup               look up a list of domains on VirusTotal
  vt-hash-lookup                 look up a list of hashes on VirusTotal
  vt-ip-lookup                   look up a list of IP addresses on VirusTotal

Command help: takajo help <COMMAND>

Examples:
  extract-scriptblocks -t ../hayabusa/timeline.jsonl [--level low] -o scriptblock-logs
  list-domains -t ../hayabusa/timeline.jsonl -o domains.txt
  list-hashes -t ../hayabusa/case-1.jsonl -o case-1
  list-ip-addresses -t ../hayabusa/timeline.jsonl -o ipAddresses.txt
  list-undetected-evtx -t ../hayabusa/timeline.csv -e ../hayabusa-sample-evtx
  list-unused-rules -t ../hayabusa/timeline.csv -r ../hayabusa/rules
  split-csv-timeline -t ../hayabusa/timeline.csv [--makeMultiline] -o case-1-csv
  split-json-timeline -t ../hayabusa/timeline.jsonl -o case-1-json
  stack-cmdlines -t ../hayabusa/timeline.jsonl [--level low] -o cmdlines.csv
  stack-computers -t ../hayabusa/timeline.jsonl [--level informational] [--sourceComputers] -o computers.csv
  stack-dns -t ../hayabusa/timeline.jsonl [--level infomational]  -o dns.csv
  stack-ip-addresses -t ../hayabusa/timeline.jsonl [--level infomational] [--targetIpAddresses] -o ipAddresses.csv
  stack-logons -t ../hayabusa/timeline.jsonl -o logons.csv
  stack-processes -t ../hayabusa/timeline.jsonl [--level low] -o processes.csv
  stack-services -t ../hayabusa/timeline.jsonl [--level infomational] -o services.csv
  stack-tasks -t ../hayabusa/timeline.jsonl [--level infomational] -o tasks.csv
  stack-users -t ../hayabusa/timeline.jsonl [--level infomational] [--sourceUsers] -o users.csv
  sysmon-process-tree -t ../hayabusa/timeline.jsonl -p <Process GUID> [-o process-tree.txt]
  timeline-logon -t ../hayabusa/timeline.jsonl -o logon-timeline.csv
  timeline-partition-diagnostic -t ../hayabusa/timeline.jsonl -o partition-diagnostic-timeline.csv
  timeline-suspicious-processes -t ../hayabusa/timeline.jsonl [--level medium] [-o suspicious-processes.csv]
  timeline-tasks -t ../hayabusa/timeline.jsonl -o task-timeline.csv
  ttp-summary -t ../hayabusa/timeline.jsonl -o ttp-summary.csv
  ttp-visualize -t ../hayabusa/timeline.jsonl -o mitre-ttp-heatmap.json
  ttp-visualize-sigma -r ../hayabusa/rules -o sigma-rules-heatmap.json
  vt-domain-lookup  -a <API-KEY> --domainList domains.txt -r 1000 -o results.csv --jsonOutput responses.json
  vt-hash-lookup -a <API-KEY> --hashList case-1-MD5-hashes.txt -r 1000 -o results.csv --jsonOutput responses.json
  vt-ip-lookup -a <API-KEY> --ipList ipAddresses.txt -r 1000 -o results.csv --jsonOutput responses.json

refactor: remove duplicated jsonl process code

YamatoSecurity · 2024-03-03T08:10:47Z

@fukusuket Sorry, I didn't realize you updated stackComputers.nim and there was a conflict so I replaced it with the main branch version. Please check if you need to update it.

YamatoSecurity · 2024-03-03T08:17:26Z

@fukusuket Maybe rename proc stackIpaddresses -> proc stackIpAddresses and stackIpaddresses.nim -> stackIpAddresses.nim to keep the capitalization convention?

fukusuket · 2024-03-03T08:39:47Z

@YamatoSecurity
Thank you so much for checking :) Sorry for conflict🙇 I fixed it!

YamatoSecurity · 2024-03-03T21:24:44Z

@fukusuket Thanks so much! Sorry about that.
For stack-ip-addresses we should probably filter out 127.0.0.1 and ::1 as we don't care about localhost.
For stack-users it is more tricky as there are many system user accounts and if we filter too much then it gives a possibility for evasion.

Window Manager\DWM-1
NT AUTHORITY\NETWORK SERVICE
NT AUTHORITY\LOCAL SERVICE
NT AUTHORITY\SYSTEM
LOCAL SERVICE
IIS APPPOOL\DefaultAppPool

(there are probably more)

What about an option --filterSystemAccounts (default: true) that will filter out the above accounts?

Also, I noticed that computer accounts (the ones that end with $) are also included so we should probably filter these by default as well. So we can add another option --filterComputerAccounts (default: true) that will ignore usernames that end with $.

fukusuket · 2024-03-04T04:00:37Z

@YamatoSecurity Thank you so much for checking :) I implemented filter out 127.0.0.1 and ::1.
It seems that the service account will be localized😵‍💫, so I would like to include the one below in the exclusion list.What do you think?
https://learn.microsoft.com/ja-jp/sql/database-engine/configure-windows/configure-windows-service-accounts-and-permissions?view=sql-server-ver16#Localized_service_names

YamatoSecurity · 2024-03-04T05:19:51Z

@fukusuket I think that's a good idea!

fukusuket · 2024-03-04T13:13:35Z

@YamatoSecurity
I added the filter option for stack-users commnad! Could you please confirm?

YamatoSecurity · 2024-03-05T08:59:24Z

Thanks so much! I just noticed that when analyzing source users, n/a is sometimes outputted so we should probably ignore that.

fukusuket · 2024-03-05T09:41:36Z

@YamatoSecurity Thank you so much for checking! I added n/a exclusion process!

YamatoSecurity

@fukusuket Thanks so much. LGTM! I just updated the messages to explain what fields we are analyzing.

feat: stack-users/stack-ip-addresses

b407189

refactor: remove duplicated jsonl process code

fukusuket self-assigned this Mar 3, 2024

fukusuket added the enhancement New feature or request label Mar 3, 2024

fukusuket added this to the v2.5.0 milestone Mar 3, 2024

fukusuket marked this pull request as ready for review March 3, 2024 00:49

feat: stack-users/stack-ip-addresses

110bdc0

refactor: remove duplicated jsonl process code

fukusuket marked this pull request as draft March 3, 2024 00:53

ui: add help message

dc974b6

fukusuket marked this pull request as ready for review March 3, 2024 01:12

fukusuket marked this pull request as draft March 3, 2024 01:15

fukusuket marked this pull request as ready for review March 3, 2024 01:22

fukusuket requested review from hitenkoku and YamatoSecurity March 3, 2024 01:23

Merge branch 'main' into 130-stack-user-and-ip

78ab636

fukusuket and others added 2 commits March 3, 2024 17:31

refactor: to keep the capitalization convention

fe7efcd

update changelog

3189da9

chg: exclude 127.0.0.1 and ::1

69dd9cc

fukusuket added 2 commits March 4, 2024 21:49

chg: exclude SystemAccounts

a0b8d9e

chg: add help message

083cbc9

chg: exclude "n/a"

e662d12

update msgs

fdd308d

YamatoSecurity approved these changes Mar 5, 2024

View reviewed changes

YamatoSecurity merged commit fa256e5 into main Mar 7, 2024
3 checks passed

YamatoSecurity deleted the 130-stack-user-and-ip branch March 7, 2024 02:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: stack-users/stack-ip-addresses #131

feat: stack-users/stack-ip-addresses #131

fukusuket commented Mar 3, 2024 •

edited

Loading

fukusuket commented Mar 3, 2024

fukusuket commented Mar 3, 2024

fukusuket commented Mar 3, 2024 •

edited

Loading

YamatoSecurity commented Mar 3, 2024

YamatoSecurity commented Mar 3, 2024

fukusuket commented Mar 3, 2024

YamatoSecurity commented Mar 3, 2024

fukusuket commented Mar 4, 2024

YamatoSecurity commented Mar 4, 2024

fukusuket commented Mar 4, 2024

YamatoSecurity commented Mar 5, 2024

fukusuket commented Mar 5, 2024 •

edited

Loading

YamatoSecurity left a comment

feat: stack-users/stack-ip-addresses #131

feat: stack-users/stack-ip-addresses #131

Conversation

fukusuket commented Mar 3, 2024 • edited Loading

What Changed

Test

Environment

fukusuket commented Mar 3, 2024

stack-users

fukusuket commented Mar 3, 2024

stack-ip-addresses

fukusuket commented Mar 3, 2024 • edited Loading

help

YamatoSecurity commented Mar 3, 2024

YamatoSecurity commented Mar 3, 2024

fukusuket commented Mar 3, 2024

YamatoSecurity commented Mar 3, 2024

fukusuket commented Mar 4, 2024

YamatoSecurity commented Mar 4, 2024

fukusuket commented Mar 4, 2024

YamatoSecurity commented Mar 5, 2024

fukusuket commented Mar 5, 2024 • edited Loading

YamatoSecurity left a comment

Choose a reason for hiding this comment

fukusuket commented Mar 3, 2024 •

edited

Loading

fukusuket commented Mar 3, 2024 •

edited

Loading

fukusuket commented Mar 5, 2024 •

edited

Loading