Python 3 library useful for getting structured IOC data from mwdb configs.
Warning: this project is only relevant to mwdb and malduck users.
Malduck configs (like the ones in mwdb) are usually unstructured (they're just JSONs with some additional metadata). On the other hand, automated processing often requires structured data.
For example, URL processing and extraction is a common use case for analysts. Sadly, every module stores them a bit differently (and due to backward compatibility we're not ready to change that).
For example, compare how we store URLs from ISFB:
"domains": [
{
"cnc": "http://fantaniz.ru"
},
{
"cnc": "http://snezhkaie.ru"
}
]Lokibot:
"urls": [
{
"url": "http://hockvvee.com/chief4/five/fre.php"
},
{
"url": "kbfvzoboss.bid/alien/fre.php"
},
{
"url": "alphastand.top/alien/fre.php"
}
]And mirai:
"cncs": [
{
"host": "107.160.244.5",
"port": 1024
}
]With mwdb-iocextract you can handle all these (and dozens more) config types in the same way:
from mwdb_iocextract import parse
config_family = "mirai"
config_data = {
"cncs": [
{
"host": "107.160.244.5",
"port": 1024
}
],
"table_key": "0xdedefbaf",
"variant": "OWARI",
"type": "mirai"
}
iocs = parse(config_family, config_data)
print(iocs.prettyprint())> python test.py
NetLoc 107.160.244.5:1024
See below for more usage examples.
$ pip install mwdb-iocextractYou can find the newest version here:
https://pypi.org/project/mwdb-iocextract/
Most configs can be parsed without any change to this library. Standard keys, like "urls", are automatically recognised and parsed correctly.
Plurality of the key name does not matter. When value is a list, all elements are added separately. For example both of these are equivalent:
{
"urls": "127.0.0.1"
}{
"url": ["127.0.0.1"]
}Right now this library supports:
"Network Locations" can be IP, domain, URL, etc. There are many available formats:
"hosts": [
"127.0.0.1", # format 1 - IP
"http://malware.com", # format 2 - URL
{ "cnc": "http://malware.com:1337" }, # format 3 - URL in a dict.
# Allowed keys: "cnc", "url", "ip", "domain", "host" (all handled in the same way)
{ "cnc": "malware.com", "port": 1337 }, # format 4 - domain/port pair
# NOT allowed: url + port
# { "cnc": "http://malware.com:1337", "port": 1337 },Config keys: urls, c2, ips, domains, url, cnc, cncs, hosts, host, cncurl.
Passwords hardcoded in malware. Plain text.
Config keys: password.
Config keys: mtx, mutex. Plain text.
Emails used by malware and hardcoded in the source. Plain text.
Config keys: email, emails.
HTML or txt ransom messages hardcoded in the source. Plain text.
Config keys: ransommessage.
RSA public or private keys hardcoded in the binary.
There are two supported formats:
- plaintext (
"-----BEGIN PUBLIC KEY-----...") - parsed - a dict with "n", "e", and (optionall) "d" keys.
Parsed format is more deterministic and recommended for your modules, but you can use both.
Config keys: publickey, rsapub, rsakey, pubkey, privkey.
AES keys hardcoded in the binary. Plaintext.
Config keys: aeskey.
XOR keys hardcoded in the binary. Plaintext.
Config keys: xorkey.
XOR keys hardcoded in the binary. Plaintext.
Config keys: serpentkey.
Other key types found in the malware. Extracted as a key of type "unknown". Plaintext.
Config keys: encryptionkey, key, keys.
Paths or filenames of files dropped by the malware. Plaintext.
Config keys: drop_name.
If you want to extend this library or add support for more modules, feel free to contribute to this repository. We're only interested in modules at least partially publicly accessible. So mwcfg.info module support is OK to merge, but your in-house internal TLP:RED modules - probably no.
How to download config from mwdb and parse it:
from mwdb_iocextract import parse
from mwdblib import Malwarecage
def main():
# See also https://mwdblib.readthedocs.io/en/latest/index.html
mwdb = Malwarecage()
mwdb.login("msm", "my_secret_password")
for cfg in mwdb.recent_configs():
if cfg.type != "static":
# This library only works with configs of type "static"
# (default mwdb config type).
continue
iocs = parse(cfg.family, cfg.cfg)
print(iocs.prettyprint()) # convert all IoCs to string
if __name__ == "__main__":
main()The result of a parse() is a IocCollection object.
You can use it like a normal Python object, for example:
iocs = parse(cfg.family, cfg.cfg)
for rsa_key in iocs.rsa_keys:
e = rsa_key.e # get e
n = rsa_key.n # get n
for netloc in iocs.network_locations:
ip = netloc.ip # get IP if known
domain = netloc.domain # get domain if known
host = netloc.host # get domain if known, otherwise IP
port = netloc.port # get port if known
loctype = netloc.location_type # get type (usually CNC)
url = netloc.url # get host + port + path
for key_type, key_data in iocs.keys:
# key_type - for example "rc4"
# key_data - hexencoded bytes, for example "6123541243"
pass
iocs.passwords # passwords or similar data found in the config
iocs.mutexes # mutex names used by malware
iocs.dropped_filenames # filenames dropped on the disk
iocs.emails # emails used by malwareYou can convert IocCollection to a MISP object:
def upload_to_misp(family, config):
iocs = parse(family, config)
if not iocs:
# Nothing actionable found - skip the config
return
# Be careful not to upload duplicated events.
# We use uuid5s generated from mwdb dhash as unique deterministic UUIDs.
event = MISPEvent()
event.add_tag(f"mwdb:family:{family}")
event.info = f"Malware configuration ({family})"
for o in iocs.to_misp():
event.add_object(o)
misp = ExpandedPyMISP(MISP_URL, MISP_KEY, MISP_VERIFYCERT)
misp.add_event(event)