Microsoft Malware Classification Challenge (BIG 2015) - Working Notes

The code was run by python3 on Ubuntu 16.04 LTS - 8 CPUs, 30GB RAM, NVIDIA Tesla K80 (CUDA 10.0, cudnn 7.1.4)

Install necessary libraries

$ pip3 install -r requirements.txt

Directory Structure

Download malware data from https://drive.google.com/file/d/18SerrpduXx0k8cunBsspa0328Yoe3hyc/view?usp=sharing and unzip data to the same level of the code directory as describled below:

• code
• code_test
• test
• train

Code Structure

• data_loader.py: contains DataLoader model to load train/val/test data
• eval_test.py: contains evaluation code for some last fully connected layers and generate csv results
• extract_features.py: extracts feature vector and generate malware image based on malware binary file
• fcnet.py: contains Fully Block model and save/load checkpoint weight
• generate_submission.py: generate submission based on .npy file or top 1 prediction csv file
• resnet.py: contains many Resnet model
• train_fcnet.py: Fully Block training code
• trans_learn.py: forward image to Resnet network to get deep features
• utils.py: contains some useful functions to support other codes

Usage

$ python3 extract_features.py
$ python3 trans_learn.py
$ python3 train_fcnet.py
$ python3 eval_test.py best.weight
$ python3 generate_submission.py --predict_filepath best_predict.csv