v0.2 Release: FP16, Distributed, and Usability updates
Our main goal with this release is two-fold:
- address concerns around usability
- Update repo with new code for FP16, distributed training
Usability
- We've brought our training/generation code more in line with the pytorch word language model example
- Provide PyTorch classifier module/function for classifying sentiment from input text tensor
- Provide pretrained classifiers/language models for this module
- Provide simple standalone classifier script/example capable of classifying an input csv/json and writing results to other csv/jsons
- Flattening our directory structure to make code easier to find
- Putting reusable PyTorch functionality (new RNN api, weight norm functionality, eventually all fp16 functionality) in its own standalone python module to be published at a later date
FP16 + Distributed
- FP16 optimizer wrapper for optimizating FP16 models according to our [best practices] (https://github.com/NVIDIA/sentiment-discovery/blob/master/analysis/reproduction.md#fp16-training)
- available in
fp16/fp16.py
- available in
- Lightweight distributed wrapper for all reducing gradients across multiple gpus with either nccl or gloo backends
model/distributed.py
- distributed worker launch script
multiproc.py