Learning framework for program property prediction.
This is a backend for tools such as JSNice (http://jsnice.org/) for JavaScript that can predict program properties such as variable names and types. This backend is designed to extend the tool to multiple programming languages. For this reason, the machine learning machinery is extracted in this tool.
To get a complete tool, one must include a parses for each programming language of interest and train on a lot of code.
We have included an example frontend for JavaScript deminification at http://github.com/eth-srl/UnuglifyJS . This tool work together with the Nice2Server
To compile, first install dependencies
on Ubuntu:
sudo apt-get install libmicrohttpd-dev libcurl4-openssl-dev bazel libgoogle-glog-dev libgflags-dev
on Mac:
brew tap caskroom/versions brew cask install java8 brew install libmicrohttpd bazel glog gflags
on Windows follow any installation instructions and install libmicrohttpd, curl and bazel.
[Optional] Install Google Performance Tools:
-
libunwind: http://download.savannah.gnu.org/releases/libunwind/libunwind-1.1.tar.gz
-
gperftools: https://code.google.com/p/gperftools/
Finally, call
bazel build -c opt //...
if glog and gflags are not found, try running the command below instead
bazel build --cxxopt -I/usr/local/include --linkopt -L/usr/local/lib -c opt //...
To run tests, call
bazel test //...
Run:
bazel run //src/training/train
To get options for training, use:
bazel run //src/training/train --help
By default, train gets input programs (converted to JSON for example with UnuglifyJS) from the file testdata in the current directory. As a result, it creates files with the trained model.
If you wish to train the model using pseudolikelihood use the following parameters:
bazel run //src/training/train -- -training_method pl -input path/to/input/file --logtostderr
you can control the pseudolikelihood specific beam size with the -beam_size parameter which is different from the beam size used during MAP Inference.
//src/training/train expects data to be in protobuf recordIO format. If you want to use JSON input - use //src/training/train_json instead.
by default the usage of factor features in Nice2Predict is enabled, however if you wish to disable it you can launch the training with the following command:
bazel run //src/training/train -- -use_factors=false -input path/to/input/file --logtostderr
To predict properties for new programs, start a server after a model was trained:
bazel run //src/server/nice2serverproto -- --logtostderr
To run old JsonRPC API:
bazel run //src/server/nice2server -- --logtostderr
One can debug and observe deobfuscation from the viewer available in the viewer/viewer.html .