NNI 2021 August~September Iteration Planning

[QuanluZhang]

This is the plan for the iteration in August~September Iteration, it's a 6 weeks iteration.

Release Plan for stableness release and use case support

• Release manager: @acured
• Feature freeze date: Sept 9 Sep 16
• Code freeze and demo date: Sept 16 Sep 23 Sep 28 Oct 9
• Branch cut and next release planning date: Sept 23 Sep 30 Oct 13
• Bugbash - Oct 22
• Release date: Sept 27 Oct 8 Oct 9 Oct 18 Oct 25

NAS

• (need review) fix benchmark @ultmaster
  • unify architecture format. Aug 20 NAS Benchmark integration (stage 1) #4090
  • provide simple API for query. Aug 20 NAS Benchmark integration (stage 1) #4090
  • support benchmark end2end (test strategies' performance) NAS benchmark integration (stage 2) - download #4205
• support serialization @ultmaster General serialization infrastructure #4172
• review and improve nas tests stage 1 @ultmaster. Aug 20 Aug 31 [Retiarii] Remove unused code and enrich integration tests #4097
• improve quick start (notebook) @QuanluZhang. Aug 12 Aug 17 Aug 20 PR improve NAS quickstart #4094
  • replace example, update mnist example. Aug 27 Sep 1
• examples for image classification, object detection, audio (show reasonably good performance)
  • transformer @skeletondyh Aug 31
    • investigate search space and common models and chose one to start.
    • review meeting for investigate. Sep 1
    • implement NAS. Sep 8
    • review meeting for result(demo). Sep 10 Sep 15 Sep 17 Sep18
    • improvement and PR(code and doc) Sep 27 [Retiarii] Transformer example #4222
• deal with pytorch lightning onnx error for NAS visualization @ultmaster

Model Compression

• support compression v2 @J-shang Aug 20 Compression v2 Stage 1 #3917 [Model Compression] update config list key #4074 [Model Compression] Pruning Scheduler #4089 [Model Compression] admm pruner #4116 [Model Compression] Add more Task Generator #4178
  • tests [Model Compression] Add Unit Test #4125
  • (bug bash)doc tutorial
• support per-channel quantization support dtype&scheme customization for QAT quantizer #4137
• end2end examples for image classification, object detection, audio, transformer (show reasonably good performance)
  • image classification (MobileNet) @xiaowu0162 Aug 18 Aug 30 [End-to-end demo] MobileNetV2 Compression #4102
    • investigate model compression paper and try to compress model by NNI.
    • review meeting for investigate and NNI result(demo).
    • compare with other opensource compression tools.
    • review meeting for comparing results.
    • improvement and PR(code and doc) [End-to-end demo] MobileNetV2 Compression #4102
  • object detection (coco, yolo3) @linbinskn Sep 1 Sep 22
    • investigate model compression paper and try to compress model by NNI.
    • review meeting for investigate and NNI result(demo). Sep 2 Sep 8
    • support multi-gpu training @chenbohua3 support dp multi-gpu training for QAT quantizer #4127
    • support bn-folding for lsq quantizer Sep 8 [Quantization] support bn-folding for lsq #4148
    • review meeting for comparing results. Sep 26
    • improvement and PR(code and doc)

HPO

• refactor Hyper-opt based algorithm - random tuner. @liuzhe-lz Refactor Hyperopt Tuners (Stage 1) - random tuner #4118

nnictl & nnimanager & training service @SparkSnail @liuzhe-lz @acured

• P0 - fix macos ut @liuzhe-lz Aug 20 Aug 24 Refactor NNI Manager UT (Stage 1) - setup toolchain #4109
  • (bug bash)ut stage 2, modify dependency injection
• P0 - add trial gpu test in UT, verify whether gpu is used in IT @acured add gpu test in it #4162
• P0 - hybrid local & remote & aml integration test @SparkSnail Aug 23 Support aml in hybrid pipeline #4124
• P0 - clean deprecated code (nasui) @ultmaster (nasui)Remove NAS UI #4088
• P0 - demo and improve distributed trials on different training services (e.g., kubeflow) @SparkSnail Sep 8
  • Is support? How to support? investigate
• fix pipeline manual az login issue @acured Aug 20 add client az login in pipeline #4082
• fix model speedup out of memory? issue @J-shang
• experiment config doc update @J-shang Update Experiment Config doc #4131
• improve doc: clarify the difference between training service storage and shared storage @J-shang Update shared storage doc #4217
• double check the cancel logic of aml (&pai?) trial runner @SparkSnail

WebUI

• bug fix - Improve trial intermediate result graph. @Lijiaoa Improve trial intermediate result graph: trials have not intermediate default key #4171
• bug fix - webui prompts error message. @Lijiaoa [fix issue#4015] webui prompts error message if user set maxExperimentDuration < execDuration #4164

regression test needed, or bugs reported during the iteration by external users (need test and close the issues)

• ERROR: Restful server start failed! Failed to create log dir: RangeError: Invalid time value #4077, Restful server started failed #4083 Restful server started failed @acured
• nni==2.3 tensorboard uninstall then install get version erro,how solve it? nni==2.3 tensorboard uninstall then install get __version__ erro,how solve it? #4087 @SparkSnail
• hyper-parameter graph are not showing properly #4079 @Lijiaoa
• Exponential Notation is not allowed anymore Exponential Notation is not allowed anymore #4080 @QuanluZhang
• SyntaxError on WebUI with old version of Firefox SyntaxError on WebUI with old version of Firefox #4081 @Lijiaoa
• Small spelling error when executing NNI via Python Script #4158 @acured
• assert len(set(num_channels_list)) == 1 assert len(set(num_channels_list)) == 1 #4160 @J-shang
• ERROR: Config V2 validation failed: ValueError('_AlgorithmConfig: Unrecognized fields builtinassessorname') #4066 users found an walkaround, we need a fix for it @liuzhe-lz
• Task is always in a waiting state in the remote machine Task is always in a waiting state in the remote machine #3905 (not an 2.4 regression but a 2.1 regression) @acured
• NNI installation issue with ConfigSpace-0.4.7 NNI installation issue with ConfigSpace-0.4.7 #3909 (not sure whether this is a regression or not) @liuzhe-lz
• Runtime Error during L1FilterPruner (and other one-shot pruners) Runtime Error during L1FilterPruner (and other one-shot pruners)  #3944 @zheng-ningxin
• submit task to OpenPAI cluster failed submit task to OpenPAI cluster failed #4022 @SparkSnail
• Kubeflow setup #3978 @SparkSnail
• Got Segmentation fault when running amc_search.py Got Segmentation fault when running amc_search.py #4000 (to check whether there are any version limits) @linbinskn
• View an experiment with experiment path. View an experiment with experiment path. #3953 @SparkSnail to verify and close the issue
• AutoCompress with level pruner leads to runtime error AutoCompress with level pruner leads to runtime error #3948 @J-shang
• SPOS hardware-aware nas multi_trial.py don't work SPOS hardware-aware nas multi_trial.py don't work #4198 @JiahangXu

delayed

• classification @jonsnows Aug 26
  • investigate search space and common models and chose one to start.
  • review meeting for investigate. Aug 30
  • implement NAS (run on openpai ne cluster). Sep 8 Sep 10
  • review meeting for result(demo). Sep 15 Sep 22 Sep 29
  • improvement and PR(code and doc)
• object detection @QuanluZhang Aug 30 Sep 6 Sep 8 Sep 9 Sep 15
  • investigate search space and common models and chose one to start.
  • review meeting for investigate. Sep 16
  • implement NAS, baseline model result
  • review meeting for result(demo).
  • improvement and PR(code and doc)

[acured]

Stretch goals

NAS

• P2 - refactor nas examples @ultmaster
• P2 - support debug mode for NAS experiment @ultmaster
• P0 - Add examples for NAS-Bench-series @ultmaster
  • P2 - 301
• P2 - Evaluate existing strategies (multi-trial) on NAS benchmarks @ultmaster
• P0 - Support one system-level optimization: CSE optimization to deduplicate computation @hzhua pr [Retiarii] cross-graph optimization: device placement and input deduplication #3202
• P0 - Improve training service to support CSE optimization @hzhua extend reusable training service to support placement constraint #3897
• P2 - TensorFlow support: graph converter and support inline mutation API @liuzhe-lz
  • add raw framework
  • add example 7.12
• P0 - Migrate FBNet from NAS v1.0 to Retiarii framework @ali
• P2 - AutoDropout @skeletondyh 7.16
• P1 - Evaluate one-shot strategies on NAS benchmarks @ultmaster
• P1 - Further improve the logic of graph generation and code generation (Further integrated with TorchScript) @QuanluZhang
• P1 - Test classic nas tuners on NNI NAS benchmark and report evaluation results @ultmaster
• P2 - Migrate Cream from NAS v1.0 to Retiarii framework @ yuhangchen
• P2 - refactor of Retiarii execution engine @ultmaster
  • support local debug mode for pure-python execution engine
  • support weight transfer from supernet to submodel
  • support export of top model for pure-python execution engine
  • P2 - refactor of "self._cur_samples" in mutators
  • P2 - support validation for other strategies
• P1 - Review strategy/experiment/rest stop condition @QuanluZhang
• P1 - support graph-based NAS algorithms @ultmaster @skeletondyh
• P1 - Deal with strategy failure
• NAS runtime improvement Sep 28
  • provide option to keep webui when experiment is done (add arg in run()) @QuanluZhang Sep 15 Sep 24
  • support hyper-parameter figure on webui for retiarii experiment @QuanluZhang Sep 15 Sep 24
  • support tuner_no_more_trial for retiarii experiment @QuanluZhang Sep 15 Sep 24

Model Compression

• P0 - support iterative pruning @J-shang 6.18 review meeting (done) / 6.23 sample / 6.25 7.14
  • P0 - implement model compression 2.0, include 8 maskers (basic pruners) 7.9 Compression v2 Stage 1 #3917
  • P0 - improve pruners to more faithfully reproduce the papers @J-shang
  • P2 - Refactor compress() in IterativePruner, unify support AGP, ADMM. (iterative pruner follow-up  #3669 )
  • P2 - Add more criteria(strategy, i.e., acc drop toleronce) (discussion needed)
• P1 - MixedMaskerPruner @J-shang
• P2 - Refactor of model graph generation (as an independent component) @zheng-ningxin

Training service & NNI manager && nnictl

• P2 - hybrid integration test on windows @SparkSnail
• P2 - hybrid integration test with azure storage @SparkSnail
• P2 - support max_trial_duration for NNI NAS
• P2 - fix security issues @liuzhe-lz
• P2 - improve the usage of shared storage for remote training service, e.g., leave storage mount to user (discussion) @acured
• P2 - improve pipeline integration test yaml (align script and dependences) (investigate) @liuzhe-lz (5 days)
• P2 - fix ut macos timeout, maybe nnimanager tests, add back some removed tests @liuzhe-lz (5 days)
• P2 - reduce wheel package size (check) @liuzhe-lz (3 days)
  • refactor some modules as plugin module (install only when used)
• P2 - review and improve tests & doc for shared storage, experiment management, jupyterlab, training service
  • experiment management @liuzhe-lz (3 days)
  • jupyterlab @liuzhe-lz
  • training service Doc
  • training service UT @acured Aug 13
    • common
    • kubernetes
    • local
    • remote
    • reusable
      • shared storage @acured Aug 13 Add shared storage ut #4068
      • enviroments
    • remove HDFS doc from pai doc
• P0 - (discussion needed) design an API to share data loading/preprocessing among trials (Did every trail need to load data? #3294 )
  • P2 - implementation
• P0 - Make reuse default @SparkSnail
• P1 - support storage service for different training services, review and implementation
• P1 - add more test for remote mode training service
• P2 - Frameworkcontroller support @SparkSnail
• P0 - fix path escape error in windows (issue nni's mnist-pytorch example runs failed #3869) @liuzhe-lz @acured @J-shang Escape environment variable PATH for Windows scripts #3898
  • add back removed ut
• P2 - Mask confidential config fields (depends on prev items) @liuzhe-lz
• P2 - nnictl experiment export improvements @J-shang
• P2 - internal tuner APIs for improving tuner debuggability, allow debugger tool @liuzhe-lz
• P2 - nnictl auto-completion @liuzhe-lz
• P2 - Dependency-aware supported in iterative pruner (discussion needed)
• P2 - Add full test (l2filter, Taylor, NetAdapt, SimulatedAnnealing, AutoCompress, AMC, Sensitivity, ADMM, Lottery Ticket)
• P1 - reuse mode: ADL kubeflow support @SparkSnail
• P2 - add/del tag of experiments (pending support from rest server)
• UI: support experiment resume/view/stop on webUI
• P0 - (discussion needed) how to support specifying initial configs (Can we set default values of hyper parameters for HPO? #3801 ) @ultmaster
• P2 - (discussion needed) support constraints on hyper-parameters @ultmaster
• P2 - handle all reject errors @J-shang
• refactor exp config v1 and v2, add validation for v2 @liuzhe-lz Sep 10
  • make python package yaml >= 5.4, double check dependency, and run pipelines
  • add test for scientific annotation (e.g., 1e-4) in search space, issue Exponential Notation is not allowed anymore #4080
• P0 - frameworkcontroller reuse support @SparkSnail Aug 23 Support frameworkcontroller reuse mode #4104

WebUI

• P0 - Adaptive UI (height) for high resolution, current window for high resolution need improvement @JunzhuZhang
  • overview page: blank at the bottom
  • align issue in overview page
  • support searching customized string in the search box

Hyper-parameter tuning

• P1 - Improve HPO benchmarks, add more concrete conclusions
  • add one benchmark and one search space 7.14 HPO Benchmark Fixes and New Features #3925
• P1 - support nested search space @kvartet 6.28 discussion meeting
• P1 - Improve hyperband, PBT with shared storage (dependency on shared storage implementation) @acured
• P1 - upgrade cuda, config_space, docker image @liuzhe-lz 7.21 Bump Python dependencies #3976
• P1 - grid search does not support _name for nested search space (update doc) @kvartet
• P1 - evolution tuner does not mutate sub search space for nested search space @kvartet

jupyter

• jupyter restart, node process is not stopped.
• jupyter does not show error message of node process.
• build jupyterlab-extension as option.

Pipeline

• P0 - support python 3.9 in pipeline (wait for pytorch upgrade from 1.6 to 1.9) @ultmaster Use python 3.9 on pipeline #3881
  • P2 - write ipc test
• P0 - fix and improve pipeline
  • nnimanager ut, nni-ci-gpu-03 failed, maybe due to this machine's environment, cannot be executed through daemon, need reimage -> create a new vm

Doc

• zh Doc translate.
• P2 - update NAS API doc to add more supported evaluators @ultmaster
• doc and demo improvement
  • end2end, based on comparison of different tuners (insights from benchmark, provide reasonable result)
  • demo how to share data, how to use shared storage, mainly refactor doc (view experiment from shared storage)

[QuanluZhang]

Bugbash manager @liuzhe-lz (check other prs)

NAS

• (need review) fix benchmark @ultmaster tester: @QuanluZhang
  • unify architecture format. Aug 20 NAS Benchmark integration (stage 1) #4090
  • provide simple API for query. Aug 20 NAS Benchmark integration (stage 1) #4090
  • support benchmark end2end (test strategies' performance) NAS benchmark integration (stage 2) - download #4205
• support serialization @ultmaster General serialization infrastructure #4172 tester: @liuzhe-lz
• review and improve nas tests stage 1 @ultmaster. [Retiarii] Remove unused code and enrich integration tests #4097 tester: @liuzhe-lz
• improve quick start (notebook) @QuanluZhang. improve NAS quickstart #4094 tester: @ultmaster
  • replace example, update mnist example.
• examples for image classification, object detection, audio (show reasonably good performance) tester: @ultmaster
  • transformer @skeletondyh [Retiarii] Transformer example #4222
• deal with pytorch lightning onnx error for NAS visualization @ultmaster, tester: @QuanluZhang

Model Compression

• support compression v2 @J-shang Compression v2 Stage 1 #3917 [Model Compression] update config list key #4074 [Model Compression] Pruning Scheduler #4089 [Model Compression] admm pruner #4116 [Model Compression] Add more Task Generator #4178 [Model Compression] Add Unit Test #4125 tester: @Fiascolsy
  • tests
  • (bug bash)doc tutorial
• support per-channel quantization @linbinskn support dtype&scheme customization for QAT quantizer #4137 tester: @J-shang
• end2end examples for image classification, object detection, audio, transformer (show reasonably good performance) tester: @J-shang
  • image classification (MobileNet) @xiaowu0162 [End-to-end demo] MobileNetV2 Compression #4102

HPO

• refactor Hyper-opt based algorithm - random tuner. @liuzhe-lz Refactor Hyperopt Tuners (Stage 1) - random tuner #4118 tester: @ultmaster

nnictl & nnimanager & training service @SparkSnail @liuzhe-lz @acured

• experiment config doc update @J-shang Update Experiment Config doc #4131 tester: @linbinskn
• improve doc: clarify the difference between training service storage and shared storage @J-shang Update shared storage doc #4217 tester: @linbinskn

WebUI

• bug fix - Improve trial intermediate result graph. @Lijiaoa Improve trial intermediate result graph: trials have not intermediate default key #4171 tester: @SparkSnail
• bug fix - webui prompts error message. @Lijiaoa [fix issue#4015] webui prompts error message if user set maxExperimentDuration < execDuration #4164 tester: @SparkSnail

regression test needed, or bugs reported during the iteration by external users (need test and close the issues)

• ERROR: Restful server start failed! Failed to create log dir: RangeError: Invalid time value #4077, Restful server started failed #4083 Restful server started failed @acured , tester: @liuzhe-lz
• nni==2.3 tensorboard uninstall then install get version erro,how solve it? nni==2.3 tensorboard uninstall then install get __version__ erro,how solve it? #4087 @SparkSnail , resolver&tester: @liuzhe-lz
• Exponential Notation is not allowed anymore Exponential Notation is not allowed anymore #4080 @QuanluZhang , tester: @linbinskn
• Small spelling error when executing NNI via Python Script #4158 @acured , tester: @linbinskn
• assert len(set(num_channels_list)) == 1 assert len(set(num_channels_list)) == 1 #4160 @J-shang , tester: @J-shang
• Task is always in a waiting state in the remote machine Task is always in a waiting state in the remote machine #3905 (not an 2.4 regression but a 2.1 regression) @acured , tester: @SparkSnail
• NNI installation issue with ConfigSpace-0.4.7 NNI installation issue with ConfigSpace-0.4.7 #3909 (not sure whether this is a regression or not) @liuzhe-lz , tester: @liuzhe-lz
• Runtime Error during L1FilterPruner (and other one-shot pruners) Runtime Error during L1FilterPruner (and other one-shot pruners)  #3944 @zheng-ningxin , tester: @J-shang
• submit task to OpenPAI cluster failed submit task to OpenPAI cluster failed #4022 @SparkSnail , tester: @QuanluZhang
• Kubeflow setup #3978 @SparkSnail , tester: @SparkSnail
• AutoCompress with level pruner leads to runtime error AutoCompress with level pruner leads to runtime error #3948 @J-shang, tester: @J-shang
• SPOS hardware-aware nas multi_trial.py don't work SPOS hardware-aware nas multi_trial.py don't work #4198 @JiahangXu , tester: @ultmaster

[liuzhe-lz]

BUG BASH (Target date Friday 10/22)

PRs seemingly need to test:

model compression

• chenbohua3
  • @linbinskn restore weight by default for qat quantizer #3992
  • @linbinskn fix wrong quantization target in weight quantization #4038
  • @linbinskn change signature of quantize_input #4039
  • @linbinskn add quantize_input to QAT quantizer #4084
  • @linbinskn should use .data in quantization wrapper #4113
  • @linbinskn support dp multi-gpu training for QAT quantizer #4127
  • @linbinskn support dtype&scheme customization for QAT quantizer #4137
    • Problem found, @linbinskn submitted a fix PR [Quantization] fix QAT export param #4252
• Fiascolsy
  • @J-shang Support 'op_partial_names' in config_list #4184 Add examples for 'op_partial_names' and use more proper words. #4225
  • @J-shang Fix v2 level pruner default config bug #4245
• J-shang
  • @Fiascolsy Compression v2 Stage 1 #3917 [Model Compression] update config list key #4074 [Model Compression] Pruning Scheduler #4089 [Model Compression] admm pruner #4116 [Model Compression] Add more Task Generator #4178
  • @Fiascolsy [Model Compression] add scheduler high level api #4236
• linbinskn
  • @J-shang [Model Compression Quantization] Unify variable name #3990 Waiting for [Quantization] fix QAT export param #4252
  • @J-shang [Model Compression] fix data contiguous bug in amc example #4049
  • @J-shang [Quantization] support bn-folding for lsq #4148
  • @J-shang [Quantization] support load_calibration_config #4163
• xiaowu0162
  • @J-shang [End-to-end demo] MobileNetV2 Compression #4102
• zheng-ningxin
  • @J-shang Set the strict to false in mask conflict utils. #4078
  • @J-shang Loose the group dependency #4128
  • @J-shang support directly loading the mask #4144
  • @J-shang Error code for Speedup Module #4173
  • @J-shang Fix a small bug raised by issue 4221. #4241

nas

• hzhua
  • @QuanluZhang Fix interface of CGO's accelerator to support pytorch-lightning 1.4.2 #4075 [Retiarii] Bugfix: wrong device placement and invalid CUDA ordinal when using CGO engine #4086 [Retiarii] Retry a failed multi-model trial by disabling CGO in CGOExecutionEngine  #4098
• JiahangXu
  • @ultmaster Fix bug in graph converter from Graph_gen #4092
  • @ultmaster Fix: refine shape attribute #4214
• QuanluZhang
  • @ultmaster improve NAS quickstart #4094
• ultmaster
  • @QuanluZhang NAS Benchmark integration (stage 1) #4090 NAS benchmark integration (stage 2) - download #4205
  • @zheng-ningxin Fix flops counter for bs>1 #4154
  • @liuzhe-lz General serialization infrastructure #4172

other

• JunweiSUN
  • @liuzhe-lz hotfix: fix time format issue #4108
• Lijiaoa
  • @SparkSnail [fix issue#4015] webui prompts error message if user set maxExperimentDuration < execDuration #4164
  • @SparkSnail Improve trial intermediate result graph: trials have not intermediate default key #4171
• liuzhe-lz
  • @QuanluZhang Make JupyterLab extension optional for develop mode #4147
  • @QuanluZhang Add NNI icon in JupyterLab extension #4177
  • @ultmaster Refactor Hyperopt Tuners (Stage 1) - random tuner #4118 Refactor Hyperopt Tuners (Stage 2) - util update and test #4238
• SparkSnail
  • @acured Support frameworkcontroller reuse mode #4104
  • @acured Fix aml stop environment logic #4199

contributors

• alibaba
  • (no environment) HPO: Alibaba DSW+DLC support #4055 @weidankong need you to test, verify and confirm, thanks!
  • @liuzhe-lz jupyter_ext_dev: pull request for jupyter extension 2.x and 3.x recap… #4167
  • @liuzhe-lz jupyter_ext: Added reminding information when there isn't nni kicked off #4188
• individuals
  • @Fiascolsy added config_windows.yml for examples in /examples/trials #4121
    • @liuzhe-lz and @Fiascolsy follow up on problems found during tests.
  • @ultmaster Pass ConfigSpace definition file directly to BOHB #4153 @QuanluZhang
  • (no need to test) Update amc_pruner.py #4161
  • (no need to test) Update ConfigSpace to 0.4.11 #4208

[liuzhe-lz]

To fix:

• Tutorial for Python experiment API training service @liuzhe-lz next release
• lightning trainer doc @ultmaster
• experiment config code default value @J-shang
• HPO benchmark
• Add "experimental" to CGO doc

[QuanluZhang]

• double check gpuIndices, trialGpuNumber, @linbinskn @liuzhe-lz

[cruiseliu]

Model Compression

• New major version of pruning framework (doc)
  • Iterative pruning is more automated, users can use less code to implement iterative pruning.
  • Support exporting intermediate models in the iterative pruning process.
  • The implementation of the pruning algorithm is closer to the paper.
  • Users can easily customize their own iterative pruning by using PruningScheduler.
  • Optimize the basic pruners underlying generate mask logic, easier to extend new functions.
  • Optimized the memory usage of the pruners.
• MobileNetV2 end-to-end example (notebook)
• Improved QAT quantizer
  • support dtype and scheme customization
  • support dp multi-gpu training
  • support load_calibration_config
• Model speed-up now supports directly loading the mask
  • work item: add ModelSpeedup API reference doc
• Support speed-up depth-wise convolution
• Support bn-folding for LSQ quantizer
• Support QAT and LSQ resume from PTQ

Neural Architecture Search

• NAS benchmark (doc)
  • Support benchmark table lookup in experiments
  • New data preparation approach
• Improved quick start doc
• Experimental CGO execution engine (doc)

Hyper-Parameter Optimization

• New training platform: Alibaba DSW+DLC (doc)
• Support passing ConfigSpace definition directly to BOHB (doc) (thanks to khituras)
• Reformatted experiment config doc
• Added example config files for Windows (thanks to politecat314)
• FrameworkController now supports reuse mode

Fixed Bugs

• Experiment cannot start due to platform timestamp format (issue ERROR: Restful server start failed! Failed to create log dir: RangeError: Invalid time value #4077 Restful server started failed #4083)
• Cannot use 1e-5 in search space (issue Exponential Notation is not allowed anymore #4080)
• Dependency version conflict caused by ConfigSpace (issue NNI installation issue with ConfigSpace-0.4.7 #3909) (thanks to jexxers)
• Hardware-aware SPOS example does not work (issue SPOS hardware-aware nas multi_trial.py don't work #4198)
• Web UI show wrong remaining time when duration exceeds limit (issue The remaining time is error when the spent time has exceeded the duration time. #4015)
• cudnn.deterministic is always set in AMC pruner (cudnn.deterministic been set while import, suggestion to avoid this kind of behavior #4117) thanks to mstczuo

And...

• New emoticons!