TensorFlow v2 Support

[liuzhe-lz]

Scope for July release:

• Review on how the current NNI Pytorch model compression implementation, survey and compare the corresponding TF 2.0 supportability. ETA 7/22
• design depends on the survey results

---

This issue tracks pending jobs for TensorFlow v2 support.

• HPO
• NAS: classic NAS tuning + ENAS, DARTS algo refresh
• model compression
• feature engineering

---

• Examples
  • P1 - Rewrite TFv1 example trials (naive example) with TFv2 (May release)
  • P2 - Rewrite TFv1 example trials (DARTS) with TFv2
  • P0 NAS performance improvement (June)
  • P2 Move pipelines to v2
• NAS
  • P0 - SDK and ENAS (May release)
  • Implement algorithms
• Static graph speed-up for NAS and Model Compression
  • Ad-hoc optimization for built-in algorithms
  • Public optimizing API
• Update building- and releasing-related scripts
• Update documentation

[scarlett2018]

released in 1.6, will keep on the rest in June release

• P1 - Rewrite TFv1 example trials (naive example) with TFv2
• P0 - SDK and ENAS

[scarlett2018]

[3.1] performance for all, slower than pytorch, GPU usage is slow, need improvement
[3.2] test pipeline for TF2.0
[3.3] TF NAS API style refactor -> to tf style
[3.4] model compression for TF2.0
[3.5] Feature Enginerring for TF2.0
[3.6] Support all NAS algorithms for TF2.0