0.5.0

Highlight

• Provide a new set of API for Tensorflow and PyTorch. It is more intuitive and user-friendly.
• Support FlinkML estimator style API in Python, you can check out the quick start examples for Tensorflow 1.15 and PyTorch.
• Support iterative training with data from Flink table.
• Dependencies management of Python API is now seamlessly integrated with the dependencies management of PyFlink, detail can be found here.

What's Changed

• Bump dl-on-flink version by @Sxnan in #700
• Update quick start doc by @Sxnan in #695
• Bump hadoop-common from 2.8.0 to 2.10.1 by @dependabot in #697
• Bump junit from 4.12 to 4.13.1 by @dependabot in #688
• Bump httpclient from 4.5.6 to 4.5.13 by @dependabot in #684
• Recommend using virtualenv for quick start by @Sxnan in #703
• Rename modules from flink-ml-* to dl-on-flink-* by @Sxnan in #702
• Add inference example to quick start by @Sxnan in #705
• Run example in github action nighlty by @Sxnan in #707
• Include pybind as submodule by @Sxnan in #708
• Add github workflow to prepare release by @Sxnan in #709
• Remove extra space of commands in quick start by @Sxnan in #710
• Increase timeout waiting for AM in testing by @Sxnan in #712
• Fix unstable unit test AppMasterServerTest#multiRegisterNode by @Sxnan in #714
• Use the archive url to download flink binary by @Sxnan in #715
• Add checkstyle and spotless by @Sxnan in #718
• Refactor RoleUtils and TFUtils API by @Sxnan in #716
• NodeUtils set python path properties with entry file path at first by @Sxnan in #721
• Install dev wheels for nightly build by @Sxnan in #722
• AMInputFormat wait for am node finish before return from reachedEnd by @Sxnan in #724
• Support Tensorflow on Flink iterative batch training by @Sxnan in #723
• Add Python TFUtils API by @Sxnan in #726
• Update TensorBoardPythonRunner by @Sxnan in #730
• Update DL on Flink Tensorflow Python API by @Sxnan in #731
• Use RowCSVCoding as TF default encoding/decoding class by @Sxnan in #732
• Run python unit test in GitHub workflow by @Sxnan in #734
• Support pyFlink env configuration by @Sxnan in #733
• Bump Supported Tensorflow version to 2.4.x by @Sxnan in #736
• Update Project Copyright Header by @Sxnan in #737
• Introduce the New PyTorch API by @Sxnan in #735
• Rename PytorchContext to PyTorchContext by @Sxnan in #738
• Update quick start examples by @Sxnan in #739
• Fix prepare release workflow by @Sxnan in #741
• Specify protobuf version for dl-on-flink-tensorflow wheel build by @Sxnan in #742
• DL on Flink support FlinkML API by @Sxnan in #743
• Fix examples by @Sxnan in #745
• Avoid tight loop in NodeOperator#onEpochWatermarkIncremented by @Sxnan in #747
• Support adding extra cluster config properties to estimator by @Sxnan in #746
• Bump fastjson from 1.2.49 to 1.2.83 by @dependabot in #748
• Add flink ml estimator examples by @Sxnan in #750
• Update NodeOperator and NodeInputFormat close timeout by @Sxnan in #749
• Fix unstable failover test by @Sxnan in #751
• NodeServer#stopMLRunner should not swallow the InterruptedException by @Sxnan in #752
• Add Java examples for Tensorflow and PyTorch by @Sxnan in #753
• Add DL on Flink API docs by @Sxnan in #754
• NodeServer is interrupted periodically if it is not terminated on time by @Sxnan in #755
• Specify grpcio version upper bound by @Sxnan in #756

Full Changelog: 0.4.0...0.5.0-rc0

0.4.0

What's Changed

• [Deep Learning on Flink] Update Flink to 1.14 by @Sxnan in #662
• Cleanup the repo after removing ai-flow by @Sxnan in #668
• Standardize python package version by @Sxnan in #670
• Prepare jar release to maven central by @Sxnan in #673
• Update python build by @Sxnan in #674
• flink-ml-tensorflow python support flink-ml api by @Sxnan in #672
• Fix protobuf version conflict by @Sxnan in #675
• Add a quick start example and docs by @Sxnan in #678
• Refactor build process and docs by @Sxnan in #680
• Use java lib to unzip file by @Sxnan in #682
• Bump log4j-api from 2.11.1 to 2.17.1 by @dependabot in #686
• Bump log4j-core from 2.11.1 to 2.17.1 by @dependabot in #685
• Bump protobuf-java from 3.8.0 to 3.16.1 by @dependabot in #671
• Nightly build upload flink-ml-dist bin package by @Sxnan in #690
• Add ut for flink-ml-tensorflow by @Sxnan in #689
• fix org.apache.hadoop.conf.Configuration cannot set in the class by @queyuexzy in #693
• flink-ml-tensorflow shade guava and apache commons library by @Sxnan in #692
• Add license check and update license notice by @Sxnan in #691
• Whitelist tensorflow dependency for auditwheel making manylinux wheel by @Sxnan in #698

Full Changelog: 0.2.2...0.4.0

support acquire gpu information

support acquire gpu information and exclude some jars which conflict with flink 1.11.0

Merge pull request #58 from tywtyw/branch-0.2.1

update flink to 1.11

flink-ai-extended 0.1.0 published

Issues

1. Support tensorflow running on flink.
2. Support pytorch running on flink.