Skip to content

Releases: dimajix/flowman

0.14.2

12 Oct 10:04
Compare
Choose a tag to compare
  • Upgrade to Spark 2.4.7 and Spark 3.0.1
  • Clean up dependencies
  • Disable build of Docker image
  • Update examples

0.14.1

03 Oct 14:26
Compare
Choose a tag to compare
  • Fix dropping of partitions which could cause issues on CDH6

0.14.0

14 Dec 11:25
Compare
Choose a tag to compare
  • Fix AWS plugin for Hadoop 3.x
  • Improve setup of logging
  • Shade Velocity for better interoperability with Spark 3
  • Add new web hook facility in namespaces and jobs
  • Existing targets will not be overwritten anymore by default. Either use the --force command line option, or set
    the configuration property flowman.execution.target.forceDirty to true for the old behaviour.
  • Add new command line option --keep-going
  • Implement new com.dimajix.spark.io.DeferredFileCommitProtocol which can be used by setting the Spark configuration
    parameter spark.sql.sources.commitProtocolClass
  • Add new flowshell application

0.13.1

14 Dec 11:25
Compare
Choose a tag to compare
  • Code improvements
  • Do not implicitly set SPARK_MASTER in configuration
  • Add support for CDH6
  • Add support for Spark 3.0
  • Improve support for Hadoop 3.x

0.13.0

14 Dec 11:26
Compare
Choose a tag to compare
  • Refactor Maven module structure
  • Implement new Scala DSL for creating projects
  • Fix ordering bug in target execution
  • Merge 'migrate' phase into 'create' phase
  • Rename 'input' field to 'mapping' in most targets
  • Lots of minor code improvements