Skip to content

Flowman 0.30.0
Compare
Choose a tag to compare
@kupferk kupferk released this 04 Jan 06:10
0.30.0
9d7d051

This release has a strong focus on improving deploying Flowman projects has a self-contained fat jar. In addition a new "executions" section in the job entity will give you better control of what target is executed in which phase, especially when executing parameter ranges (for example to process multiple days in a row).

  • github-278: Parallelize execution of data quality checks. This also introduces a new configuration property
    flowman.execution.check.parallelism (default 1)
  • github-282: Improve implementation for counting records
  • github-288: Support reading local CSV files from fatjar
  • github-290: Simplify specifying project name in fatjar
  • github-291: Simplify create/destroy Relation interface
  • github-292: Upgrade AWS EMR to 6.9
  • github-289: Color log output via log4j configuration (requires log4j 2.x)
  • Bump postgresql from 42.4.1 to 42.4.3 in /flowman-plugins/postgresql
  • Bump loader-utils from 1.4.0 to 1.4.2
  • Bump json5 from 2.2.1 to 2.2.3
  • github-293: [BUG] Fatal exceptions in parallel mapping instantiation cause deadlock
  • github-273: Support projects contained in (fat) jar files
  • github-294: [BUG] Parallel execution should not execute more targets after errors
  • github-295: Create build profile for CDP 7.1 with Spark 3.2
  • github-296: Update npm dependencies (vuetify & co)
  • github-297: Parametrize when to execute a specific phase
  • github-299: Move migrationPolicy and migrationStrategy from target into relation
  • github-115: Implement additional build policy in relation target for forcing dirty. This also introduces a new
    configuration property flowman.default.target.buildPolicy (default COMPAT).
  • github-298: Support fine-grained control when to execute each target of a job
  • github-300: Implement new 'observe' mapping
  • github-301: Upgrade Spark to 3.2.3
  • github-302: Upgrade DeltaLake to 2.2.0
  • github-303: Use multi-stage build for Docker image
  • github-304: Upgrade Cloudera profile to CDP 7.1.8
  • github-312: Fix build with Spark 2.4 and Maven 3.8

This version is fully backwards compatible until and including version 0.27.0.

Assets 16
Loading