Flowman 0.27.0
The highlights of this release are
- New ' jdbcCommand' target for executing arbitrary SQL statements for JDBC sinks
- Support direct SQL statements in JDBC relations for creating tables
- Upgrade Delta Lake to 2.0.0/2.1.0 (for Spark 3.2 and 3.3 respectively)
- Better error messages
- Bug fixes and other improvements
In detail, this release contains the following changes:
- github-232: [BUG] Column descriptions should be propagates in UNIONs
- github-233: [BUG] Missing Hadoop dependencies for S3, Delta, etc
- github-235: Implement new
rest
hook with fine control - github-229: A build target should not fail if Impala "COMPUTE STATS" fails
- github-236: 'copy' target should not apply output schema
- github-237: jdbcQuery relation should use fields "sql" and "file" instead of "query"
- github-239: Allow optional SQL statement for creating jdbcTable
- github-238: Implement new 'jdbcCommand' target
- github-240: [BUG] Data quality checks in documentation should not fail on NULL values
- github-241: Throw an error on duplicate entity definitions
- github-220: Upgrade Delta-Lake to 2.0 / 2.1
- github-242: Switch to Spark 3.3 as default
- github-243: Use alternative Spark MS SQL Connector for Spark 3.3
- github-244: Generate project HTML documentation with optional external CSS file