Release version 1.1.0

CHANGELOG.md

@@ -18,7 +18,7 @@ Breaking changes will be documented in this changelog file for each version.
 
 # Changelog
 
-## Version 1.1.0 - Upcoming
+## Version 1.1.0 - 2023-10-10
 
 * github-413: Support Azure Key Vault for retrieving secrets
 * github-415: Improve documentation for Velocity templating

QUICKSTART.md

@@ -16,7 +16,7 @@ Fortunately, Spark is rather simple to install locally on your machine:
 
 ### Download & Install Spark
 
-As of this writing, the latest release of Flowman is 1.0.0 and is available prebuilt for Spark 3.3.2 on the Spark
+As of this writing, the latest release of Flowman is 1.1.0 and is available prebuilt for Spark 3.4.1 on the Spark
 homepage. So we download the appropriate Spark distribution from the Apache archive and unpack it.
 
 ```shell
@@ -25,10 +25,10 @@ mkdir playground
 cd playground
 
 # Download and unpack Spark & Hadoop
-curl -L https://archive.apache.org/dist/spark/spark-3.3.2/spark-3.3.2-bin-hadoop3.tgz | tar xvzf -
+curl -L https://archive.apache.org/dist/spark/spark-3.4.1/spark-3.4.1-bin-hadoop3.tgz | tar xvzf -
 
 # Create a nice link
-ln -snf spark-3.3.2-bin-hadoop3 spark
+ln -snf spark-3.4.1-bin-hadoop3 spark
 ```
 
 The Spark package already contains Hadoop, so with this single download you already have both installed and integrated with each other.
@@ -47,14 +47,14 @@ environment variables:
 ## 2. Install Flowman
 
 You find prebuilt Flowman packages on the corresponding release page on GitHub. For this quickstart, we chose
-`flowman-dist-1.0.0-oss-spark3.3-hadoop3.3-bin.tar.gz` which nicely fits to the Spark package we just downloaded before.
+`flowman-dist-1.1.0-oss-spark3.4-hadoop3.3-bin.tar.gz` which nicely fits to the Spark package we just downloaded before.
 
 ```shell
 # Download and unpack Flowman
-curl -L https://github.com/dimajix/flowman/releases/download/1.0.0/flowman-dist-1.0.0-oss-spark3.3-hadoop3.3-bin.tar.gz | tar xvzf -
+curl -L https://github.com/dimajix/flowman/releases/download/1.1.0/flowman-dist-1.1.0-oss-spark3.4-hadoop3.3-bin.tar.gz | tar xvzf -
 
 # Create a nice link
-ln -snf flowman-1.0.0-oss-spark3.3-hadoop3.3 flowman
+ln -snf flowman-1.1.0-oss-spark3.4-hadoop3.3 flowman
 ```
 
 ### Flowman Configuration

docker/pom.xml

@@ -12,7 +12,7 @@
     <parent>
         <groupId>com.dimajix.flowman</groupId>
         <artifactId>flowman-root</artifactId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

docs/conf.py

@@ -64,9 +64,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.0'
+version = '1.1'
 # The full version, including alpha/beta/rc tags.
-release = '1.0.0'
+release = '1.1.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/quickstart.md

@@ -14,7 +14,7 @@ Of course, you need a working Docker installation, which should be quite easy. T
 containing Flowman via the following command:
 
 ```shell
-docker run --rm -ti dimajix/flowman:1.0.0-oss-spark3.3-hadoop3.3 bash   
+docker run --rm -ti dimajix/flowman:1.1.0-oss-spark3.4-hadoop3.3 bash   
 ```
 
 Note that this simply starts a bash shell, but Flowman is only away some fingertips. 
@@ -26,7 +26,7 @@ may want to make local data accessible to Flowman or you may simply want to exec
 on your local machine. This can be easily achieved as follows:
 
 ```shell
-docker run --rm -ti -v /home/kaya/flowman/example/weather:/home/flowman/local dimajix/flowman:1.0.0-oss-spark3.3-hadoop3.3 bash   
+docker run --rm -ti -v /home/kaya/flowman/example/weather:/home/flowman/local dimajix/flowman:1.1.0-oss-spark3.4-hadoop3.3 bash   
 ```
 
 This will mount the local directory `/home/kaya/flowman/example/weather` on your host computer into the Docker

docs/releases.md

@@ -30,7 +30,7 @@ Breaking changes will be documented in the changelog file for each version.
 The following gives an (incomplete) list of past releases of the last 12 months. This will help you to spot breaking 
 changes over time.
 
-### Version 1.1.0 - Upcoming
+### Version 1.1.0 - 2023-10-10
 
 * github-413: Support Azure Key Vault for retrieving secrets
 * github-415: Improve documentation for Velocity templating

docs/setup/building.md

@@ -13,13 +13,13 @@ versions. The naming is very simple:
 flowman-dist-<version>-oss-spark<spark-version>-hadoop<hadoop-version>-bin.tar.gz
 ```
 You simply have to use the package which fits to the Spark and Hadoop versions of your environment. For example the
-package of Flowman 0.24.1 and for Spark 3.1 and Hadoop 3.2 would be
+package of Flowman 1.1.0 and for Spark 3.4 and Hadoop 3.3 would be
 ```
-flowman-dist-0.24.1-oss-spark3.1-hadoop3.2-bin.tar.gz
+flowman-dist-1.1.0-oss-spark3.4-hadoop3.3-bin.tar.gz
 ```
 and the full URL then would be
 ```
-https://github.com/dimajix/flowman/releases/download/0.24.1/flowman-dist-0.24.1-oss-spark3.1-hadoop3.2-bin.tar.gz
+https://github.com/dimajix/flowman/releases/download/1.1.0/flowman-dist-1.1.0-oss-spark3.4-hadoop3.3-bin.tar.gz
 ```
 
 ## Build with Maven

docs/setup/docker.md

@@ -15,12 +15,12 @@ We publish Flowman Docker images on [Docker Hub](https://hub.docker.com/reposito
 which are good enough for local work. You can easily start a Flowman session in Docker as follows:
 
 ```shell
-docker run --rm -ti dimajix/flowman:1.0.0-oss-spark3.3-hadoop3.3 bash
+docker run --rm -ti dimajix/flowman:1.1.0-oss-spark3.4-hadoop3.3 bash
 ```
 When you are using [git bash](https://git-scm.com/download/win), you will probably need to use `winpty`, which 
 translates to the following command
 ```shell
-winpty docker run --rm -ti dimajix/flowman:1.0.0-oss-spark3.3-hadoop3.3 bash
+winpty docker run --rm -ti dimajix/flowman:1.1.0-oss-spark3.4-hadoop3.3 bash
 ```
 
 Once the Docker image has started, you will be presented with a bash prompt. Then you can easily build the
@@ -35,7 +35,7 @@ flowexec -f examples/weather job build main
 By using Docker volumes, you can easily mount a Flowman project into the Docker container, for example
 
 ```shell
-docker run --rm -ti --mount type=bind,source=$(pwd)/my_project,target=/home/flowman/my_project dimajix/flowman:1.0.0-oss-spark3.3-hadoop3.3 bash
+docker run --rm -ti --mount type=bind,source=$(pwd)/my_project,target=/home/flowman/my_project dimajix/flowman:1.1.0-oss-spark3.4-hadoop3.3 bash
 ```
 The command above will start a Docker container running Flowman, and the local subdirectory `my_project` within the 
 current working directory is mounted into the container at `/home/flowman/my_project`. Then you open your project
@@ -58,7 +58,7 @@ version: "3"
 services:
   flowman:
     # Set the appropriate Flowman Docker image of your choice.
-    image: dimajix/flowman:1.0.0-oss-spark3.3-hadoop3.3
+    image: dimajix/flowman:1.1.0-oss-spark3.4-hadoop3.3
     # Let Docker start a bash if nothing else is specified.
     command: bash
     # Mount local volumes into the Docker container. Adjust the example entries to your needs!

docs/setup/emr.md

@@ -29,13 +29,13 @@ opening firewall ports, providing your public ssh key and more. You can read det
 Once you successfully logged in to the master node, you need to download an EMR optimized version of Flowman using
 `wget` or `curl`:
 ```shell
-wget https://github.com/dimajix/flowman/releases/download/1.0.0/flowman-dist-1.0.0-emr6.10-spark3.3-hadoop3.3-bin.tar.gz
+wget https://github.com/dimajix/flowman/releases/download/1.1.0/flowman-dist-1.0.0-emr6.12-spark3.4-hadoop3.3-bin.tar.gz
 ```
 Next you unpack Flowman as follows:
 ```shell
-tar xvzf flowman-dist-1.0.0-emr6.10-spark3.3-hadoop3.3-bin.tar.gz
+tar xvzf flowman-dist-1.1.0-emr6.12-spark3.4-hadoop3.3-bin.tar.gz
 ```
-This will create a directory `flowman-1.0.0-emr6.10-spark3.3-hadoop3.3` which contains all executables and libraries
+This will create a directory `flowman-1.1.0-emr6.12-spark3.4-hadoop3.3` which contains all executables and libraries
 of Flowman.
 
 #### Directory Layout
@@ -191,7 +191,7 @@ In this case, we need a small `pom.xml` file (this is the build descriptor for M
             <plugin>
                 <groupId>com.dimajix.flowman.maven</groupId>
                 <artifactId>flowman-maven-plugin</artifactId>
-                <version>0.1.0</version>
+                <version>0.4.0</version>
                 <extensions>true</extensions>
                 <configuration>
                     <deploymentDescriptor>deployment.yml</deploymentDescriptor>
@@ -216,7 +216,7 @@ This deployment descriptor contains all required information
 ```yaml
 flowman:
   # Specify the Flowman version to use
-  version: 1.0.0-emr6.10-spark3.3-hadoop3.3
+  version: 1.1.0-emr6.12-spark3.4-hadoop3.3
   plugins:
     # Specify the list of plugins to use
     - flowman-avro

docs/setup/installation.md

@@ -21,7 +21,7 @@ needs to match the ones of your platform
 
 ### Download & Install Spark
 
-As of this writing, the latest release of Flowman is 1.0.0 and is available prebuilt for Spark 3.3.2 on the Spark
+As of this writing, the latest release of Flowman is 1.1.0 and is available prebuilt for Spark 3.4.1 on the Spark
 homepage. So we download the appropriate Spark distribution from the Apache archive and unpack it.
 
 ```shell
@@ -30,8 +30,8 @@ mkdir playground
 cd playground
 
 # Download and unpack Spark & Hadoop
-curl -L https://archive.apache.org/dist/spark/spark-3.3.2/spark-3.3.2-bin-hadoop3.tgz | tar xvzf -# Create a nice link
-ln -snf spark-3.3.2-bin-hadoop3 spark
+curl -L https://archive.apache.org/dist/spark/spark-3.4.1/spark-3.4.1-bin-hadoop3.tgz | tar xvzf -# Create a nice link
+ln -snf spark-3.4.1-bin-hadoop3 spark
 ```
 The Spark package already contains Hadoop, so with this single download you already have both installed and integrated with each other.
 
@@ -64,13 +64,13 @@ versions. The naming is straight forward:
 flowman-dist-<version>-oss-spark<spark-version>-hadoop<hadoop-version>-bin.tar.gz
 ```
 You simply have to use the package which fits to the Spark and Hadoop versions of your environment. For example, the
-package of Flowman 1.0.0 and for Spark 3.3 and Hadoop 3.3 would be
+package of Flowman 1.1.0 and for Spark 3.4 and Hadoop 3.3 would be
 ```
-flowman-dist-1.0.0-oss-spark3.3-hadoop3.3-bin.tar.gz
+flowman-dist-1.1.0-oss-spark3.4-hadoop3.3-bin.tar.gz
 ```
 and the full URL then would be
 ```
-https://github.com/dimajix/flowman/releases/download/1.0.0/flowman-dist-1.0.0-oss-spark3.3-hadoop3.3-bin.tar.gz
+https://github.com/dimajix/flowman/releases/download/1.1.0/flowman-dist-1.1.0-oss-spark3.4-hadoop3.3-bin.tar.gz
 ```
 
 ### Supported Spark Environments
@@ -84,11 +84,11 @@ Flowman is available for many different Spark/Hadoop environments. The following
 | Open Source      | 3.0.3 | 3.2    | 11   | 2.12  | oss-spark3.0-hadoop3.2        |
 | Open Source      | 3.1.2 | 2.7    | 11   | 2.12  | oss-spark3.1-hadoop2.7        |
 | Open Source      | 3.1.2 | 3.2    | 11   | 2.12  | oss-spark3.1-hadoop3.2        |
-| Open Source      | 3.2.3 | 2.7    | 11   | 2.12  | oss-spark3.2-hadoop2.7        |
-| Open Source      | 3.2.3 | 3.3    | 11   | 2.12  | oss-spark3.2-hadoop3.3        |
-| Open Source      | 3.3.2 | 2.7    | 11   | 2.12  | oss-spark3.3-hadoop2.7        |
-| Open Source      | 3.3.2 | 3.3    | 11   | 2.12  | oss-spark3.3-hadoop3.3        |
-| Open Source      | 3.4.0 | 3.3    | 11   | 2.12  | oss-spark3.4-hadoop3.3        |
+| Open Source      | 3.2.4 | 2.7    | 11   | 2.12  | oss-spark3.2-hadoop2.7        |
+| Open Source      | 3.2.4 | 3.3    | 11   | 2.12  | oss-spark3.2-hadoop3.3        |
+| Open Source      | 3.3.4 | 2.7    | 11   | 2.12  | oss-spark3.3-hadoop2.7        |
+| Open Source      | 3.3.3 | 3.3    | 11   | 2.12  | oss-spark3.3-hadoop3.3        |
+| Open Source      | 3.4.1 | 3.3    | 11   | 2.12  | oss-spark3.4-hadoop3.3        |
 | AWS EMR 6.10     | 3.3.1 | 3.3    | 1.8  | 2.12  | emr6.10-spark3.3-hadoop3.3    |
 | AWS EMR 6.12     | 3.4.0 | 3.3    | 1.8  | 2.12  | emr6.12-spark3.4-hadoop3.3    |
 | Azure Synapse    | 3.3.1 | 3.3    | 1.8  | 2.12  | synapse3.3-spark3.3-hadoop3.3 |

docs/setup/synapse.md

@@ -45,7 +45,7 @@ In this case, we need a small `pom.xml` file (this is the build descriptor for M
             <plugin>
                 <groupId>com.dimajix.flowman.maven</groupId>
                 <artifactId>flowman-maven-plugin</artifactId>
-                <version>0.1.0</version>
+                <version>0.4.0</version>
                 <extensions>true</extensions>
                 <configuration>
                     <deploymentDescriptor>deployment.yml</deploymentDescriptor>
@@ -70,7 +70,7 @@ This deployment descriptor contains all required information
 ```yaml
 flowman:
   # Specify the Flowman version to use
-  version: 1.0.0-synapse3.3-spark3.3-hadoop3.3
+  version: 1.1.0-synapse3.3-spark3.3-hadoop3.3
   plugins:
     # Specify the list of plugins to use
     - flowman-avro

docs/tutorial/12-deployment.md

@@ -89,15 +89,15 @@ The `pom.xml` generated by the archetype will look as follows:
     <encoding>UTF-8</encoding>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
     <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
-    <flowman.version>1.0.0</flowman.version>
+    <flowman.version>1.1.0</flowman.version>
   </properties>
 
   <build>
     <plugins>
       <plugin>
         <groupId>com.dimajix.flowman.maven</groupId>
         <artifactId>flowman-maven-plugin</artifactId>
-        <version>0.3.0</version>
+        <version>0.4.0</version>
         <extensions>true</extensions>
         <configuration>
           <deploymentDescriptor>deployment.yml</deploymentDescriptor>

docs/workflow/maven-classic.md

@@ -80,7 +80,7 @@ The `pom.xml` generated by the archetype will look as follows:
     <parent>
         <groupId>com.dimajix.flowman</groupId>
         <artifactId>flowman-parent</artifactId>
-        <version>1.0.0-SNAPSHOT</version>
+        <version>1.1.0</version>
     </parent>
 
     <build>

docs/workflow/maven-plugin.md

@@ -90,7 +90,7 @@ The `pom.xml` generated by the archetype will look as follows:
             <plugin>
                 <groupId>com.dimajix.flowman.maven</groupId>
                 <artifactId>flowman-maven-plugin</artifactId>
-                <version>0.1.0</version>
+                <version>0.4.0</version>
                 <extensions>true</extensions>
                 <configuration>
                     <deploymentDescriptor>deployment.yml</deploymentDescriptor>
@@ -120,7 +120,7 @@ for the Flowman Maven plugin. Its contents look as follows:
 ```yaml
 flowman:
   # Specify the Flowman version to use
-  version: 1.0.0
+  version: 1.1.0
   plugins:
     # Specify the list of plugins to use
     - flowman-avro

flowman-common-java/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <groupId>com.dimajix.flowman</groupId>
         <artifactId>flowman-root</artifactId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-common-jersey/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <artifactId>flowman-root</artifactId>
         <groupId>com.dimajix.flowman</groupId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-common/pom.xml

@@ -11,7 +11,7 @@
     <parent>
 	    <groupId>com.dimajix.flowman</groupId>
 	    <artifactId>flowman-root</artifactId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-core/pom.xml

@@ -11,7 +11,7 @@
     <parent>
 	    <groupId>com.dimajix.flowman</groupId>
 	    <artifactId>flowman-root</artifactId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-dist/pom.xml

@@ -12,7 +12,7 @@
     <parent>
 	    <groupId>com.dimajix.flowman</groupId>
 	    <artifactId>flowman-root</artifactId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-dsl/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <artifactId>flowman-root</artifactId>
         <groupId>com.dimajix.flowman</groupId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-hub/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <artifactId>flowman-root</artifactId>
         <groupId>com.dimajix.flowman</groupId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-kernel-api/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <artifactId>flowman-root</artifactId>
         <groupId>com.dimajix.flowman</groupId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-kernel-common/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <artifactId>flowman-root</artifactId>
         <groupId>com.dimajix.flowman</groupId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-kernel-server/pom.xml

@@ -11,7 +11,7 @@
     <parent>
         <artifactId>flowman-root</artifactId>
         <groupId>com.dimajix.flowman</groupId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

flowman-kernel-tools/pom.xml

@@ -11,7 +11,7 @@
     <parent>
 	    <groupId>com.dimajix.flowman</groupId>
 	    <artifactId>flowman-root</artifactId>
-        <version>1.1.0-SNAPSHOT</version>
+        <version>1.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>