Dependencies should be verbosely enumerated or automated

[slominskir]

We need to either explicitly list the jar file dependencies needed by the transform or possibly bundle them up with the release. Currently installation requires downloading dependencies manually and copying them into the Kafka libs directory. This needs to either be documented or automated. The main dependencies are : avro and confluent avro (which depends on 6 other confluent jars and Google Guava). For example currently to install this transform I download confluent community edition and copy jars:

wget http://packages.confluent.io/archive/7.1/confluent-community-7.1.1.zip
unzip -d /opt/confluent confluent-community-7.1.1.zip
cd /opt/confluent
mv confluent-7.1.1 7.1.1
cp /opt/confluent/7.1.1/share/java/confluent-common/common-utils-7.1.1.jar /opt/kafka/libs/
cp /opt/confluent/7.1.1/share/java/confluent-common/common-config-7.1.1.jar /opt/kafka/libs/
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/kafka-avro-serializer-7.1.1.jar /opt/kafka/libs/
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/kafka-connect-avro-data-7.1.1.jar /opt/kafka/libs
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/kafka-connect-avro-converter-7.1.1.jar /opt/kafka/libs
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/kafka-schema-serializer-7.1.1.jar /opt/kafka/libs
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/kafka-schema-registry-client-7.1.1.jar /opt/kafka/libs
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/avro-1.11.0.jar /opt/kafka/libs
cp /opt/confluent/7.1.1/share/java/kafka-serde-tools/guava-30.1.1-jre.jar /opt/kafka/libs

[slominskir]

README updated, plus java-library-distribution plugin now in use by Gradle build, which automatically accumulates dependencies.

[slominskir]

Re-opening because this still isn't working as expected. First I was under the impression that dependencies could be included with the plugin, but this isn't reliable. It turns out that shared dependencies (dependencies used by Kafka itself, or other plugins) cause issues if found in the plugin directory as mentioned in the Debezium Dockerfile. In our case I've observed NoSuchClassExceptions for guava, even when the jar is inside the plugin subdirectory (and moving it to Kafka libs fixes the issue). We need to re-visit the README instructions to indicate the cleanest path is to do as mentioned initially and simply copy files from confluent community edition directly into Kafka libs dir. We may want to revisit the distribution plugin configuration to isolate the plugin jar and possibly any direct dependencies not found in Kafka libs.