Document driver installation for Hive metastore backend database

[Jimvin]

Certain databases (Oracle, MySQL) do not allow shipping their database driver, and so as a user of that database with Hive, I have to manually provide the driver to Hive. As a user, I want to find instructions in the Stackable documentation on how to install and configure such a driver with a Stackable-deployed Hive instance, so I can use Hive with my proprietary database.

Suggested steps to document

• instruct user to deploy a PV (ReadOnlyMany mode)
• explain how to copy stuff into the PV
• mount the PV on the Pods with podOverrides
• set the aux jar path: https://chetnachaudhari.github.io/2016-02-16/how-to-add-auxiliary-jars-in-hive/ with configOverrides

Also: we should update the "required external components" page to note that some drivers have to be installed manually.

Considered alternatives

• building a custom image (quite a bit of overhead, also requires a registry/way to deploy the image)
• copy the driver from s3 or another place to curl from

Tasks

Tasks

Preview Give feedback

1. Write the tutorial/guide (and test it while writing)
2. Mention the guide in the "Required external components" page

Open Questions

• Should we have an integration test for this? Should it be somehow entirely coded so we can test if it keeps working?

Acceptance criteria

• The "Usage guide" for the Hive operator contains a document that explains how to install custom drivers
• The guide is mentioned in relevant places to make it easy to find for users

[lfrancke]

The follow up for this would be #415

[lfrancke]

Refinement by Malte, Felix

[fhennig]

Refinement notes

Due to licensing issues certain drivers cannot be shipped, and customers have to deploy them on their own.

Suggested steps

• instruct user to deploy a PV (ReadOnlyMany mode)
• explain how to copy stuff into the PV
• mount the PV on the Pods with podOverrides
• set the aux jar path: https://chetnachaudhari.github.io/2016-02-16/how-to-add-auxiliary-jars-in-hive/ with configOverrides

Considered alternatives

• building a custom image (quite a bit of overhead, also requires a registry/way to deploy the image)
• copy the driver from s3 or another place to curl from

Misc.

we should also update the "required external components" page to note that some drivers have to be installed manually.

A common way to add volumes to product images would be good. We have something in Druid in Airflow, but no standardized way: stackabletech/issues#471

Maybe we can also write a generic tutorial about how to get arbitrary files into Pods

[lfrancke]

We can't ship certain drivers for licensing reasons. So we'll need a documented way to include new ones either way.

[soenkeliebau]

looks good to me, just to double check, are the "considered alternatives" planned to include as an "honorable mention" in the docs?

[fhennig]

I think they weren't, but good idea!

[adwk67]

Another alternative might be to use git-sync in "one-off" mode to download external resources. This would involve more work/code changes, but would be more robust than using PVCs (we got rid of our PVC-based external-resources-tests because they only worked on certain clusters) and is something we were looking at using for e.g. spark anyway.

[lfrancke]

Could you please add a link to the documentation? (can be nightly)

[adwk67]

Docs are here: https://docs.stackable.tech/home/nightly/hive/usage-guide/database-driver