Support Cassandra 4.1 as a persistence backend

[jeffreyscarpenter]

Now that Cassandra 4.1 is generally available, add a C* 4.1 persistence backend implementation for Stargate.

[marianne-manaog]

I am currently running C* performance tests using k8ssandra-operator that would need to run the latest C* v4.1, which has the following Paxos v2 bug fixes:

1. fix of the ContentionStrategy backoff and Clock.waitUntil, as per this Git commit;

2. fix for the WriteTimeoutException when using a single datacenter, as per this Git commit.

So, it would be great to have Stargate compatible with the latest k8ssandra-operator (v1.4.1).

[jeffreyscarpenter]

My assumption is that this would require creating a new persistence module persistence-cassandra-4.1, and that we'd continue to support the persistence-cassandra-4.0 module to match the schedule published by the Apache Cassandra project. This has implications for a variety of areas like adding the module to CI testing and publishing Docker images.

Since the K8ssandra project has not upgraded to Stargate v2, there's an argument for adding this new module to both the main (v2) branch and to v1 - as much as we'd like to encourage forward motion.

[jeffreyscarpenter]

Expanding on previous, here is a list of changes required:

• Add new persistence module
• Add ITs for new persistence module and incorporate in CI workflows
• Create Docker image for new persistence module and publish to Hub in release workflows
• Add Docker compose scripts for new persistence module
• Update documentation site to include new persistence module

[tatu-at-datastax]

Created #2609 before realizing there was this one already. But TL;DNR; at least 2 breaking changes between 4.0 and 4.1 APIs (note: https://fossies.org/diffs/apache-cassandra/4.0.7_vs_4.1.0/ has helpful diff):

• LocalAwareExecutorService renamed as LocalAwareExecutorPlus -- seems like possibly mechanical change, easy to do
• SchemaChangeListener was changed to interface (from (abstract?) class); but more importantly, completely changed to be typed, taking typed parameters instead of Strings. Need to change 2 implementations.

[ivansenic]

Would we have persistence 4 updated? or a completely new one?

[tatu-at-datastax]

I think it needs to be new one since APIs we use are not all compatible between 4.1 and 4.0 so it is not (I think) possible to have one backend that would work with both C*4.0.x and 4.1.x.

But it can probably be bootstrapped by copying 4.0 backend and making modifications from there.

[mamccorm]

Hi all. Any updates on this one? Looks like Cassandra 4.1 was first released back in December 2022. Does this affect both Stargate v1 (EOL) and also Stargate v2 (maintained)?

[jeffreyscarpenter]

Closed by #2711

[jeffreyscarpenter]

Hi all. Any updates on this one? Looks like Cassandra 4.1 was first released back in December 2022. Does this affect both Stargate v1 (EOL) and also Stargate v2 (maintained)?

Hi @mamccorm - we added this persistence module on the v2.1 branch a few months back but apparently did not link the PR to this issue.