Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

collection limit rollback change #929

Merged
merged 14 commits into from
Mar 11, 2024
Merged

Conversation

Yuqi-Du
Copy link
Contributor

@Yuqi-Du Yuqi-Du commented Mar 6, 2024

This is a rollback change for Astra collection limit relaxation, and part of the short term solution we have for db collection limit.
In general, before violates Data API collection guardrail, if tenant hit the astra side guardrail and has indexes creation failure, Data api will also drop the table(plus indexes)

if index creation violates DB index limit and collection not existed before, and rollback is enabled, then drop the collection

Which issue(s) this PR fixes:
Fixes #912
Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

DEFAULT_INDEXES_AVAILABLE_PER_DATABASE 50 to 100,
fail on DB 50 guardrail
@Yuqi-Du Yuqi-Du requested a review from a team as a code owner March 6, 2024 00:24
Copy link
Contributor

@tatu-at-datastax tatu-at-datastax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable, suggested some changes.

@jeffreyscarpenter
Copy link
Contributor

Do we need to add a new test case or two to make sure this is working as expected @Yuqi-Du ?

@Yuqi-Du
Copy link
Contributor Author

Yuqi-Du commented Mar 6, 2024

Do we need to add a new test case or two to make sure this is working as expected @Yuqi-Du ?

I have tested this by writing our createCollection and deleteCollection code against Astra by using driver.
But it is hard to add test cases to our IT, since we can not trigger the DB guardrail (there is no such guardrail in IT).

@Yuqi-Du Yuqi-Du changed the title collection limit change, Short term approach collection limit rollback change Mar 6, 2024
…before,

 and rollback is enabled, then drop the collection
@Yuqi-Du
Copy link
Contributor Author

Yuqi-Du commented Mar 8, 2024

I have added a 'feature flag' to disable the rollback.
Currently, it is if index creation violates DB index limit and collection not existed before and rollback flag is enabled, then drop the collection

Copy link
Collaborator

@vkarpov15 vkarpov15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link
Contributor

@maheshrajamani maheshrajamani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flags sent to the create collection doesn't look correct.
If possible can you add unit tests?

@Yuqi-Du Yuqi-Du force-pushed the collection_limit_short_term_approach branch from 399fd85 to d600cb4 Compare March 11, 2024 17:35
Copy link
Contributor

@maheshrajamani maheshrajamani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Yuqi-Du Yuqi-Du merged commit 7159f12 into main Mar 11, 2024
2 checks passed
@Yuqi-Du Yuqi-Du deleted the collection_limit_short_term_approach branch March 11, 2024 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rollback on CreateCollection failure
5 participants