Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to limit index creation #560

Closed
jeffreyscarpenter opened this issue Oct 9, 2023 · 1 comment
Closed

Add option to limit index creation #560

jeffreyscarpenter opened this issue Oct 9, 2023 · 1 comment
Assignees

Comments

@jeffreyscarpenter
Copy link
Contributor

jeffreyscarpenter commented Oct 9, 2023

When deploying in an managed environment such as Astra, operators need to be able to limit the number of indexes that JSON API creates.

From a practical perspective, the number of indexes is the bigger limiting factor than the number of collections, since the current implementation creates 8 or 9 SAI indexes per collection (Cassandra table), and could potentially create additional indexes once we add text search. Note: Astra currently enforces a lower limit on the number of indexes per keyspace (100) than the number of tables (200).

We need to be able to specify a limit on the number of indexes created. The current number of indexes should be checked prior to creating a table so that the limit is not exceeded while creating a collection. To account for the possibility of simultaneous collection creation, we may also need to implement recovery behavior such that failure to create an index for a new collection results in removal of the table and any indexes already created.

@jeffreyscarpenter
Copy link
Contributor Author

One relatively simple way to do the check might be to query the system_schema.indexes table (which should give a different value per tenant when running in multi-tenant mode)

SELECT count(*) FROM system_schema.indexes;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants