Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove harcoded first value when getting all indicators #796

Conversation

jorgeacjitx
Copy link
Contributor

Proposed changes

  • The page size listing all indicators with the "getAll" param was hardcoded to 100. To increase performance, this restriction is proposed to be removed.

Related issues

Checklist

  • I consider the submitted work as finished
  • I tested the code for its functionality
  • I wrote test cases for the relevant uses case
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

@yassine-ouaamou
Copy link
Member

Hi @jorgeacjitx,
Thanks for your contribution. Can you fix the formatting of you code using "black"?

@jorgeacjitx
Copy link
Contributor Author

jorgeacjitx commented Dec 11, 2024

Hi @jorgeacjitx, Thanks for your contribution. Can you fix the formatting of you code using "black"?

Yes, @yassine-ouaamou the problem is that the pre-commit hook is reformatting my code again. I just skipped the pre-commit hook and now the formatting looks good. Please look at the formatting pre-commit hook because it's not working fine.

@@ -63,8 +63,6 @@ def list(self, **kwargs):
get_all = kwargs.get("getAll", False)
with_pagination = kwargs.get("withPagination", False)
with_files = kwargs.get("withFiles", False)
if get_all:
first = 100
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you take "first" value when present and keep 100 as default when "fisrt" is not present ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @aHenryJard,
The param "first" has a default value of 500:

first = kwargs.get("first", 500)

When the "get_all" param is true, it is still used for querying (L121):

if get_all:
final_data = []
data = self.opencti.process_multiple(result["data"]["indicators"])
final_data = final_data + data
while result["data"]["indicators"]["pageInfo"]["hasNextPage"]:
after = result["data"]["indicators"]["pageInfo"]["endCursor"]
self.opencti.app_logger.info("Listing Indicators", {"after": after})
result = self.opencti.query(
query,
{
"filters": filters,
"search": search,
"first": first,

Copy link
Member

@aHenryJard aHenryJard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally all good for me, thank you.

@aHenryJard aHenryJard merged commit 3bc66d2 into OpenCTI-Platform:master Jan 17, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

first parameter is not customizable when using the getAll option in list method
3 participants