Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace channel with concurrency #154

Merged
merged 8 commits into from
Aug 9, 2023
Merged

Replace channel with concurrency #154

merged 8 commits into from
Aug 9, 2023

Conversation

dwertent
Copy link

@dwertent dwertent commented Aug 8, 2023

Overview

Replaces channel with concurrency.
Set the desired number with the WORKER_CONCURRENCY environment variable.

David Wertenteil added 2 commits August 8, 2023 09:13
Signed-off-by: David Wertenteil <dwertent@armosec.io>
Signed-off-by: David Wertenteil <dwertent@armosec.io>
@codiumai-pr-agent-free
Copy link

PR Analysis

  • 🎯 Main theme: Replacing channel with concurrency
  • 📌 Type of PR: Refactoring
  • 🧪 Relevant tests added: No
  • Focused PR: Yes, the PR is focused on replacing channel with concurrency for better performance and scalability.
  • 🔒 Security concerns: No, the PR does not introduce any apparent security issues.

PR Feedback

  • General suggestions: The PR is generally well-structured and the changes are clear. However, it lacks tests to verify the new concurrency implementation. It's crucial to ensure that the new implementation works as expected and doesn't introduce any race conditions or other concurrency-related issues.

  • 🤖 Code feedback:

    • relevant file: mainhandler/handlerequests.go
      suggestion: Consider handling the error returned by ants.NewPoolWithFunc. Ignoring this error might lead to unexpected behavior if the pool creation fails. [important]
      relevant line: pool, _ := ants.NewPoolWithFunc(utils.ConcurrencyWorkers, func(i interface{}) {

    • relevant file: utils/utils.go
      suggestion: Consider returning an error from ParseIntEnvVar function when the environment variable is not an integer. This can help in identifying configuration issues early. [medium]
      relevant line: return defaultValue, fmt.Errorf("failed to parse %s env var as int: %w", varName, err)

    • relevant file: mainhandler/handlerequests.go
      suggestion: Consider adding a limit or a control mechanism for the number of goroutines that can be created. This can prevent potential issues with system resources if too many goroutines are created. [medium]
      relevant line: mainHandler.eventWorkerPool = pool

How to use

To invoke the PR-Agent, add a comment using one of the following commands:
/review [-i]: Request a review of your Pull Request. For an incremental review, which only considers changes since the last review, include the '-i' option.
/describe: Modify the PR title and description based on the contents of the PR.
/improve: Suggest improvements to the code in the PR.
/ask <QUESTION>: Pose a question about the PR.

To edit any configuration parameter from 'configuration.toml', add --config_path=new_value
For example: /review --pr_reviewer.extra_instructions="focus on the file: ..."
To list the possible configuration parameters, use the /config command.

@dwertent dwertent added the release Create release label Aug 8, 2023
@github-actions
Copy link

github-actions bot commented Aug 8, 2023

Summary:

  • License scan: success
  • Credentials scan: success
  • Vulnerabilities scan: success
  • Unit test: success
  • Go linting: failure

Copy link
Collaborator

@vladklokun vladklokun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the comments I’ve left, LGTM.

mainhandler/handlerequests.go Outdated Show resolved Hide resolved
utils/globalvars.go Outdated Show resolved Hide resolved
utils/types.go Outdated Show resolved Hide resolved
David Wertenteil added 3 commits August 8, 2023 15:40
Signed-off-by: David Wertenteil <dwertent@armosec.io>
Signed-off-by: David Wertenteil <dwertent@armosec.io>
Signed-off-by: David Wertenteil <dwertent@armosec.io>
@github-actions
Copy link

github-actions bot commented Aug 8, 2023

Summary:

  • License scan: success
  • Credentials scan: success
  • Vulnerabilities scan: success
  • Unit test: success
  • Go linting: success

Copy link
Collaborator

@vladklokun vladklokun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the test, LGTM.

utils/globalvar_test.go Outdated Show resolved Hide resolved
Comment on lines 44 to 48
if tt.varValue != "" {
os.Setenv(tt.varName, tt.varValue)
} else {
os.Unsetenv(tt.varName)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: an environment variable leaks after running this test

Since there is no cleanup of variables after setting them up, an environment variable will leak into the runner system. Which is why it’s a good idea to use test doubles for environment variables, not actual the actual live ones. If we do want the live variables, just use the function provided for testing:

Suggested change
if tt.varValue != "" {
os.Setenv(tt.varName, tt.varValue)
} else {
os.Unsetenv(tt.varName)
}
if tt.varValue != "" {
t.Setenv(tt.varName, tt.varValue)
} else {
t.Setenv(tt.varName, "")
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I implemented it with defer

David Wertenteil added 2 commits August 8, 2023 18:48
Signed-off-by: David Wertenteil <dwertent@armosec.io>
Signed-off-by: David Wertenteil <dwertent@armosec.io>
Copy link
Collaborator

@vladklokun vladklokun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines +45 to +46
t.Setenv(tt.varName, tt.varValue)
defer t.Setenv(tt.varName, "")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick(non-blocking): there’s no need to use defer testing.T.Setenv rolls back your change at the end of test automatically

Signed-off-by: David Wertenteil <dwertent@armosec.io>
@github-actions
Copy link

github-actions bot commented Aug 9, 2023

Summary:

  • License scan: success
  • Credentials scan: success
  • Vulnerabilities scan: success
  • Unit test: success
  • Go linting: success

@dwertent dwertent merged commit c135902 into main Aug 9, 2023
@dwertent dwertent deleted the concurrency branch October 26, 2023 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Create release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants