Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of .NET Core Analyze #296

Merged
merged 1 commit into from
Oct 3, 2018

Conversation

johnduhart
Copy link
Contributor

Calling file.Sync() for each log message has an extremely negative performance impact. Additionally, logging each Package/Project reference in a mid-sized solution will result in a 2GB log file.

There's likely some additional perf gains to be had in this analyze implementation, but this is the low-hanging fruit.

@CLAassistant
Copy link

CLAassistant commented Sep 27, 2018

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@elldritch elldritch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR LGTM. Thanks for the contribution! We'll merge once we get the tests passing (this looks like CI misconfiguration on our end, we'll take care of it).

@zlav can you take a look at the CI failures here? They're related to the coverage statistics we added.

It looks like we might want to do something similar to cloudflare/cfssl#203.

@elldritch
Copy link
Contributor

I'm a bit concerned that panics and log.Fatals might not flush the log if we get rid of per-entry sync'ing.

To improve the performance of this, we should probably have a defer func() { logFile.Sync() } in main. We'll need to migrate away from log.Fatal usage before that, since log.Fatal calls os.Exit which bypasses deferred functions.

@johnduhart
Copy link
Contributor Author

@liftM I put the sync back in, it provided no perf benefit when the log message in question was removed.

@elldritch elldritch force-pushed the fix/net-performance branch from 31d62f3 to e03ee77 Compare October 3, 2018 06:26
result in a 2GB log file, and will be extremely slow.
@elldritch elldritch force-pushed the fix/net-performance branch from e03ee77 to fc4d0c7 Compare October 3, 2018 07:15
@elldritch
Copy link
Contributor

Merging despite end-to-end test failure so I can cut a release.

@lackstein: this end-to-end test failure is because push-only keys still do not work correctly, so we're unable to do tests with API keys for public forks. What's the status on that?

@elldritch elldritch merged commit 03b24fb into fossas:master Oct 3, 2018
@coveralls
Copy link

coveralls commented Oct 3, 2018

Coverage Status

Coverage remained the same at 35.631% when pulling fc4d0c7 on mdsol:fix/net-performance into e9ac753 on fossas:master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants