Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove facetconfig locks #410

Closed
wants to merge 3 commits into from
Closed

remove facetconfig locks #410

wants to merge 3 commits into from

Conversation

eladmarg
Copy link
Contributor

@eladmarg eladmarg commented Feb 4, 2021

No description provided.

@eladmarg
Copy link
Contributor Author

eladmarg commented Feb 4, 2021

failed because of environment problems

Error: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

Copy link
Contributor

@NightOwl888 NightOwl888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

However as is, many of the methods are no longer atomic and may cause concurrency issues. Please change the declaration of fieldTypes from:

private readonly IDictionary<string, DimConfig> fieldTypes = new ConcurrentDictionary<string, DimConfig>();

to:

private readonly ConcurrentDictionary<string, DimConfig> fieldTypes = new ConcurrentDictionary<string, DimConfig>();

to expose the methods that make ConcurrentDictionary atomic and see the other comments about the changes to each of the methods.

src/Lucene.Net.Facet/FacetsConfig.cs Outdated Show resolved Hide resolved
src/Lucene.Net.Facet/FacetsConfig.cs Outdated Show resolved Hide resolved
src/Lucene.Net.Facet/FacetsConfig.cs Outdated Show resolved Hide resolved
src/Lucene.Net.Facet/FacetsConfig.cs Outdated Show resolved Hide resolved
@NightOwl888 NightOwl888 linked an issue Feb 4, 2021 that may be closed by this pull request
@NightOwl888
Copy link
Contributor

I got it wrong. Upon closer inspection of the AddOrUpdate documentation:

If you call AddOrUpdate simultaneously on different threads, addValueFactory may be called multiple times, but its key/value pair might not be added to the dictionary for every call.

For modifications and write operations to the dictionary, ConcurrentDictionary<TKey,TValue> uses fine-grained locking to ensure thread safety. (Read operations on the dictionary are performed in a lock-free manner.) However, the addValueFactory and updateValueFactory delegates are called outside the locks to avoid the problems that can arise from executing unknown code under a lock. Therefore, AddOrUpdate is not atomic with regards to all other operations on the ConcurrentDictionary<TKey,TValue> class.

So in other words, this PR is not the equivalent functionality of the locks that exist now, as this method does not make the entire block atomic like the lock does.

I am curious, what issues are you actually seeing with these locks?

@eladmarg
Copy link
Contributor Author

eladmarg commented Feb 5, 2021

I am curious, what issues are you actually seeing with these locks?

nothing special, just performance penalty on GetTopChildrens method

by the way, I would keep this in this implementation, its just changing the property configuration, hierarchical / multi dimension etc..

@eladmarg
Copy link
Contributor Author

any updates with this?

@NightOwl888
Copy link
Contributor

@eladmarg

Given the fact that these locks exist in Lucene and this is not a reasonable locking alternative, we should keep them in place here. I would suggest holding off on this until we investigate #417 further, as it is still up in the air whether we should just use Lazy<T> or build our own wrapper around LazyInitializer. Once we have that solution (or solutions) worked out, we can apply it to FacetConfig as well.

Your last commit, 0f18945 does contain some useful removal of casts except for 1 call to Math.Abs. On this line, this is a bit twiddle that is equivalent to Java's >>> operator (unsigned right shift). The cast was done to make it unsigned, however, there is a faster approach that doesn't require a cast (at least not with int) is to use TripleShift.

I removed more than 1000 casts last night and have passing tests, but I would like to run some benchmarks to see how removing them compares with the casts in place before submitting the PR. Benchmarking the TripleShift(this int number, int bits) method against the two casts shows almost a 20% improvement in performance, and during the tests there seems to be less locking contention.

@eladmarg
Copy link
Contributor Author

cool, the expensive parts are the casts, if you have this on separate PR, this one can be closed.
waiting to see it in action

@eladmarg
Copy link
Contributor Author

closed in #423

@eladmarg eladmarg closed this Feb 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

remove unnecessary locks from FacetsConfig
2 participants