Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2vec saveload fixes #11

Merged
merged 40 commits into from
Sep 8, 2020

Conversation

piskvorky
Copy link

@piskvorky piskvorky commented Sep 7, 2020

Merged current develop, resolved conflicts + a few minor docstring edits in 49b35b7.

xh2 and others added 30 commits July 24, 2020 14:09
Make docs clearer on `alpha` parameter in LDA model
Make docs clearer on `alpha` parameter in LDA model
[MRG] Docs: Update Hoffman paper link for Online LDA
[MRG] Fix similarity bug in NMSLIB indexer + documentation fixes
piskvorky and others added 9 commits August 3, 2020 10:28
[MRG] Refresh docs for run_annoy tutorial
…e the positive_definite parameter, and extend normalization capabilities of the inner_product method (#2783)

* Deprecate SparseTermSimilarityMatrix's positive_definite parameter

* Reference paper on efficient implementation of soft cosine similarity

* Add example with Annoy indexer to SparseTermSimilarityMatrix

* Add example of obtaining word embeddings from SparseTermSimilarityMatrix

* Reduce space complexity of SparseTermSimilarityMatrix construction
Build matrix using arrays and bitfields rather than DOK sparse format

This work is based on the following blog post by @maciejkula:
https://maciejkula.github.io/2015/02/22/incremental-construction-of-sparse-matrices/

* Fix a typo in the soft cosine similarity Jupyter notebook

* Add human-readable string representation for TermSimilarityIndex

* Avoid sparse term similarity matrix computation when nonzero_limit <= 0

* Extend normalization in the inner_product method

Support the `maintain` vector normalization scheme.
Support separate vector normalization schemes for queries and documents.

* Remove a note in the docstring of SparseTermSimilarityMatrix

* Rerun continuous integration tests

* Use ==/!= to compare constant literals

* Add human-readable string representation for TermSimilarityIndex (cont.)

* Prod flake8 with a coding style violation in a docstring

* Collapse two lambdas into one internal function

* Revert "Prod flake8 with a coding style violation in a docstring"

This reverts commit 6557b84.

* Avoid str.format()

* Slice SparseTermSimilarityMatrix.inner_product tests by input types

* Remove similarity_type_code local variable

* Remove starting underscore from local function name

* Save indentation level and define populate_buffers function

* Extract SparseTermSimilarityMatrix constructor body to _create_source

* Extract NON_NEGATIVE_NORM_ASSERTION_MESSAGE to a module-level constant

* Extract cell assignment logic to cell_full local function

* Split variable swapping into three separate statements

* Extract normalization from the body of SparseTermSimilarityMatrix.inner_product

* Wrap overlong line

* Add test_inner_product_zerovector_zerovector and test_inner_product_zerovector_vector tests

* Further split test_inner_product into 63 test cases

* Raise ValueError when dictionary is empty
* bug fix: wikicorpus getstream from data file-path \n Replace fname with input

* refactor: use property decorator for input

Co-authored-by: jshah02 <jenisnehal.shah@factset.com>
Co-authored-by: Radim Řehůřek <me@radimrehurek.com>
@gojomo gojomo merged commit b5794ee into gojomo:2vec_saveload_fixes Sep 8, 2020
gojomo added a commit that referenced this pull request Sep 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants