Feature/enable lucene query parsing #6799

DominikVoigt · 2020-08-27T09:37:32Z

This PR enables the conversion of user queries, formulated in a simplified lucene syntax (see ADR 0014), into complex search queries for fetchers. Refs JabRef#456.

Change in CHANGELOG.md described (if applicable)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

827b986 add DOI preprint to american-society-for-microbiology.csl (JabRef#4946) 7cab2f7 Create depro-ufs.csl (JabRef#4947) 2fcda1f Create journal-of-sport-science-and-medicine.csl (JabRef#4949) cd457d4 Update american-marketing-association.csl (JabRef#4945) cacc4ee Create nejm-catalyst.csl (JabRef#4943) bd769b9 Update and rename dependent/chinese-medical-journal.csl to chinese-me… (JabRef#4941) 1f706cd Update health-services-research.csl (JabRef#4939) 76bcd1d Update journal-of-the-royal-society-of-western-australia.csl (JabRef#4932) e79640e Create afro-asia.csl (JabRef#4934) c601aa4 Update lancaster-university-harvard.csl (JabRef#4938) 184fd90 Update collection-du-centre-jean-berard.csl (JabRef#4936) 82f9aec fix et-al & add DOI thyroid.csl (JabRef#4937) 4af169f Create universidade-estadual-de-alagoas-uneal-abnt (JabRef#4885) 47165b3 Update and rename medical-physics.csl to dependent/medical-physics.csl (AMA) (JabRef#4905) 3cab27d Update thieme-german.csl (JabRef#4931) eb2c977 Create zeitschrift-fur-zahnarztliche-implantologie.csl (JabRef#4925) 4adb1ea Create karstenia.csl (JabRef#4929) 530a136 Fix author substitute in Universita Cattolica git-subtree-dir: src/main/resources/csl-styles git-subtree-split: 827b986

eb0d37e Create natbib-plainnat-author-dat.csl (JabRef#4967) fb1592a Update journal-of-fish-biology.csl (JabRef#4969) f6876cb Update .travis.yml (JabRef#4970) d6d400b Create london-review-of-international-law.csl (JabRef#4966) d2a5ae1 Update harvard-stellenbosch-university.csl (JabRef#4965) 9c62141 Create phytopathologia-mediterranea.csl (JabRef#4964) 8ca2ea1 Update historical-materialism.csl (JabRef#4960) 80456dc Update historical-materialism.csl be2d910 Update historical-materialism.csl 1d1cf09 Create atlande.csl (JabRef#4930) 89f41d4 Create juristische-zitierweise-oeffentliches-recht.csl (JabRef#4944) 8c677a0 Create korean-journal-of-gastroenterology.csl (JabRef#4954) eadb950 Create historical-materialism.csl (JabRef#4955) 5553dcd Create revista-materia.csl (JabRef#4957) d23a3ab Bug fix in APA 6 original publication macro (JabRef#4959) 66f9974 Always print publisher in APA 6th edition (JabRef#4899) 868809c Create agora.csl (JabRef#4940) 99c19c3 Update anthropologie-et-societes.csl (JabRef#4952) ee17423 Create critical-reviews-in-solid-state-and-materials-science.csl (JabRef#4951) 7a13a7d Create korean-journal-of-internal-medicine.csl (JabRef#4953) git-subtree-dir: src/main/resources/csl-styles git-subtree-split: eb0d37e

Fix some FetcherTests. Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

…cene-query-parsing

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

tobiasdiez

Thanks! This looks nice, and is a really helpful addition for the user. I've some remarks, but they are mostly minor.

Do you want to develop the user interface also in this PR, or do you plan to have a follow-up PR?

src/main/java/org/jabref/logic/importer/QueryConverter.java

src/main/java/org/jabref/logic/importer/fetcher/ComplexSearchQuery.java

src/test/java/org/jabref/logic/importer/QueryConverterTest.java

Rename QueryConverter to QueryParser Modify parser tests to compare complex queries Make complex query construction from terms independent from normal building Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

DominikVoigt · 2020-08-27T12:48:36Z

Thanks for the comments! :)
I added all of your suggestions.
The UI changes will be added as a follow-up PR.

tobiasdiez

Thanks! Looks good to me now.

There are the two things that slipped through the rename, if this is fixed and 5.1 released, then this can be merged.

src/test/java/org/jabref/logic/importer/QueryParserTest.java

src/main/java/org/jabref/logic/importer/QueryParser.java

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Siedlerchr

LGTM. I would say, we merge this after the release of 5.1

…cene-query-parsing

Siedlerchr · 2020-08-29T21:07:11Z

Maybe also Refs #840 I don't know if this is already possible. If yes, then we can close that super old issue.
Refs/Fixes probably #1975 as well.

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

…cene-query-parsing

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

…com/DominikVoigt/jabref into feature/enable-lucene-query-parsing

koppor

Some improvement suggestions at the ADR

docs/adr/0015-support-an-abstract-query-syntax-for-query-conversion.md

src/main/java/org/jabref/logic/importer/SearchBasedFetcher.java

src/main/java/org/jabref/logic/importer/SearchBasedParserFetcher.java

src/main/java/org/jabref/logic/importer/fetcher/ComplexSearchQuery.java

Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

…com/DominikVoigt/jabref into feature/enable-lucene-query-parsing

koppor · 2020-08-31T11:46:11Z

This refs JabRef#341.

* upstream/master: (120 commits) Follow up fix for copy paste (#6820) Add CSS Customisation (#6725) Separate signing and notarizing (#6822) Remove checkstyle hack. 8.36 got released (#6816) Feature/enable lucene query parsing (#6799) New release cycle Bump WyriHaximus/github-action-wait-for-status from v1.1.2 to v1.2 (#6814) Bump mockito-core from 3.5.5 to 3.5.7 (#6813) Bump classgraph from 4.8.87 to 4.8.89 (#6812) Bump me.champeau.gradle.jmh from 0.5.0 to 0.5.1 (#6811) Bump checkstyle from 8.35 to 8.36 (#6810) Improve Changelog Refactor edit action (#6808) Fixed typo in BuildInfo (#6807) disable checkstyle for generated fix checkstyle Simplify check-links.yaml (markdown-link-check) (#6720) Rename /gen to /generate (#6800) Disable CSL refresh on push (#6803) New Crowdin updates (#6804) ... # Conflicts: # src/main/java/org/jabref/gui/JabRefFrame.java

github actions and others added 11 commits August 1, 2020 02:13

Merge commit 'c8ebc91d4799e59518ac887f69e584040cf29994'

f5f02cf

Merge commit '37c2d64e449ef617bb98c0ad3b39a903d8f3d886' into master

b85ebe3

Merge remote-tracking branch 'upstream/master'

f10ea5f

Enable Query String into ComplexQuery parsing using lucene.

172b113

Fix some FetcherTests. Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Add ADR

90a629e

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Add Changelog entry

3af3781

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Merge remote-tracking branch 'upstream/master' into feature/enable-lu…

26851a7

…cene-query-parsing

Move Changelog entry

8468ad4

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Increment ADR number

d592a47

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

tobiasdiez requested changes Aug 27, 2020

View reviewed changes

Remove Optional from Lists

4610451

Rename QueryConverter to QueryParser Modify parser tests to compare complex queries Make complex query construction from terms independent from normal building Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

tobiasdiez requested changes Aug 27, 2020

View reviewed changes

src/test/java/org/jabref/logic/importer/QueryParserTest.java Outdated Show resolved Hide resolved

src/main/java/org/jabref/logic/importer/QueryParser.java Outdated Show resolved Hide resolved

tobiasdiez added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Aug 27, 2020

Rename slipped occurences of converter

86c6576

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

tobiasdiez approved these changes Aug 27, 2020

View reviewed changes

Siedlerchr approved these changes Aug 28, 2020

View reviewed changes

Siedlerchr added this to the v5.2 milestone Aug 28, 2020

DominikVoigt mentioned this pull request Aug 28, 2020

Feature/add ui for query parsing #6805

Merged

5 tasks

Merge remote-tracking branch 'upstream/master' into feature/enable-lu…

e2447f6

…cene-query-parsing

koppor removed the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Aug 28, 2020

DominikVoigt and others added 6 commits August 30, 2020 14:05

Mapping of unknown fields onto default field

0642d49

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Ignores unknown fields

11afbe9

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Merge branch 'master' into feature/enable-lucene-query-parsing

6fde881

Merge remote-tracking branch 'upstream/master' into feature/enable-lu…

e27ebcc

…cene-query-parsing

Fix checkstyle issues

239095e

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Merge branch 'master' into feature/enable-lucene-query-parsing

467cbfc

Merge branch 'feature/enable-lucene-query-parsing' of https://github.…

ff6a7d3

…com/DominikVoigt/jabref into feature/enable-lucene-query-parsing

koppor requested changes Aug 31, 2020

View reviewed changes

DominikVoigt and others added 4 commits August 31, 2020 13:11

Apply suggestions from code review

542d3a7

Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>

Handle year-range parsing in a more robust way

4b2ab98

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Move extracted method into builder

8b3b7cd

Signed-off-by: Dominik Voigt <dominik.ingo.voigt@gmail.com>

Merge branch 'feature/enable-lucene-query-parsing' of https://github.…

ec60734

…com/DominikVoigt/jabref into feature/enable-lucene-query-parsing

koppor merged commit dfefd4d into JabRef:master Aug 31, 2020

koppor mentioned this pull request Dec 28, 2020

Reuse query parser #1975

Closed

DominikVoigt deleted the feature/enable-lucene-query-parsing branch January 1, 2021 16:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/enable lucene query parsing #6799

Feature/enable lucene query parsing #6799

DominikVoigt commented Aug 27, 2020 •

edited

Loading

tobiasdiez left a comment

DominikVoigt commented Aug 27, 2020

tobiasdiez left a comment

Siedlerchr left a comment

Siedlerchr commented Aug 29, 2020 •

edited

Loading

koppor left a comment

koppor commented Aug 31, 2020

Feature/enable lucene query parsing #6799

Feature/enable lucene query parsing #6799

Conversation

DominikVoigt commented Aug 27, 2020 • edited Loading

tobiasdiez left a comment

Choose a reason for hiding this comment

DominikVoigt commented Aug 27, 2020

tobiasdiez left a comment

Choose a reason for hiding this comment

Siedlerchr left a comment

Choose a reason for hiding this comment

Siedlerchr commented Aug 29, 2020 • edited Loading

koppor left a comment

Choose a reason for hiding this comment

koppor commented Aug 31, 2020

DominikVoigt commented Aug 27, 2020 •

edited

Loading

Siedlerchr commented Aug 29, 2020 •

edited

Loading