Add support for range-based ID searching #819
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds functions to HEPDataQueryParser to support range-based queries: Determine if a query is a valid query, and to parse a valid query.
Adds a range-based publication_recid search functionality to the OpenSearch api search. Also modifies supporting merge_results function to support single-dict.
Adds an accidentally removed if statement that causes empty queries to not load data table results.
Add search term shorthand publication_recid truncated to recid in OpenSearch
Adds support for potential different range terms, defined in hepdata/config.py. Updates OpenSearch API search to use this.
Adds default descending order to range query search in opensearch api
Update tests to use correct range search syntax
Add query parser test case for publication_recid in the test _query_parser test.
Alters inspire_id mapping from text to integer to allow range-based searching. Also updates some testing and required configuration addition for range searching. (keeping it all in one commit)
Updates help text in search_help to display inspire ID range search guidance
Adds more tolerance to extra whitespace after the colon in range queries, and between the "TO" and range numbers. Also updates parsing query to remove whitespace to accommodate this tolerance.
Adds more cases to test_query_parser_is_range_query which include valid and invalid extra whitespace cases.
Adds a specific test for range-searching on recid/inspire_id entries in the OpenSearch API
Updates test_search_range_ids to include a number of expected failure cases.
Modifies the range query parser functionality to fix a bug which disallowed range queries simultaneously with other queries. Improves errors. Renamed is_range_query to get_range_queries as it now returns range query data only.
Adds improved testing for range query functionality. Adds a specific test for range queries alongside alternate search types.
Updates commenting in range query code
Adds commenting and improves clarity of get_range_query
Improves the test for test_query_parser_get_range_queries, adds some new cases and improves code clarity in the test.
|
I have made improvements and bugfixes to this branch to solve cases where using range searches disabled searching by other means, and to improve test robustness in the search functionality |
Proper implementation has been figured out. Removes range parsing methods as raw range strings can be passed through to the search object. This in turn allows proper use of logic in query strings with range queries.
Updates search testing to test current range search verification and updates range search testing cases. Removes tests for removed range parsing functions.
Add inspire_id and publication_recid as possible sorting fields by adding them to the possible_sorting_fields in display_results_options.html
Remove display of recid and inspire id search options from search "sort by" widget
Fixes a bug where range queries with publication_recid were not sorted by default. Now sets CFG_SEARCH_RANGE_TERMS in config to use the correct OpenSearch field keyword (recid). Also updates the sort_fields_mapping in the OpenSearch api to use correct keyword.
Add new recid and inspire_id keywords to search function docstring for the sort_field in the opensearch api.
Update verify_range_query_term docstring to clarify that query used is a parsed query.
Updates test_verify_range_query_term to account for using a parsed range query (with recid and not publication_recid). Updates
Renames test_composite_range_queries to test_range_queries and adds some extra testing for single-range keyword searching. Checks against correct expected sort order.
Adds the keyword: "inspire_id" as a range searchable keyword term in the config. This is used to verify the query keyword in a range search.
Adds "publication_recid" to the sort_fields_mapping functon to allow use as a sort field.
Fixes intended keyword searching of publication_recid for publication only searches and recid for pub/table searches.
Updates range search query testing to match updated changes and add some new changes to the parsing
Update range test cases to include fixes for default range search ordering and correct use of recid vs publication_recid
Updates the range query parser test to match current functionality/changes
Adds more test cases to the test_range_queries test function, to include more boundary testing and differentiation between recid and publication_recid.
GraemeWatt
approved these changes
Dec 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds support for range-based searching in the OpenSearch index using HEPData ID and Inspire IDs. Updates the search help page to display new search syntax.
NOTE: inspire_id is changed from "text", to "integer". A re-index is required.
Closes #791.