Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable users to simultaneously search all SearchBasedFetchers #6504

Merged
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
703d000
Add CompositeSearchBasedFetcher to offer the option to query all libr…
DominikVoigt May 19, 2020
3045da1
Add Test for CompositeSearchBasedFetcher.
DominikVoigt May 19, 2020
c6838a5
Reformat stream to increase readability.
DominikVoigt May 20, 2020
81c4d81
Add CHANGELOG entry under Added.
DominikVoigt May 20, 2020
a6cf33a
Add Linebreak to CHANGELOG.
DominikVoigt May 20, 2020
abf4701
Add Linebreak to CompositeSearchBasedFetcherTest to fix checkstyle is…
DominikVoigt May 21, 2020
51ec020
Update CHANGELOG.md
DominikVoigt May 22, 2020
55ac9be
Add parameterized tests for CompositeSearchBasedFetcher.
DominikVoigt May 23, 2020
c3c0b65
Add better parameterized test naming.
DominikVoigt May 23, 2020
7fa24e4
Implement requested change.
DominikVoigt May 24, 2020
1947b1f
Merge branch 'feature/add-option-to-fetch-from-all-sources' of https:…
DominikVoigt May 24, 2020
836fc31
Merge branch 'master' into feature/add-option-to-fetch-from-all-sources
DominikVoigt May 25, 2020
1acd5aa
Remove GrobidCitationFetcher from test set as it was removed from set…
DominikVoigt May 25, 2020
da09f97
Add null check for CompositeSearchBasedFetcher.
DominikVoigt May 25, 2020
f07fa33
Modify Stream of Arguments according to recommendation.
DominikVoigt May 27, 2020
2c70b3f
Change size of inkrement from 3 to 273 due to too many test arguments…
DominikVoigt May 27, 2020
7d2b006
Merge branch 'master' into feature/add-option-to-fetch-from-all-sources
DominikVoigt May 27, 2020
5f446c1
Reformat steam operators.
DominikVoigt May 27, 2020
d9454c5
Merge branch 'master' into feature/add-option-to-fetch-from-all-sources
koppor May 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Note that this project **does not** adhere to [Semantic Versioning](http://semve
- We fixed the bug when strike the delete key in the text field. [#6421](https://github.com/JabRef/jabref/issues/6421)
- We added a BibTex key modifier for truncating strings. [#3915](https://github.com/JabRef/jabref/issues/3915)
- We added support for jumping to target entry when typing letter/digit after sorting a column in maintable [#6146](https://github.com/JabRef/jabref/issues/6146)
- We added a new fetcher to enable users to search all available E-Libraries simultaneously. [#369](https://github.com/koppor/jabref/issues/369)
DominikVoigt marked this conversation as resolved.
Show resolved Hide resolved

### Changed

Expand Down
2 changes: 2 additions & 0 deletions src/main/java/org/jabref/logic/importer/WebFetchers.java
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import org.jabref.logic.importer.fetcher.ArXiv;
import org.jabref.logic.importer.fetcher.AstrophysicsDataSystem;
import org.jabref.logic.importer.fetcher.CiteSeer;
import org.jabref.logic.importer.fetcher.CompositeSearchBasedFetcher;
import org.jabref.logic.importer.fetcher.CrossRef;
import org.jabref.logic.importer.fetcher.DBLPFetcher;
import org.jabref.logic.importer.fetcher.DOAJFetcher;
Expand Down Expand Up @@ -101,6 +102,7 @@ public static SortedSet<SearchBasedFetcher> getSearchBasedFetchers(ImportFormatP
set.add(new DOAJFetcher(importFormatPreferences));
set.add(new IEEE(importFormatPreferences));
set.add(new GrobidCitationFetcher(importFormatPreferences));
set.add(new CompositeSearchBasedFetcher(set));
return set;
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package org.jabref.logic.importer.fetcher;

import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import org.jabref.logic.help.HelpFile;
import org.jabref.logic.importer.FetcherException;
import org.jabref.logic.importer.SearchBasedFetcher;
import org.jabref.model.entry.BibEntry;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class CompositeSearchBasedFetcher implements SearchBasedFetcher {

private static final Logger LOGGER = LoggerFactory.getLogger(CompositeSearchBasedFetcher.class);

private final Set<SearchBasedFetcher> fetchers;

public CompositeSearchBasedFetcher(Set<SearchBasedFetcher> searchBasedFetchers) {
// Remove the Composite Fetcher instance from its own fetcher set to prevent a StackOverflow
this.fetchers = searchBasedFetchers.stream()
.filter(searchBasedFetcher -> searchBasedFetcher != this)
.collect(Collectors.toSet());
}

@Override
public List<BibEntry> performSearch(String query) {
return fetchers.stream().flatMap(searchBasedFetcher -> {
try {
return searchBasedFetcher.performSearch(query).stream();
} catch (FetcherException e) {
LOGGER.warn(String.format("%s API request failed", searchBasedFetcher.getName()), e);
return Stream.empty();
}
}).parallel().collect(Collectors.toList());
DominikVoigt marked this conversation as resolved.
Show resolved Hide resolved
}

@Override
public String getName() {
return "SearchAll";
}

@Override
public Optional<HelpFile> getHelpPage() {
return Optional.empty();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
package org.jabref.logic.importer.fetcher;

import java.util.HashSet;
import java.util.Set;

import org.jabref.logic.importer.FetcherException;
import org.jabref.logic.importer.ImportFormatPreferences;
import org.jabref.logic.importer.SearchBasedFetcher;
import org.jabref.logic.importer.SearchBasedParserFetcher;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import static org.mockito.Mockito.mock;

public class CompositeSearchBasedFetcherTest {

private CompositeSearchBasedFetcher finder;
private SearchBasedFetcher searchBasedFetcher;
private SearchBasedParserFetcher searchBasedParserFetcher;

@BeforeEach
public void setUp() {
ImportFormatPreferences importFormatPreferences = mock(ImportFormatPreferences.class);

Set<SearchBasedFetcher> setOfSearchBasedFetchers = new HashSet<>();
// Add one searchBasedFetcher and one searchBasedParserFetcher
searchBasedFetcher = new ArXiv(importFormatPreferences);
searchBasedParserFetcher = new CiteSeer();
setOfSearchBasedFetchers.add(searchBasedFetcher);
setOfSearchBasedFetchers.add(searchBasedParserFetcher);
finder = new CompositeSearchBasedFetcher(setOfSearchBasedFetchers);
}

@Test
public void performSearchOnEmptyQuery() {
Assertions.assertTrue(finder.performSearch("").isEmpty());
}

@Test
public void performSearchContainsResultsOfSearchBasedFetcher() throws FetcherException {
Assertions.assertTrue(finder.performSearch("quantum").containsAll(searchBasedFetcher.performSearch("quantum")));
}

@Test
public void performSearchContainsResultsOfSearchBasedParserFetcher() throws FetcherException {
Assertions.assertTrue(finder.performSearch("quantum").containsAll(searchBasedParserFetcher.performSearch("quantum")));
}
}