Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instructions how to work with fetchers #6731

Merged
merged 4 commits into from
Aug 5, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions docs/advanced-reading/fetchers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Working on fetchers

Fetchers are the implementation of the [search using online services](https://docs.jabref.org/collect/import-using-online-bibliographic-database).
Some fetchers require API keys to get them working.
To get the fetchers running in a JabRef development setup, the keys need to be placed in the respective enviornment variable.
The following table lists the respective fetchers, where to get the key from and the environment variable where the key has to be placed.

| Service | Key Source | Environment Variable | Rate Limit |
| -- | -- | -- | -- |
| [IEEEXplore](https://docs.jabref.org/collect/import-using-online-bibliographic-database/ieeexplore) | [IEEE Xplore API portal](https://developer.ieee.org/) | `IEEEAPIKey` | 200 calls/day |
| [MathSciNet](http://www.ams.org/mathscinet) | (none) | (none) | Depending on the current network |
| [SAO/NASA Astrophysics Data System](https://docs.jabref.org/collect/import-using-online-bibliographic-database/ads) | [ADS UI](https://ui.adsabs.harvard.edu/user/settings/token) | `AstrophysicsDataSystemAPIKey` | 5000 calls/day |
| [Springer Nature](https://docs.jabref.org/collect/import-using-online-bibliographic-database/springer) | [Springer Nature API Portal](https://dev.springernature.com/) | `SpringerNatureAPIKey`| 5000 calls/day |
| [Zentralblatt Math](https://www.zbmath.org/) | (none) | (none) | Depending on the current network |

"Depending on the current network" means that it depends whether your request is routed through a network having paid access.
For instance, some universities have subscriptions to MathSciNet.

On Windows, you have to log-off and log-on to let IntelliJ know about the environment variable change.
Execute the gradle task "processResources" in the group "others" within IntelliJ to ensure the values have been correctly written.
Now, the fetcher tests should run without issues.

## Background on embedding the keys in JabRef

The keys are placed into the `build.properties` file.

```properties
springerNatureAPIKey=${springerNatureAPIKey}
```

In `build.gradle`, these variables are filled:

```groovy
"springerNatureAPIKey": System.getenv('SpringerNatureAPIKey')
```

The `BuildInfo` class reads from that file.

```java
new BuildInfo().springerNatureAPIKey
```

When executing `./gradlew run`, gradle executes `processResources` and populates `build/build.properties` accordingly.
However, when working directly in the IDE, Eclipse keeps reading `build.properties` from `src/main/resources`.
In IntelliJ, the task `JabRef Main` is executing `./gradlew processResources` before running JabRef from the IDE to ensure the `build.properties` is properly populated.
22 changes: 17 additions & 5 deletions src/main/java/org/jabref/logic/util/BuildInfo.java
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.util.Locale;
import java.util.Optional;
import java.util.Properties;

public final class BuildInfo {
Expand Down Expand Up @@ -48,11 +49,22 @@ public BuildInfo(String path) {
authors = properties.getProperty("authors", "");
year = properties.getProperty("year", "");
developers = properties.getProperty("developers", "");
azureInstrumentationKey = properties.getProperty("azureInstrumentationKey", "");
springerNatureAPIKey = properties.getProperty("springerNatureAPIKey", "");
astrophysicsDataSystemAPIKey = properties.getProperty("astrophysicsDataSystemAPIKey", "");
ieeeAPIKey = properties.getProperty("ieeeAPIKey", "");
azureInstrumentationKey = BuildInfo.getValue(properties, "azureInstrumentationKey", "");
springerNatureAPIKey = BuildInfo.getValue(properties, "springerNatureAPIKey", "118d90a519d0fc2a01ee9715400054d4");
astrophysicsDataSystemAPIKey = BuildInfo.getValue(properties, "AstrophysicsDataSystemAPIKey", "tAhPRKADc6cC26mZUnAoBt3MAjCvKbuCZsB4lI3c");
ieeeAPIKey = BuildInfo.getValue(properties, "ieeeAPIKey", "5jv3wyt4tt2bwcwv7jjk7pc3");
minRequiredJavaVersion = properties.getProperty("minRequiredJavaVersion", "1.8");
allowJava9 = "true".equals(properties.getProperty("allowJava9", ""));
allowJava9 = "true".equals(properties.getProperty("allowJava9", "true"));
}

private static String getValue(Properties properties, String key, String defaultValue) {
String result = Optional.ofNullable(properties.getProperty(key))
// workaround unprocessed build.properties file --> just remove the reference to some variable used in build.gradle
.map(value -> value.replaceAll("\\$\\{.*\\}", ""))
.orElse("");
if (!result.equals("")) {
return result;
}
return defaultValue;
}
}