Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix slow ratio requests #114

Merged
merged 2 commits into from
Feb 5, 2021
Merged

Fix slow ratio requests #114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
550ad4b
replace snapshotfilter with combinedfilter
rtroilo Feb 4, 2021
31e1746
Update CHANGELOG.md
rtroilo Feb 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,18 @@ Changelog

## 1.4.0-SNAPSHOT (current master)


### Bug Fixes

* update all tests using the filter parameter instead of deprecated types, keys, values ([#98])
* fix some invalid filters in the default swagger examples ([#111])

### Performance and Code Quality

* improve performance of ratio requests ([#114])

[#98]: https://github.com/GIScience/ohsome-api/issues/98
[#111]: https://github.com/GIScience/ohsome-api/issues/111
[#114]: https://github.com/GIScience/ohsome-api/pull/114


## 1.3.2
Expand Down
5 changes: 2 additions & 3 deletions src/main/lombok/org/heigit/ohsome/ohsomeapi/executor/ElementsRequestExecutor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -685,8 +685,7 @@ public static Response aggregateRatio(RequestResource requestResource,
new InputProcessor(servletRequest, isSnapshot, isDensity);
inputProcessorCombined.setProcessingData(processingDataCombined);
MapReducer<OSMEntitySnapshot> mapRed = inputProcessorCombined.processParameters();

mapRed = exeUtils.newSnapshotFilter(mapRed, filterExpr1, filterExpr2);
mapRed = mapRed.filter(combinedFilter);
var preResult = mapRed.aggregateByTimestamp().aggregateBy(snapshot -> {
OSMEntity entity = snapshot.getEntity();
boolean matches1 = filterExpr1.applyOSMGeometry(entity, snapshot.getGeometry());
Expand Down Expand Up @@ -996,7 +995,7 @@ public static <P extends Geometry & Polygonal> Response aggregateRatioGroupByBou
Map<Integer, P> geoms =
arrGeoms.stream().collect(Collectors.toMap(arrGeoms::indexOf, geom -> (P) geom));
var mapRed2 = mapRed.aggregateByTimestamp().aggregateByGeometry(geoms);
mapRed2 = exeUtils.newSnapshotFilter(mapRed2, filterExpr1, filterExpr2);
mapRed2 = mapRed2.filter(combinedFilter);
var preResult =
mapRed2.aggregateBy((SerializableFunction<OSMEntitySnapshot, MatchType>) snapshot -> {
OSMEntity entity = snapshot.getEntity();
Expand Down
25 changes: 0 additions & 25 deletions src/main/lombok/org/heigit/ohsome/ohsomeapi/executor/ExecutionUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,31 +132,6 @@ public MapReducer<OSMEntitySnapshot> snapshotFilter(MapReducer<OSMEntitySnapshot
|| snapshotMatches(snapshot, osmTypes2, simpleFeatureTypes2, keysInt2, valuesInt2));
}

/** Applies a filter on the given MapReducer object using the given filter expressions. */
public MapReducer<OSMEntitySnapshot> newSnapshotFilter(MapReducer<OSMEntitySnapshot> mapRed,
FilterExpression filterExpr1, FilterExpression filterExpr2) {
return mapRed.filter(snapshot -> {
OSMEntity entity = snapshot.getEntity();
return filterExpr1.applyOSMGeometry(entity, snapshot::getGeometry)
|| filterExpr2.applyOSMGeometry(entity, snapshot::getGeometry);
});
}

/**
* Applies a filter on the given MapReducer object using the given filter expressions. Used in
* /ratio/groupBy/boundary requests.
*/
public MapAggregator<OSHDBCombinedIndex<OSHDBTimestamp, Integer>, OSMEntitySnapshot>
newSnapshotFilter(
MapAggregator<OSHDBCombinedIndex<OSHDBTimestamp, Integer>, OSMEntitySnapshot> mapRed,
FilterExpression filterExpr1, FilterExpression filterExpr2) {
return mapRed.filter(snapshot -> {
OSMEntity entity = snapshot.getEntity();
return filterExpr1.applyOSMGeometry(entity, snapshot::getGeometry)
|| filterExpr2.applyOSMGeometry(entity, snapshot::getGeometry);
});
}

/** Compares the type(s) and tag(s) of the given snapshot to the given types|tags. */
public boolean snapshotMatches(OSMEntitySnapshot snapshot, Set<OSMType> osmTypes,
Set<SimpleFeatureType> simpleFeatureTypes, Integer[] keysInt, Integer[] valuesInt) {
Expand Down