Improve support for EPIP NeTex profile #4863

bas-hbt · 2023-02-22T14:48:30Z

Summary

This PR improves compatibility with NeTEx-dataset specified in the EU-profile.

replace the direct usage of JourneyPattern and OperatingPeriod with their abstract VersionStructure-representations.
add support for UicOperatingPeriods
retrieve authority-reference directly from line
ignore StopPointInJourneyPattern without alighting and boarding
add fallback for GroupOfStopPlaces to retrieve name from member
add null-checks to prevent NullPointerExceptions

Issue

#3640

Unit tests

Added test for parsing UicOperationPeriod in DateTypeAssignmentMapper

…pplanner#3640)

opentripplanner#3640)

…8 symbols (opentripplanner#3640)

… common super class OperatingPeriod_VersionStructure (opentripplanner#3640)

…planner#3640)

…etex-profile(opentripplanner#3640) # Conflicts: # src/main/java/org/opentripplanner/datastore/file/ZipFileDataSource.java # src/main/resources/Message_de.properties

hannesj

Looks good, a couple of questions.

hannesj · 2023-02-22T21:12:41Z

src/main/java/org/opentripplanner/netex/mapping/StopTimesMapper.java

+      if (
+        stopPoint != null &&
+        (isFalse(stopPoint.isForAlighting()) && isFalse(stopPoint.isForBoarding()))
+      ) {
+        continue;


Is there a reason to skip these? They are already marked as for not for alighting/boarding in mapToStopTime

Our dataset contains many of these stops as they're usually used to model tariff barriers. There are no PassengerStopAssignments for these stopPoints, so "lookUpStopLocation" yields null which results in the trip not being mapped at all.
The attempt was to ignore these for now, but I guess we should find a better solution

src/main/java/org/opentripplanner/netex/loader/parser/ServiceCalendarFrameParser.java

src/main/java/org/opentripplanner/netex/mapping/GroupOfStationsMapper.java

src/main/java/org/opentripplanner/netex/mapping/calendar/DayTypeAssignmentMapper.java

leonardehrenfried · 2023-02-23T08:19:04Z

Can you run the autoformatter with mvn prettier:write? If you're on Windows you need to merge the latest dev-2.x for it to work.

codecov · 2023-02-23T14:01:13Z

Codecov Report

Patch coverage: 69.04% and project coverage change: +0.08 🎉

Comparison is base (bc69337) 62.93% compared to head (646ce0c) 63.02%.

Additional details and impacted files

@@              Coverage Diff              @@
##             dev-2.x    #4863      +/-   ##
=============================================
+ Coverage      62.93%   63.02%   +0.08%     
- Complexity     13263    13301      +38     
=============================================
  Files           1662     1662              
  Lines          66330    66376      +46     
  Branches        7215     7227      +12     
=============================================
+ Hits           41748    41831      +83     
+ Misses         22200    22168      -32     
+ Partials        2382     2377       -5

Impacted Files	Coverage Δ
.../opentripplanner/netex/index/NetexEntityIndex.java	`99.01% <ø> (ø)`
...ipplanner/netex/mapping/GroupOfStationsMapper.java	`15.38% <0.00%> (-2.80%)`	⬇️
...entripplanner/netex/mapping/ServiceLinkMapper.java	`51.66% <ø> (ø)`
...tripplanner/netex/mapping/TripCalendarBuilder.java	`89.58% <ø> (ø)`
...entripplanner/netex/mapping/TripPatternMapper.java	`70.80% <ø> (ø)`
...g/opentripplanner/netex/mapping/StationMapper.java	`61.81% <16.66%> (+8.87%)`	⬆️
...opentripplanner/netex/mapping/StopTimesMapper.java	`66.66% <33.33%> (+3.38%)`	⬆️
...lanner/netex/loader/parser/ServiceFrameParser.java	`73.97% <66.66%> (+2.05%)`	⬆️
.../org/opentripplanner/netex/mapping/TripMapper.java	`71.95% <66.66%> (+10.97%)`	⬆️
...etex/mapping/calendar/DayTypeAssignmentMapper.java	`86.95% <76.19%> (-4.12%)`	⬇️
... and 21 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

vpaturet

Looks good to me, I have only a few minor comments. Will you add NeTEx test data as part of this PR or in a separate one?

vpaturet · 2023-02-24T14:23:03Z

src/main/java/org/opentripplanner/netex/issues/InvalidTimeZone.java

+
+import org.opentripplanner.graph_builder.issue.api.DataImportIssue;
+
+public class InvalidTimeZone implements DataImportIssue {


It is now recommended to use the generic Issue builder instead of creating a custom class. See

OpenTripPlanner/src/main/java/org/opentripplanner/netex/mapping/FlexStopsMapper.java

Line 67 in 1c18194

Issue.issue(

vpaturet · 2023-02-24T14:26:15Z

src/main/java/org/opentripplanner/netex/loader/parser/ServiceFrameParser.java

@@ -244,8 +244,8 @@ private void parseJourneyPatterns(JourneyPatternsInFrame_RelStructure journeyPat
    if (journeyPatterns == null) return;

    for (JAXBElement<?> pattern : journeyPatterns.getJourneyPattern_OrJourneyPatternView()) {
-      if (pattern.getValue() instanceof JourneyPattern) {
-        this.journeyPatterns.add((JourneyPattern) pattern.getValue());
+      if (pattern.getValue() instanceof JourneyPattern_VersionStructure) {


Use pattern matching for instanceof (was missing in the original code but this is an opportunity to improve it)

vpaturet · 2023-02-27T07:33:42Z

src/main/java/org/opentripplanner/netex/mapping/calendar/DayTypeAssignmentMapper.java

    boolean isAvailable = isDayTypeAvailableForAssigment(dayTypeAssignment);

    String ref = dayTypeAssignment.getOperatingPeriodRef().getRef();
-    OperatingPeriod period = operatingPeriods.lookup(ref);
+    OperatingPeriod_VersionStructure period = operatingPeriods.lookup(ref);

    if (period != null) {


You can remove the null check (instanceof is false with null)

loj-hbt and others added 17 commits February 2, 2023 18:20

support european netex profile (opentripplanner#3640)

cfcae63

add support for UicOperatingPeriod and ServiceJourneyPattern (opentri…

2802545

…pplanner#3640)

ignore stopPoints without alighting or boarding when mapping StopTimes (

60319b4

opentripplanner#3640)

fix test (opentripplanner#3640)

0aad6ec

add cibuild for branch (opentripplanner#3640)

0c14c95

fix codestyle (opentripplanner#3640)

8aef2ee

fix codestyle (opentripplanner#3640)

d2b5ac5

fix codestyle (opentripplanner#3640)

fd22bf6

add resources-plugin to pom to fix failing CI job(opentripplanner#3640)

d2f10b6

fix typos in Message_de.properties an replace unicode chars with UTF-…

641d6d7

…8 symbols (opentripplanner#3640)

replace references to OperatingPeriod and UicOperatingPeriod with the…

8c55682

… common super class OperatingPeriod_VersionStructure (opentripplanner#3640)

fix handling of OperatingPeriods in DayTypeAssignmentMapper (opentrip…

cc08772

…planner#3640)

format DayTypeAssignmentMapper.java (opentripplanner#3640)

afe7198

fix typo DayTypeAssignmentMapper.java (opentripplanner#3640)

1ad2dcf

fix formatting in DayTypeAssignmentMapper.java (opentripplanner#3640)

6e9f110

Merge remote-tracking branch 'origin/dev-2.x' into support-european-n…

ffdf55d

…etex-profile(opentripplanner#3640) # Conflicts: # src/main/java/org/opentripplanner/datastore/file/ZipFileDataSource.java # src/main/resources/Message_de.properties

add test and cleanup code(opentripplanner#3640)

7089001

bas-hbt requested a review from a team as a code owner February 22, 2023 14:48

leonardehrenfried changed the title ~~Improve support for european netex profile (opentripplanner#3640)~~ Improve support for european netex profile Feb 22, 2023

leonardehrenfried changed the title ~~Improve support for european netex profile~~ Improve support for european NeTex profile Feb 22, 2023

hannesj reviewed Feb 22, 2023

View reviewed changes

Implement requested changes from review (opentripplanner#3640)

ab3f75d

hannesj previously approved these changes Feb 23, 2023

View reviewed changes

hannesj requested a review from vpaturet February 23, 2023 13:54

leonardehrenfried changed the title ~~Improve support for european NeTex profile~~ Improve support for EPIP NeTex profile Feb 23, 2023

dekarl mentioned this pull request Feb 23, 2023

Fix NPEs in NeTEx EPIP profile #4872

Merged

leonardehrenfried added the improvement label Feb 23, 2023

vpaturet requested changes Feb 27, 2023

View reviewed changes

loj-hbt dismissed hannesj’s stale review via 4270056 February 28, 2023 10:17

WIP implementing review comments (opentripplanner#3640)

2751fff

loj-hbt force-pushed the support-european-netex-profile(opentripplanner#3640) branch from 4270056 to 2751fff Compare February 28, 2023 10:18

Implement integration test for NeTEx epip data (opentripplanner#3640)

646ce0c

leonardehrenfried requested review from vpaturet and hannesj February 28, 2023 12:40

t2gran added this to the 2.3 milestone Feb 28, 2023

vpaturet approved these changes Feb 28, 2023

View reviewed changes

hannesj approved these changes Feb 28, 2023

View reviewed changes

hannesj merged commit 67f94aa into opentripplanner:dev-2.x Feb 28, 2023

t2gran pushed a commit that referenced this pull request Feb 28, 2023

Add changelog entry for #4863 [ci skip]

e0957be

t2gran mentioned this pull request Mar 1, 2023

Revert "Improve support for EPIP NeTex profile" #4921

Merged

sjthiessen added a commit to HBTGmbH/OpenTripPlanner that referenced this pull request Mar 2, 2023

Fix compatibility issues with nordic profile from opentripplanner#4863

cb2598f

sjthiessen mentioned this pull request Mar 2, 2023

Fix Netex EPIP implementation bug #4925

Merged

loj-hbt pushed a commit to HBTGmbH/OpenTripPlanner that referenced this pull request Mar 15, 2023

Fix compatibility issues with nordic profile from opentripplanner#4863

2506b8f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve support for EPIP NeTex profile #4863

Improve support for EPIP NeTex profile #4863

bas-hbt commented Feb 22, 2023 •

edited

Loading

hannesj left a comment

hannesj Feb 22, 2023

sjthiessen Feb 23, 2023 •

edited

Loading

leonardehrenfried commented Feb 23, 2023

codecov bot commented Feb 23, 2023 •

edited

Loading

vpaturet left a comment

vpaturet Feb 24, 2023

vpaturet Feb 24, 2023

vpaturet Feb 27, 2023


		import org.opentripplanner.graph_builder.issue.api.DataImportIssue;

		public class InvalidTimeZone implements DataImportIssue {

Improve support for EPIP NeTex profile #4863

Improve support for EPIP NeTex profile #4863

Conversation

bas-hbt commented Feb 22, 2023 • edited Loading

Summary

Issue

Unit tests

hannesj left a comment

Choose a reason for hiding this comment

hannesj Feb 22, 2023

Choose a reason for hiding this comment

sjthiessen Feb 23, 2023 • edited Loading

Choose a reason for hiding this comment

leonardehrenfried commented Feb 23, 2023

codecov bot commented Feb 23, 2023 • edited Loading

Codecov Report

vpaturet left a comment

Choose a reason for hiding this comment

vpaturet Feb 24, 2023

Choose a reason for hiding this comment

vpaturet Feb 24, 2023

Choose a reason for hiding this comment

vpaturet Feb 27, 2023

Choose a reason for hiding this comment

bas-hbt commented Feb 22, 2023 •

edited

Loading

sjthiessen Feb 23, 2023 •

edited

Loading

codecov bot commented Feb 23, 2023 •

edited

Loading