-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor OTP to have new OTP classes to replace the OBA GTFS classes #2495
Refactor OTP to have new OTP classes to replace the OBA GTFS classes #2495
Conversation
…es to replace the OBA GTFS classes opentripplanner#2494)
…ebusaway2'. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…g package is done in 2 commits to preserve git history. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
… best place to map between the to models. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
… serialization. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…resentation). This is probably a god idea, but it was not used. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…nd after refactoring. Remove unused code, collapse DAO inheritance structure to get rid of unnecessary complexity. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…replace the OBA GTFS classes opentripplanner#2494)
…place the OBA GTFS classes opentripplanner#2494)
Fixed member formatting, removing prefix: '_', and changing constant names to upper case. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
4f944d7
to
3c4f5dd
Compare
*/ | ||
public class MapCollection { | ||
static <S,T> Collection<T> mapCollection(Collection<S> entites, Function<S,T> mapper) { | ||
return entites == null ? null : entites.stream().map(mapper).collect(Collectors.toList()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: "s/entites/entities/"
- Spelling errors fixed. - All new code formatted with the project profile. - Minor code style improvements, like Java 8 style generics. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
CalendarServiceDataFactoryImpl into one. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
onebusaway2 into package 'org.opentripplanner.calendar.impl'. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
ef6d328
to
52c98c5
Compare
…ceInterval.java (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
(Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…P transit model. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…fy it. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…lanner.util'. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…ner.model'. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…ner.model'. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
… to 'org.opentripplanner.model.impl'. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
(Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
(Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
…P Tranist model - Note these objects do not have ids in the GTFS model. (Refactor OTP to have new OTP classes to replace the OBA GTFS classes opentripplanner#2494)
0be366b
to
f6d27f1
Compare
A final comment on memory consumption (I don't believe the memory usage situation is a serious concern now, just something we need to understand and remember). In the mailing list message https://groups.google.com/d/msg/opentripplanner-dev/5ij1PJAVvs4/PommfsgOCQAJ you observed no difference between the pre- and post-split OTP. The code included in the message has the line that forces garbage collection at every measurement commented out. Was it commented out when the measurement CSV files were produced? |
… copied from ..." to "This file is based on code copied from project X, see the LICENSE file for further information."
@abyrd I am not sure, probably tested with and without, but the submitted log files are mote likely without. Note! The |
- Rename GTFSPatternHopFactory to PatternHopFactory - Rename method mapDao(...) to mapGtfsDaoToOTPTransitService(...) - Rename OtpTransitDaoMapper to GTFSToOtpTransitServiceMapper - Rename OtpTransitBuilder to OtpTransitServiceBuilder - Rename GTFSPatternHopFactory to PatternHopFactory - Add JavaDoc to all mapper classes and OtpTransitServiceBuilder - Cleanup JavaDoc in OtpTransitServiceImpl - Delete InterliningTrip (not used in PatternHopFactory)
Thanks for the comments and updates @t2gran. What do you think about renaming |
I will do that. What should i do with the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have covered all the details, thanks for all the effort on this change! @fpurcell we need a second positive review before we can merge this - you have actually tested this code on Portland data and observed no ill effects. Can you give it one last review from that same point of view - either test it out or let us know if you can approve the changes based on your past testing?
As a matter of procedure @t2gran can you add a line to the changelog as part of this PR? |
Hey Andrew & Thomas ... I'll get it tested before the weekend, and add an approval if it passes my test suite. BTW, there's quite a few files being changed, and I'm sorry I don't have the bandwidth to do anything more than build-graph, run-tests, report-back-on-results, approve-on-test-success. Thus, my approval is simply based on running a regression check. I'd like to note being slightly uncomfortable giving approval w/only a minimal scan of the actual code changes (but not uncomfortable enough to not submit a review :-)); and assume that other reviewers are comfortable with the actual code changes. |
FYI: firstly, I am able to successfully build otp.jar (using jdk 1.8.0_181). But that said, I am seeing the following when I run 'mvn package' :
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR passed the TriMet test suite. The graph, data and config files are all located here: http://maps7.trimet.org/p/
On 23 Aug 2018, at 20:27, Frank Purcell ***@***.***> wrote:
BTW, there's quite a few files being changed, and I'm sorry I don't have the bandwidth to do anything more than build-graph, run-tests, report-back-on-results, approve-on-test-success. Thus, my approval is simply based on running a regression check. I'd like to note being slightly uncomfortable giving approval w/only a minimal scan of the actual code changes (but not uncomfortable enough to not submit a review :-)); and assume that other reviewers are comfortable with the actual code changes.
Hi Frank, thanks a lot for your feedback. Thomas, Gard and I have already spent quite a lot of time going over the code itself, and I believe I’ve actually read every changed file, so never fear, I requested a review from you specifically to get a different point of view. Many months back, you were the first person to test out these changes with a different data source and give the thumbs up. So the checks you performed were exactly what we were looking for.
|
Also, thanks for pointing out that NumberFormat warning in the tests @fpurcell. I will see if I can fix that before I merge. |
@fpurcell checking out the test org.opentripplanner.updater.bike_rental.TestBikeRentalStationSource#testSmoove. This uses a JSON data source containing the following bike station definition:
And a comment in the test states |
After catching this PR up to master, we get a build error. This is due to the fact that the OBA FeedInfo objects now have String IDs instead of Integer IDs. It is easy to update the OTP code to expect Strings (I've already done this), but in the process I also realize that Agencies are also identified by a single String, rather than an AgencyAndId / FeedScopedId. This discrepancy is due to the same difference of perspective that gives us OBA's AgencyAndId versus OTP's new FeedScopedId. OTP considers entities to be unique within a feed (and all feeds must have a unique ID), OBA considers them to be unique within an Agency (and all Agencies must have a unique ID or be remapped). So unfortunately we'll need to somehow inject feed ID information into these IDs. But this leads me to realize that we haven't verified how this code behaves when loading multiple feeds containing the same Agency ID with colliding stop IDs (as is/was the case in New York and other places where multiple feeds defined agency 1, all with stop IDs 1, 2, 3, 4...). Unfortunately some additional testing is necessary here, and to avoid a regression we may need some additional code to handle feed IDs, and some additional Javadoc explaining the process by which multiple feeds are loaded and merged. It is very hard to predict the effect of some existing code, e.g. the call to Now that we have our own internal model, I think we can skip these hacks and just provide the FeedId to the mapper classes, allowing them to blanket overwrite whatever OBA thinks the identifier scope string is. We should also add automated tests that load multiple feeds with and without declared IDs, to ensure that it behaves as expected and entities do not collide. I can do that. |
I've created an automated test which shows there are unexpected ID collisions (which may have existed even before these changes). The existing code is trying to make both agency IDs and feed IDs unique, and keys some internal OTP maps on Agency IDs as if they are expected to be unique. All this can be fixed with FeedScopedId now under our control. I'm going to see how the test behaves on the master branch code and propose a fix. |
FeedId id changed from int to String in #2637 when switching org.onebusaway:onebusaway-gtfs from 1.3.5-conveyal-SNAPSHOT-2 to 1.3.41. @abyrd Should I fix this - I also had to do this to get the code running? When I did this, I did my best to keep the exact same behavior - replicating both wanted and more suspicious behavior. I agree we should fix the agency id and inject feed id to it, but I think that is a separate issue - @abyrd I am not sure if you want to fix this issue before merging or as a separate issue later?
This should work exactly the same as before - but I have to admit that it is a long time since I did this, and I do not remember how and what tested. I do know, that I tried to add UnitTests on all new code (not the code i imported, but mapping and so on), and that I did test the feed id loading. |
@abyrd I did not see you last comment - let me know if I can do some of this. |
@t2gran I suspect that the "incorrect" ID collision behavior I was seeing already exists in mainline OTP. I hope you didn't go to too much effort to replicate that behavior in this PR :) What I'll do is create a separate issue for the ID collision problem, create a branch adding a new unit test in commits referencing that issue, and see how it behaves when branching off master OTP. If it fails there too, we'll just make the Integer -> String ID change on this PR and merge, and fix the collisions as a separate issue. |
…anged from int to String.
🎆 Success! Now we can move on to adding Netex... |
This PR is discussed in the opentripplanner-dev group.
Issue 2494 describe the changes i detail.
There is a lot of commits here, feel free to squash them, but be aware that som of them is necessary to preserve the git history.