Other imports and fixes #801
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
shankari
commented
Oct 16, 2021
shankari
commented
- Import additional instrumentation from the tripaware branch
- Fix live-reload of the diary screen
- Ensure that unprocessed trips also have (empty) values for the new confirmed trip fields around inferred labels
- Fix minor performance issues with the retrieval code (added during refactoring)
Remove import from the filters since we don't generate stats there Add import to the survey launch since we now do generate stats there Testing done: - verified that there are no errors on the label screen
- Return an "undefined" trip/trip wrapper if we are live reloading and don't
have the timeline maps populated
- we could also choose to check for the existence of the maps directly, but
this implementation also fixes the case in which the map exists but the
trip doesn't exist in the map
- Skip the `$scope` instantiation code if the trip is undefined
- Return to the main list in this case, since otherwise we have a blank detail screen
In shankari/e-mission-server@78a3ebb we changed the geojson export code to add in other confirmed trip fields - user inputs, inferred labels, etc Then, as part of e-mission@d9cc13b we copied these over, so that the inference code would Just Work. In this commit, we add empty top-level versions of these fields for unprocessed trips as well. Without this change, we sometimes get an error in `www/js/survey/multilabel/multi-label-ui.js` ``` 274 if (!angular.isDefined(userInputLabel)) { 275 userInputLabel = trip.user_input[mls.inputType2retKey(inputType)]; 276 } ``` because `user_input` is undefined Testing done: without the change: user_input is defined for processed trips ``` For checking user_input, trip starting at 2016-08-04T16:34:45.744782-10:00 is {"mode_confirm":"walk","purpose_confirm":"airporttransfer"} ``` but not for unprocessed trips ``` For checking user_input, trip starting at 2016-08-05T04:55:09-07:00 is undefined while populating trips, user_input not found distance: 3408.6004334046456 duration: 1022.720999956131 end_fmt_time: "2016-08-05T05:12:12-07:00" end_loc: {type: "Point", coordinates: Array(2)} end_local_dt: {timezone: undefined, year: 2016, month: 8, day: 5, weekday: 5, …} end_place: {$oid: "unprocessed_1470399132.638"} end_ts: 1470399132.638 feature_type: "trip" raw_trip: [] sensed_mode: "MotionTypes.UNPROCESSED" source: "unprocessed" start_fmt_time: "2016-08-05T04:55:09-07:00" start_loc: {type: "Point", coordinates: Array(2)} start_local_dt: {timezone: undefined, year: 2016, month: 8, day: 5, weekday: 5, …} start_place: {$oid: "unprocessed_1470398109.917"} start_ts: 1470398109.917 trip_id: {$oid: "unprocessed_1470398109.917_1470399132.638"} __proto__: Object ``` with the change, we don't get the "is undefined" log statement for any trips
The retrieval code was refactored for greater performance e-mission@f3ea875 However, this introduced some minor regressions. e-mission/e-mission-docs#679 (comment) which were caused due to incorrect parameter passing and resulting "undefined" values. e-mission/e-mission-docs#679 (comment) Testing done: while reading data for 4th August ---- we detect that the pipeline has finished running ``` INFO:Selected date is :1470294000000 getting pipeline complete timestamp DEBUG:complete_ts = 1470951950.271(Thu Aug 11 2016 14:45:50 GMT-0700) end of current day = 1470380399(Thu Aug 04 2016 23:59:59 GMT-0700) retVal = true Returning 1470951950.271,true ``` we only read entries after `1470951950.271 = 2016-08-11T21:45:50.271000+00:00` ``` getRawEntries: about to get pushGetJSON for the timestamp About to return message {"key_list":["manual/mode_confirm"],"start_time":1470951950.271,"end_time":1634403476,"key_time":"metadata.write_ts"} getRawEntries: about to get pushGetJSON for the timestamp About to return message {"key_list":["manual/purpose_confirm"],"start_time":1470951950.271,"end_time":1634403476,"key_time":"metadata.write_ts"} ``` we got a fair number of unprocessed inputs, mainly because we only filled in the inputs way after the pipeline ``` About to show 'Reading from server' DEBUG:About to dedup localResult = 0remoteResult = 52 DEBUG:Deduped list = 52 DEBUG:About to dedup localResult = 0remoteResult = 55 DEBUG:Deduped list = 55 unprocessed manual inputs 52 MODE, 55 PURPOSE ``` And we only processed trips from the 4th. ``` DEBUG:while reading data for 1470294000000 from server, got nTrips = 9 About to hide 'Reading from server' Finished hiding ionicLoading, returning list of size 9 Reading trips from server finished successfully with length 9 completeStatus = true trip count = 9, calling processTripsForDay About to show 'Processing trips' 0:2016-08-04T10:03:51.235000-10:00, 1880.7650001049042 1:2016-08-04T10:41:32.136385-10:00, 8056.584614753723 2:2016-08-04T13:10:38.739684-10:00, 605.260315656662 3:2016-08-04T13:40:36.959000-10:00, 135.75 4:2016-08-04T13:46:26.561801-10:00, 745.4381992816925 5:2016-08-04T14:06:52.592000-10:00, 280.5680000782013 6:2016-08-04T14:18:35.840464-10:00, 936.7305357456207 7:2016-08-04T14:39:38.288795-10:00, 5956.420205116272 8:2016-08-04T16:34:45.744782-10:00, 232.60321807861328 Processing trip starting at 2016-08-04T13:10:38.739684-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} Processing trip starting at 2016-08-04T13:40:36.959000-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} Processing trip starting at 2016-08-04T13:46:26.561801-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} Processing trip starting at 2016-08-04T14:06:52.592000-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} Processing trip starting at 2016-08-04T14:18:35.840464-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} Processing trip starting at 2016-08-04T14:39:38.288795-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} Processing trip starting at 2016-08-04T16:34:45.744782-10:00 {data: {…}, start_place: {…}, style: ƒ, onEachFeature: ƒ, pointToLayer: ƒ, …} About to hide 'Processing trips' ```
This was referenced Oct 16, 2021
Merge generic code (primarily stats) from tripaware/ceo_ebike to master
e-mission/e-mission-docs#679
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.