-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deploy the NREL OpenPATH app to staging #732
Comments
Ok, so if I change the it redirects to https://stage-host/profile/create and returns a 405 error ("Method not allowed"). The 405 error is consistent with supporting only POST, not GET on that method. Let's try:
|
Trying on android
|
Trying on python
|
Continuing on python, but explicitly using json, we get a more informative response
Looking further at the configuration for our server, we had set it up with authtype = secret Let's revert that since we no longer have that requirement for an NREL-only app |
Does not work even after reverting. This appears to be because the redeploy does not work.
|
While waiting for the server to be resolved, let's look at the iOS issue. In
Note that the baseURL is not set and instead the base URL is populated in something called
According to the NSURL documentation: this should work
|
Ah, printing out the absolute URL gives us a clue
|
We create the URL like this
which is similar to
This returns |
This is not happening because the sample code doesn't work. For the copy-pasted example in the NSURL documentation
the output is
|
Changing to the instance method does not fix it. The static method probably just turns around and calls the instance method.
returns
|
Aha! Appending at the string level works well
returns
|
Making a similar change to the actual URL code
results in the expected
which, in turn, results in the same 403 error as android and python
and on looking at the error in detail
|
While implementing this, let's change the exported value from the connection settings to return a string instead of a URL.
|
This allows us to construct relative URLs by appending strings instead of using the relative URL method, which doesn't work if the base URL is a URL as opposed to just a hostname. e-mission/e-mission-docs#732 (comment)
Instead on the relativeURL method. This is because the relative URL appraoch does not work on iOS even though the documentation says that it should Relative URL doesn't work: e-mission/e-mission-docs#732 (comment) Appending strings does work: e-mission/e-mission-docs#732 (comment)
Instead on the relativeURL method. This is because the relative URL appraoch does not work on iOS even though the documentation says that it should Relative URL doesn't work: e-mission/e-mission-docs#732 (comment) Appending strings does work: e-mission/e-mission-docs#732 (comment) Related updates: https://github.com/e-mission/cordova-connection-settings/releases/tag/v1.2.3
This is a fix for the issue where the relative URL code on iOS doesn't work per documentation. Relative URL doesn't work: e-mission/e-mission-docs#732 (comment) Appending strings does work: e-mission/e-mission-docs#732 (comment) Includes fixes to: - connection settings to return a string instead of a URL e-mission/cordova-connection-settings#22 - server-communication and server-sync to concatenate directly using strings instead of using a relative URL e-mission/cordova-server-communication#30 e-mission/cordova-server-sync#52
Let's now work on (2) and (3) from the original list while waiting for the server configuration to change. |
This is a fix for the issue where the relative URL code on iOS doesn't work per documentation. Relative URL doesn't work: e-mission/e-mission-docs#732 (comment) Appending strings does work: e-mission/e-mission-docs#732 (comment) Includes fixes to: - connection settings to return a string instead of a URL e-mission/cordova-connection-settings#22 - server-communication and server-sync to concatenate directly using strings instead of using a relative URL e-mission/cordova-server-communication#30 e-mission/cordova-server-sync#52
there used to be very clear documentation of how to use xforms in enketo, but I can't find it now. https://enketo.org/develop/#transformation Since we have merged the survey code into master, let's add this as a devDependency as well to make it easier to update the forms. The UNSW team has checked this into the rciti branch Merging it over to master and then using it on the xforms that we got from the enketo toolbox... |
npm install failure
It looks like there are no pre-built binaries, we fall back to source compile which then fails. The survey response code uses transformer v1.31.0; most recent release is 2.1.5.
Upgrading... That resulted in a lot of warnings but no errors while compiling gyp. Successful install logs
But the removal hasn't worked its way down the dependency chain since then |
Works! In English and Spanish (without translations) no less
Pending tasks:
|
Let's edit it to be similar to the previous survey to deal properly with save vs. next:
|
Original code by @atton16 in https://github.com/e-mission/e-mission-phone/tree/rk-unsw/survey-resources Modified and simplified survey from Bingrong Sun Modified further by @shankari to work with the enketo core - remove the UUID field - remove the final "estimate of commute time" field since we can get that from OpenPATH - turn on pagination e-mission/e-mission-docs#732 (comment)
Which includes adding a few additional entries to the i18n file Everything except the actual survey now shows up properly. The survey shows up in english first, and the user needs to select Spanish Let's fix that in a subsequent commit Note that we had to use a span sometimes to translate to avoid weird formatting issues e-mission/e-mission-docs#732 (comment)
Finished almost all the spanish translations e-mission/e-mission-phone#836 except that the survey is not initialized to spanish by default even when the phone language is Spanish, and we pass in the language in the options per the documentation https://github.com/enketo/enketo-core/blob/51c5c2f494f1515a67355543b435f6aaa4b151b4/tutorials/10-configuration.md#explicitly-set-the-default-form-language
Poking around this some more, it looks like this is because we are not using the most recent version of the library. Our version has
The most recent version passes in the language.
Before upgrading, and re-testing let's see how we can change the selection on our version.
when phone language is spanish, the language array order is ["es", "en"]. Maybe we can change the code to pass in the opts language? |
Here's the unreleased? patch to override the form language Let's apply it to our copy of the forms library and see if it works. |
Final steps left:
|
Finished the first step in Tracking second step in |
Merged both steps. Now, let's make the NREL OpenPATH-specific changes in the NREL fork. These changes include:
|
Made all the changes and published a staging version of the app on the stores. |
While testing, ran into platform issue with iOS12 moving on since that is a corner case |
Made some other changes:
Going to finish at least an initial implementation of the config feature and then publish the next version. |
Config feature implemented and merged into the https://github.com/e-mission/e-mission-phone/tree/master_for_platform branch. Ran into issue with scheduled tasks; they are currently built off the Tracking this in #752 |
We would also like an improvement to the token generation. For the CEO e-bike program, we had the tokens include the program as a prefix - e.g. This means that later, we will not be able to as easily distinguish between the NREL commute and open access datasets. Inserting the study name into the token will help there as well. Unfortunately, since we have already had people join both studies, including some people outside NREL, we need to handle this in a backward compat fashion. The steps for this are:
So to summarize the server case where there is a study prefix:
Then, in a month or so, we can remove this hack on both server and phone. |
…s not found This is a backwards compat fix for the issue that we prefix the opcode with the program name for programs but not for autogenerated study tokens We will fix this on the phone side, but will need a backwards compat plan for existing users until their app has been upgraded. The plan is to keep the server tokens without the study prefix (for now) and to strip out the study prefix if needed to find the matching token And once all the phones have been upgraded, we can upgrade the tokens on the server and be all done. e-mission/e-mission-docs#732 (comment) Testing so far: - Logged in with a valid token ``` 2022-07-10 17:37:53,019:DEBUG:123145553195008:Using the skip method to verify id token user-0 of length 6 2022-07-10 17:37:53,024:DEBUG:123145553195008:retUUID = ... 2022-07-10 17:37:53,030:DEBUG:123145553195008:Updating user ... ``` - Logged in with an invalid token ``` 2022-07-10 17:47:44,355:DEBUG:123145542684672:retUUID = None 2022-07-10 17:47:44,355:DEBUG:123145542684672:token ... is valid, but no account found. stripping out study name and retrying... abort(400, "Invalid study format %s, does not end with -openpath" % first_domain) ``` Need to test further on a server with a valid domain name.
This is a workaround/stopgap measure until we have the deployer dashboard working in the NREL hosted version of OpenPATH e-mission/e-mission-docs#732 Testing done: ``` ./e-mission-py.bash bin/debug/export_participants_trips_csv.py -a 2020-01-01 2022-12-31 /tmp/foo Looking up details for [UUID('a1e8c061-3932-4e9f-8d3a-8102b4c9458b'), UUID('b0a38a40-b03c-462a-9266-f381d3a5390b'), UUID('9d6d309b-b2a8-4a1e-9f94-db6d5dd1cda8'), UUID('user_id_1'), UUID('feba7c0d-33c1-4f5f-8c5a-543b320b09bc'), UUID('93ca26ce-8d8a-4d45-a41d-c6615ab791ab'), UUID('3d9dbd5b-148b-42f3-93bd-a92fa96e7650'), UUID('8e206a0f-4ec4-47d1-95c5-3436f2cf1ec0'), UUID('6076ae51-12a7-4086-9dbd-ea77e8054f78'), UUID('7dbf503c-e559-4af0-a29c-fe541c58d358'), UUID('c56053cb-699f-4a2b-a996-909c3606ddab')] INFO:root:================================================== INFO:root:Extracting trip list for user a1e8c061-3932-4e9f-8d3a-8102b4c9458b day 2020-01-01 -> 2022-12-31 and saving to file /tmp/foo_a1e8c061-3932-4e9f-8d3a-8102b4c9458b_trip_table.csv DEBUG:root:start_day_ts = 1577836800 (2020-01-01T00:00:00+00:00), end_day_ts = 1672444800 (2022-12-31T00:00:00+00:00) DEBUG:root:curr_query = {'user_id': UUID('a1e8c061-3932-4e9f-8d3a-8102b4c9458b'), '$or': [{'metadata.key': 'analysis/confirmed_trip'}], 'data.start_ts': {'$lte': 1672444800, '$gte': 1577836800}}, sort_key = data.start_ts DEBUG:root:orig_ts_db_keys = [], analysis_ts_db_keys = ['analysis/confirmed_trip'] DEBUG:root:finished querying values for [], count = 0 DEBUG:root:finished querying values for ['analysis/confirmed_trip'], count = 0 DEBUG:root:orig_ts_db_matches = 0, analysis_ts_db_matches = 0 DEBUG:root:Found 0 results INFO:root:================================================== ... INFO:root:================================================== INFO:root:Extracting trip list for user user_id_1 day 2020-01-01 -> 2022-12-31 and saving to file /tmp/foo_user_id_1_trip_table.csv DEBUG:root:start_day_ts = 1577836800 (2020-01-01T00:00:00+00:00), end_day_ts = 1672444800 (2022-12-31T00:00:00+00:00) DEBUG:root:curr_query = {'user_id': UUID('user_id_1'), '$or': [{'metadata.key': 'analysis/confirmed_trip'}], 'data.start_ts': {'$lte': 1672444800, '$gte': 1577836800}}, sort_key = data.start_ts DEBUG:root:orig_ts_db_keys = [], analysis_ts_db_keys = ['analysis/confirmed_trip'] DEBUG:root:finished querying values for [], count = 0 DEBUG:root:finished querying values for ['analysis/confirmed_trip'], count = 512 DEBUG:root:orig_ts_db_matches = 0, analysis_ts_db_matches = 512 DEBUG:root:Found 512 results DEBUG:root:After de-duping, converted 512 points to 512 ... INFO:root:================================================== INFO:root:Extracting trip list for user c56053cb-699f-4a2b-a996-909c3606ddab day 2020-01-01 -> 2022-12-31 and saving to file /tmp/foo_c56053cb-699f-4a2b-a996-909c3606ddab_trip_table.csv DEBUG:root:start_day_ts = 1577836800 (2020-01-01T00:00:00+00:00), end_day_ts = 1672444800 (2022-12-31T00:00:00+00:00) DEBUG:root:curr_query = {'user_id': UUID('c56053cb-699f-4a2b-a996-909c3606ddab'), '$or': [{'metadata.key': 'analysis/confirmed_trip'}], 'data.start_ts': {'$lte': 1672444800, '$gte': 1577836800}}, sort_key = data.start_ts DEBUG:root:orig_ts_db_keys = [], analysis_ts_db_keys = ['analysis/confirmed_trip'] DEBUG:root:finished querying values for [], count = 0 DEBUG:root:finished querying values for ['analysis/confirmed_trip'], count = 0 DEBUG:root:orig_ts_db_matches = 0, analysis_ts_db_matches = 0 DEBUG:root:Found 0 results $ ls /tmp/foo_* /tmp/foo_user_id_1_trip_table.csv /tmp/foo_participant_table.csv ```
While investigating an issue reported by a community user, Need to follow up with NREL Ops, and either add it, or reduce the batch size on the phone side. |
Ah it turns out that it was set to 1G, due to copying over from CanBikeCO. |
We have multiple deployments now; marking this as done and unpinning |
Following up on hack removal in #855 |
I didn't think that we needed this since it should be a simple matter of changing the URL, but I'm getting a 404 error, so maybe I do.
Completed task:
Current known tasks (in order):
We may want to switch to an internal fork at step 2 or just make the changes to master.
But let's finish (1) first.
The text was updated successfully, but these errors were encountered: