Errors prompt re-login #276

asiripanich · 2018-07-05T02:02:14Z

After a few hours of leaving the app on the background on my phone, I received these errors when I launched the app to check. It went back to the login step which I had done when I first installed the app.

Any idea what might be the cause of these errors?

shankari · 2018-07-05T04:40:55Z

@asiripanich, can you upload your logs too? (Profile -> Email log). You can email the log to yourself and upload it here, or you can email directly to me for greater privacy.

shankari · 2018-07-05T04:48:46Z

Others have reported this issue before, but they have not been developers, so I have not been able to get the logs in time.

Current speculation is that this is happens because of apps that are "offloaded" when you are short of space.
www.iphonehacks.com/2017/08/how-to-automatically-remove-unused-apps-free-up-storage-space-iphone-ipad-ios-11.html

So can you comment on how much space you have available? Also the logs will help me debug exactly what is not getting restored when the app is restored again.

asiripanich · 2018-07-05T05:50:30Z

@shankari

Sent my log file.

The offload option has never been enable on my phone (iOS 11.4, iPhoneX). So that might not be the cause for my case.

Space available on my phone: 54.4 GB of 64 GB used.

shankari · 2018-08-23T18:36:21Z

A similar issue was reported by @manuelsantanap while testing on emTripLog.
Looking at his logs, it looks like the underlying issue was with sharedPreferences not being stored properly (https://github.com/e-mission/e-mission-base/issues/3#issuecomment-411586627). This is an issue with the ionic deploy plugin.

Let's use this issue to track the reset UI issue.

shankari · 2018-08-23T20:27:39Z

I quickly looked through these logs to see if this was the same issue, and unfortunately, there are no such logs.

$ grep "A newer version" /tmp/loggerDB.amarin.unable_to_login.withdate.log | wc -l
       0

However, this is a not a debug build, so we really will not expect it to be.

shankari · 2018-08-23T20:31:48Z

I tried to upgrade the plugin to see if that fixed the issue. Also, if I wanted to file an issue with ionic, they would likely be more responsive to a test against the most recent version. I ran into a separate issue with testing the most recent plugin (https://github.com/e-mission/e-mission-base/issues/3#issuecomment-411832143) that I filed an issue for. They have now apparently fixed it in v5.0.9.

I can upgrade to 5.0.9, but I need help from @asiripanich and @manuelsantanap since I am unable to test locally.

shankari · 2018-08-24T17:36:10Z

So v5.0.9 has fixed the immediate issue. Trying to see it if works even after rebooting...

shankari · 2018-08-24T17:36:45Z

Current PR (e-mission/e-mission-base#5)
Note that this is on a separate branch so that we can leave it out of the release if we want.

shankari · 2018-08-25T05:21:08Z

I deployed a version with both this change and e-mission/e-mission-base#5 and tested by rebooting multiple times. Every time, the correct client was launched.

@asiripanich @manuelsantanap do you have an ETA for when you can test this version out and see if it fixes the issue?

shankari · 2018-08-25T05:27:48Z

An apk that includes changes from both e-mission/e-mission-base#4 and e-mission/e-mission-base#5 is attached here. For iOS testing, Manuel has already accepted my TestFlight invite - I will send out a new one to @asiripanich once I get his email. Please accept those invites so that you can test this version before I release it.

shankari · 2018-08-25T15:47:46Z

Invited @asiripanich, waiting for response. Incidentally, waiting for response from @brianmickel as well, just resent invitation.

shankari · 2018-08-25T17:45:18Z

Added in fixes in e-mission/cordova-jwt-auth#28 and e-mission/cordova-jwt-auth#31 as well by pulling them to a local copy of the auth plugin and then including it instead of the version from the internet.

shankari · 2018-08-25T18:09:25Z

Here's the apk, uploading the ios version to TestFlight... emTripLog-ionic-5-plugin.apk.gz

asiripanich · 2018-08-26T04:00:58Z

Hi @shankari, I can't find your invitation. Could you please resent it again? Thanks.

shankari · 2018-08-26T05:25:44Z

Done. Please check the email that is associated with your AppleID - the invite is sent to that address.

asiripanich · 2018-08-26T12:04:27Z

For some reason, the email that I shared with you haven't been receiving email since 04/2017. There is something wrong with its inbox which I still cannot fix. Is there an alternative?

shankari · 2018-08-27T01:02:33Z

I originally thought that there wasn't, but it looks like the rules have changed.
I can now send an invite to any email ID, not just the one associated with your apple ID.
https://help.apple.com/app-store-connect/#/devdbefef011

The email address must be valid to activate the account. It can be any email address, not necessarily > one that’s associated with an Apple ID.

Resending to your current email address...

asiripanich · 2018-08-27T11:35:15Z

It is not letting me sign in with my appleID (not the one that you sent the invite to). :/ Sorry for the trouble.

shankari · 2018-08-27T14:04:30Z

I think that if I send you an invite to a different address, it makes a new AppleID with that address -

If the account was created with an email address not already associated with an Apple ID, App Store Connect creates an Apple ID using the information supplied for the App Store Connect account. The user is asked to enter a password and create a security question and answer before App Store Connect opens.

Users manage their own Apple ID names and passwords on the My Apple ID webpage.

So you need to click on the link in the invite, make a new AppleID and then sign in with that one.

shankari · 2018-09-06T00:28:28Z

I met with @manuelsantanap and he was able to give me instructions to reproduce the issue in the android emulator:

install the app
switch to a client
reboot the phone
UI gets reset to the default

However, when I try the same set of steps in the iOS emulator, I don't see the UI getting reset. @asiripanich can you give me instructions to reproduce? How did you manage to get the UI to reset? If I can reproduce the error, I can verify which ionic deploy update works, and then you don't need to test using TestFlight.

shankari · 2018-09-06T00:33:04Z

I also tried on a physical device (running iOS 9.2) and it works fine. I force killed and then rebooted. No issues.

shankari · 2018-09-06T00:51:41Z

ok, so I found an issue, although it is not the same one reported. But this is on iOS 9.2, so maybe it manifests itself differently. The steps that I followed, that worked only on iOS were:

install emTripLog
switch to a client
create a backup using iTunes
restore the phone from the backup

emTripLog is still installed, however, when I try to launch it, it crashes consistently.

@asiripanich do you have iTunes automatic backup turned on by any chance?

shankari · 2018-09-06T02:21:51Z

@asiripanich also, what version of iOS are you running?
@manuelsantanap, do you have iCloud backup turned on your test phone? (https://support.apple.com/en-us/HT203977#icloud)
If you do, are you able to reproduce the original error reported in this issue?

shankari · 2018-09-06T06:36:32Z

ok, so I launched the app again on iOS 9.2, and captured the logs from the console when it crashed. The relevant logs are:

1900-09-05 19:50:43.000000 -0800    emTripLog    [CDVTimer][usercache] 0.584006ms
1900-09-05 19:50:43.000000 -0800    emTripLog    [CDVTimer][keyboard] 1.835942ms
1900-09-05 19:50:43.000000 -0800    emTripLog    [CDVTimer][splashscreen] 16.667962ms
1900-09-05 19:50:43.000000 -0800    emTripLog    VERSION LABEL: 1.0.6:1535864385:67f4d71a-e337-4a8b-99b2-eebf40b40c52
1900-09-05 19:50:43.000000 -0800    emTripLog    uuid is: 67f4d71a-e337-4a8b-99b2-eebf40b40c52
1900-09-05 19:50:43.000000 -0800    emTripLog    ignore version: NOTHING_TO_IGNORE
1900-09-05 19:50:43.000000 -0800    emTripLog    [CDVTimer][ionicdeploy] 9.752989ms
1900-09-05 19:50:43.000000 -0800    emTripLog    [CDVTimer][ioniccordovacommon] 0.433087ms
1900-09-05 19:50:43.000000 -0800    emTripLog    BEMJWTAuth:pluginInitialize singleton -> initialize completion handler
1900-09-05 19:50:43.000000 -0800    emTripLog    DEBUG: BEMJWTAuth:pluginInitialize singleton -> initialize completion handler
1900-09-05 19:50:43.000000 -0800    emTripLog    *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[NSRegularExpression enumerateMatchesInString:options:range:usingBlock:]: nil argument'
    *** First throw call stack:
    (0x181fdae38 0x18163ff80 0x181fdad80 0x182920324 0x182937eb0 0x1000969f8 0x181a254bc 0x181a2547c 0x181a314c0 0x181a28f80 0x181a2547c 0x181a33914 0x181a330b0 0x181c3d470 0x181c3d020)
1900-09-05 19:50:43.000000 -0800    ReportCrash  platform_thread_get_unique_id matched 27168
1900-09-05 19:50:43.000000 -0800    ReportCrash  Formulating report for corpse[291] emTripLog
26540-09-05 19:50:4.0000004 -0700   SpringBoard  Application 'UIKitApplication:edu.berkeley.eecs.embase[0xe211]' crashed.
1900-09-05 19:50:44.000000 -0800    UserEventAgent   46419344340: id=edu.berkeley.eecs.embase pid=291, state=0
26540-09-05 19:50:4.0000004 -0700   ReportCrash  Saved type '109(109_emTripLog)' report (2 of max 25) at /var/mobile/Library/Logs/CrashReporter/emTripLog-2018-09-05-195044.ips.beta

There are many matches for NSRegularExpression

$ grep -r NSRegularExpression platforms/ios | grep -v build | wc -l
      24

but none for enumerateMatchesInString.

$ grep -r enumerateMatchesInString platforms/ios | grep -v build | wc -l
       0

Let's see if we can find the crash log for more details.

shankari · 2018-09-06T06:37:28Z

Can't find the crash logs even after plugging in the phone and opening the "Devices" tab in Xcode.
Let's try running in debug mode and see if any crash logs show up.

shankari · 2018-09-06T06:48:37Z

Aha! the crash was reproducible in debug mode, and the crash logs showed up with symbolication. The backtrace is

Last Exception Backtrace:
0   CoreFoundation                	0x182e66e38 0x182d3c000 + 1224248
1   libobjc.A.dylib               	0x1824cbf80 0x1824c4000 + 32640
2   CoreFoundation                	0x182e66d80 0x182d3c000 + 1224064
3   Foundation                    	0x1837ac324 0x183748000 + 410404
4   Foundation                    	0x1837c3eb0 0x183748000 + 507568
5   emTripLog                     	0x10013ac80 __25-[IonicDeploy doRedirect]_block_invoke + 699520 (IonicDeploy.m:555)
6   libdispatch.dylib             	0x1828b14bc 0x1828b0000 + 5308
7   libdispatch.dylib             	0x1828b147c 0x1828b0000 + 5244
8   libdispatch.dylib             	0x1828bd4c0 0x1828b0000 + 54464
9   libdispatch.dylib             	0x1828b4f80 0x1828b0000 + 20352
10  libdispatch.dylib             	0x1828b147c 0x1828b0000 + 5244
11  libdispatch.dylib             	0x1828bf914 0x1828b0000 + 63764
12  libdispatch.dylib             	0x1828bf0b0 0x1828b0000 + 61616
13  libsystem_pthread.dylib       	0x182ac9470 0x182ac8000 + 5232
14  libsystem_pthread.dylib       	0x182ac9020 0x182ac8000 + 4128

Related code is

                // Load the target index.html and init values
                NSString *htmlData = [NSString
                                      stringWithContentsOfFile:components.path
                                      encoding:NSUTF8StringEncoding
                                      error:nil];
                NSString *newReference = [NSString
                                          stringWithFormat:@"<script src=\"%@\"></script>", self.cordova_js_resource];
                NSError *error = nil;

                // Find an existing cordova.js tag
                NSRegularExpression *cordovaRegex = [NSRegularExpression
                                                     regularExpressionWithPattern:@"<script src=(\"|')(.*\\/|)cordova\\.js.*(\"|')>.*<\\/script>"
                                                     options:NSRegularExpressionCaseInsensitive
                                                     error:&error];
--->            NSArray *matches = [cordovaRegex matchesInString:htmlData options:0 range:NSMakeRange(0, [htmlData length])];
                if (matches && matches.count){
                    // We found the script, update it
                    htmlData = [cordovaRegex
                                stringByReplacingMatchesInString:htmlData
                                options:0
                                range:NSMakeRange(0, [htmlData length])
                                withTemplate:newReference];
                }

                // Write new index.html
                [htmlData writeToFile:components.path atomically:YES encoding:NSUTF8StringEncoding error:nil];

Note that the code at the line is [NSRegularExpression matchesInString] not [NSRegularExpression enumerateMatchesInString]. However, matchesInString is

a convenience method that calls enumerateMatchesInString:options:range:usingBlock: passing the appropriate string, options, and range.

https://developer.apple.com/documentation/foundation/nsregularexpression/1412446-matchesinstring

This explains why we didn't find the relevant code through grepping.

shankari · 2018-09-06T06:54:16Z

The last recommended ionic deploy if we want to support older android and iOS versions is 4.1.7

The maximum supported versions for the full legacy implementation of deploy with support for iOS 10.2 and lower are shown below. If you'd like to stick with the earlier implementation we recommend these versions.
cordova-plugin-ionic v4.1.7

But that doesn't appear to have fixed this issue.
https://github.com/ionic-team/cordova-plugin-ionic/blob/4.1.7/src/ios/IonicDeploy.m#L614

It looks like the issue is that the new HTML was nil argument. So it seems like upgrading to v4.1.7 is unlikely to fix the issue. Since ionic has stated that they will not bugfix ionic deploy 4

Deploy v4 will still be available, but will not be fully supported going forward. If you use that version you're getting the plugin "as is". I don't anticipate any future releases or bugfixes for v4.

I need to fork the ionic deploy code and fix it there.

shankari · 2018-09-06T07:01:20Z

Ionic deploy 4.1.7 depends on ionic webview version 1.1.19, which should work on iOS 9+
https://github.com/ionic-team/cordova-plugin-ionic-webview/tree/v1.1.19
So technically we can upgrade first and fix the cloned version from 4.1.7

shankari · 2018-09-06T07:04:02Z

This seems to be a slightly different issue than android, though, because

1900-09-05 19:50:43.000000 -0800    emTripLog    uuid is: 67f4d71a-e337-4a8b-99b2-eebf40b40c52

which means that it is read properly from the shared preferences. It looks like the real issue is that there is no html file for that uuid

shankari · 2018-09-06T07:19:34Z

After adding some more logging, this is what I see:

2018-09-06 00:08:05.435 emTripLog[321:24295] UUID is:  and upstream_uuid is: 67f4d71a-e337-4a8b-99b2-eebf40b40c52
2018-09-06 00:08:05.435 emTripLog[321:24295] Download Finished...
2018-09-06 00:08:05.511 emTripLog[321:24484] Versions: (
)
2018-09-06 00:08:05.511 emTripLog[321:24484] Path for zip file: /var/mobile/Containers/Data/Application/08C38781-5A5F-4056-BF61-7D69DBECCF69/Library/Application Support/www.zip
....
2018-09-06 00:08:11.698 emTripLog[321:24484] Excluding /var/mobile/Containers/Data/Application/08C38781-5A5F-4056-BF61-7D69DBECCF69/Library/Application Support/67f4d71a-e337-4a8b-99b2-eebf40b40c52 from backup
...
2018-09-06 00:08:12.903 emTripLog[321:24295] uuid is: 67f4d71a-e337-4a8b-99b2-eebf40b40c52
2018-09-06 00:08:12.903 emTripLog[321:24295] ignore version: NOTHING_TO_IGNORE
2018-09-06 00:08:12.904 emTripLog[321:24637] paths in the application support directory = (
    "/var/mobile/Containers/Data/Application/08C38781-5A5F-4056-BF61-7D69DBECCF69/Library/Application Support"
)
2018-09-06 00:08:12.906 emTripLog[321:24637] about to load data from <NSURLComponents 0x127a14b40> {scheme = file, user = (null), password = (null), host = (null), port = (null), path = /var/mobile/Containers/Data/Application/08C38781-5A5F-4056-BF61-7D69DBECCF69/Library/Application%20Support/67f4d71a-e337-4a8b-99b2-eebf40b40c52/index.html, query = cordova_js_bootstrap_resource=/var/containers/Bundle/Application/95BEAEEE-C9F7-4B5B-B0E4-B3238B98DAAA/emTripLog.app/www/cordova.js, fragment = (null)}
2018-09-06 00:08:12.907 emTripLog[321:24637] loaded HTML data = <!DOCTYPE html>
<html>
  <head>
  </body>
</html>
2018-09-06 00:08:12.929 emTripLog[321:24637] Redirecting to: file:/var/mobile/Containers/Data/Application/08C38781-5A5F-4056-BF61-7D69DBECCF69/Library/Application%20Support/67f4d71a-e337-4a8b-99b2-eebf40b40c52/index.html?cordova_js_bootstrap_resource=/var/containers/Bundle/Application/95BEAEEE-C9F7-4B5B-B0E4-B3238B98DAAA/emTripLog.app/www/cordova.js
2018-09-06 00:08:12.930 emTripLog[321:24295] Reloading the web view.
2018-09-06 00:08:12.938 emTripLog[321:24295] Resetting plugins due to page load.
2018-09-06 00:08:12.944 emTripLog[321:24295] Failed to load webpage with error: The operation couldn’t be completed. (NSURLErrorDomain error -999.)
2018-09-06 00:08:12.945 emTripLog[321:24295] Resetting plugins due to page load.
2018-09-06 00:08:13.471 emTripLog[321:24295] Finished load of: file:///var/mobile/Containers/Data/Application/08C38781-5A5F-4056-BF61-7D69DBECCF69/Library/Application%20Support/67f4d71a-e337-4a8b-99b2-eebf40b40c52/index.html?cordova_js_bootstrap_resource=/var/containers/Bundle/Application/95BEAEEE-C9F7-4B5B-B0E4-B3238B98DAAA/emTripLog.app/www/cordova.js
2018-09-06 00:08:13.742 emTripLog[321:24295] DEBUG: ionicPlatform is ready

shankari · 2018-09-06T16:02:51Z

Maybe this works better in "auto" mode for the plugin; which is the default, instead of "none" which is what I am using now. I'm going to try that out now...

shankari · 2018-09-06T16:47:34Z

Also, the current version of the plugin (v5) doesn't seem to exclude the newly unzipped directory from backup, so a cheap fix may just be to comment out that code.

shankari · 2018-09-06T19:29:15Z

wrt https://github.com/e-mission/e-mission-phone/issues/443#issuecomment-419149774, version 3.0.0 auto mode also exhibits the same behavior on both android and iOS. Next step, testing the most recent backwards compatible release. Step after that, testing on an iOS 11.4 physical device, since the most recent version of the deploy plugin doesn't support iOS 9.2 anyway.

shankari · 2018-09-06T20:00:11Z

most recent backwards compatible release (v4.1.7) also fails on both android and iOS in exactly the same way.

shankari · 2018-09-06T22:35:37Z

upgraded to v5.0.9 because moving to @latest (5.1.2) requires upgrading the ionic cli in order to support the deploy command and run ionic deploy manifest

This is the version I had tested earlier, but it is now crashing consistently on the android 8 emulator, and I don't know why. Might try to rebuild from the checked in version to see if that fixes it, but don't want to spend too much time on this since I know that it is not going to be the final solution anyway because of the need to support older versions at least for now.

shankari · 2018-09-07T21:07:36Z

wrt https://github.com/e-mission/e-mission-phone/issues/443#issuecomment-419263343,

This is the version I had tested earlier, but it is now crashing consistently on the android 8 emulator, and I don't know why.

This was because of Out Of Memory (OOM) errors. I closed everything except for android studio and it ran. Current outcome of 5.0.9 -

android: new UI sticks after reboot
iOS: after backup and restore, app does not crash, but does not work either; just hangs on startup

shankari · 2018-09-07T21:53:13Z

Filed issue against ionic for the iOS stuff.
Obvious fix is to store the snapshot in a backed up location, but not sure why they didn't do that in the first place.
Once I hear back from them, I will fork, fix these issues on both android and iOS, and switch to the forked plugin (at least for now).

asiripanich · 2018-09-09T23:57:27Z

Hi @shankari, this has happened to me many times even with iOS 12 beta. Thank you for all the updates.

shankari · 2018-09-11T14:17:06Z

Ok, so I was finally able to reproduce this exact behavior in an emulator. I wasn't expecting it to happen, so I didn't get as many logs as I had hoped, but maybe if I try it again, it will happen again.

Steps that I did:

Launched a newly built app on an emulator where the app was already installed with the specified channel. App was the same version, but had been re-built.
App started with the base screen.
Clicked on the customization link
After the snapshot was extracted, while the app was re-loading, I saw this error

Since the app was still running in the debugger, I was able to extract logs, which I have attached here.
bug_443_repro.gz

shankari · 2018-09-11T15:16:17Z

Unfortunately, I don't really have the console logs, unless they are reproduced in xcode.
But here's what I see:

All plugins were loaded properly and ionicPlatform fired correctly.

2018-09-10 22:23:36.395530-0700 emTripLog[74123:7728962] DEBUG: ionicPlatform is ready

Initially, we were on production and didn't have any updates.

2018-09-10 22:23:36.401524-0700 emTripLog[74123:7728962] Got app preferences: {
    appId = e0d8cdec;
    channel = Production;
    debug = true;
    host = "https://api.ionicjs.com";
    maxVersions = 2;
    minBackgroundDuration = 30;
    updateMethod = none;
    updates =     {
    };
}
2018-09-10 22:23:36.686374-0700 emTripLog[74123:7728962] DEBUG:Ionic Deploy: Update available: {"partial":false,"snapshot":null,"compatible":true,"available":false}

Received a change to the opentoall client

2018-09-10 22:43:40.572393-0700 emTripLog[74123:7728962] in handleNotification, received url = emission://change_client?new_client=opentoall&clear_local_storage=true&clear_usercache=

Update is now available

2018-09-10 22:43:40.701579-0700 emTripLog[74123:7728962] DEBUG:Ionic Deploy: Update available: {"url":"https://api.ionicjs.com/apps/e0d8cdec/snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/manifest","partial":false,"snapshot":"67f4d71a-e337-4a8b-99b2-eebf40b40c52","compatible":true,"available":true}

We apply the update

2018-09-10 22:43:42.792341-0700 emTripLog[74123:7728962] Downloading update...
2018-09-10 22:43:58.876675-0700 emTripLog[74123:7728962] Wrote files in 10.639 seconds.

We extracted a bunch of files. Note that for angularLocalStorage, and several other javascript packages, we only download .bower.json. I suspect this is because the update had been downloaded earlier, so we don't need to extract the files again.

2018-09-10 22:44:01.580271-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/angular-sanitize/.bower.json
2018-09-10 22:44:01.586858-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/angular-ui-router/.bower.json
2018-09-10 22:44:01.591436-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/angularLocalStorage/.bower.json

However, we later do extract other files for angular-sanitize in a second round that begins ~ 2 mins later

$ zgrep angular-sanitize /tmp/bug_443_repro.gz
2018-09-10 22:44:01.580271-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/angular-sanitize/.bower.json
2018-09-10 22:44:03.198921-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/ionic/js/angular/angular-sanitize.js
2018-09-10 22:44:03.199864-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/ionic/js/angular/angular-sanitize.min.js

and angular-ui-router

$ zgrep angular-ui-router /tmp/bug_443_repro.gz
2018-09-10 22:44:01.586858-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/angular-ui-router/.bower.json
2018-09-10 22:44:03.202858-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/ionic/js/angular-ui/angular-ui-router.js
2018-09-10 22:44:03.203845-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/ionic/js/angular-ui/angular-ui-router.min.js

but not for angularLocalStorage

$ zgrep angularLocalStorage /tmp/bug_443_repro.gz
2018-09-10 22:44:01.591436-0700 emTripLog[74123:7728962] New file file:///Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52/lib/angularLocalStorage/.bower.json

Extraction is complete

2018-09-10 22:44:06.776821-0700 emTripLog[74123:7728962] extract progress = 100
2018-09-10 22:44:10.738349-0700 emTripLog[74123:7728962] DEBUG:Ionic Deploy: Update Success! true

Current base path is not yet for the snapshot but for the app.

2018-09-10 22:44:12.345894-0700 emTripLog[74123:7728962] fetched current base path as /Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Bundle/Application/9FA9E212-9007-4C76-ABFF-7477F6762E9F/emTripLog.app/www

Now, we start getting a bunch of method not found errors in javascript.

2018-09-10 22:44:14.227104-0700 emTripLog[74123:7728962] DEBUG:Ionic Deploy: Unable to check for updatesTypeError: deploy.init is not a function. (In 'deploy.init(config, resolve, reject)', 'deploy.init' is undefined)
2018-09-10 22:44:14.227328-0700 emTripLog[74123:7728962] DEBUG: Ionic Deploy: Unable to check for updatesTypeError: deploy.init is not a function. (In 'deploy.init(config, resolve, reject)', 'deploy.init' is undefined)
2018-09-10 22:44:14.229757-0700 emTripLog[74123:7728962] ERROR: Ionic Deploy: Unable to check for updates TypeError: deploy.init is not a function. (In 'deploy.init(config, resolve, reject)', 'deploy.init' is undefined)

2018-09-10 22:44:14.968682-0700 emTripLog[74123:7728962] DEBUG:local intro_done false, remote intro_done 2018-09-01T21:53:03-07:00, setting local
2018-09-10 22:44:14.968919-0700 emTripLog[74123:7728962] DEBUG: local intro_done false, remote intro_done 2018-09-01T21:53:03-07:00, setting local
2018-09-10 22:44:14.971550-0700 emTripLog[74123:7728962] DEBUG:error in checkUsercacheStorage = storage.put is not a function. (In 'storage.put(key, true)', 'storage.put' is undefined)
http://localhost:8080/js/splash/startprefs.js:198:32
promiseReactionJob@[native code]
2018-09-10 22:44:14.971782-0700 emTripLog[74123:7728962] DEBUG: error in checkUsercacheStorage = storage.put is not a function. (In 'storage.put(key, true)', 'storage.put' is undefined)
http://localhost:8080/js/splash/startprefs.js:198:32
promiseReactionJob@[native code]

And only now, we switch the base path to the snapshot

2018-09-10 22:44:15.042263-0700 emTripLog[74123:7728962] fetched current base path as /Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52

When the app stopped at that point, I connected Safari and confirmed that angularLocalStorage was loaded as a script.

index.html	angularLocalStorage.min.js

After I clicked through the errors, the rest of the onboarding was fine, with no errors.

shankari · 2018-09-11T15:19:39Z

So right now, the issue appears to be the race between resetting the base path, and reloading the new javascript. It is only in that window of time that we see errors.

$ zegrep "fetched current base path|not a function" /tmp/bug_443_repro.gz
2018-09-10 22:44:12.345894-0700 emTripLog[74123:7728962] fetched current base path as /Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Bundle/Application/9FA9E212-9007-4C76-ABFF-7477F6762E9F/emTripLog.app/www
2018-09-10 22:44:14.227104-0700 emTripLog[74123:7728962] DEBUG:Ionic Deploy: Unable to check for updatesTypeError: deploy.init is not a function. (In 'deploy.init(config, resolve, reject)', 'deploy.init' is undefined)
2018-09-10 22:44:14.227328-0700 emTripLog[74123:7728962] DEBUG: Ionic Deploy: Unable to check for updatesTypeError: deploy.init is not a function. (In 'deploy.init(config, resolve, reject)', 'deploy.init' is undefined)
2018-09-10 22:44:14.229757-0700 emTripLog[74123:7728962] ERROR: Ionic Deploy: Unable to check for updates TypeError: deploy.init is not a function. (In 'deploy.init(config, resolve, reject)', 'deploy.init' is undefined)
2018-09-10 22:44:14.971550-0700 emTripLog[74123:7728962] DEBUG:error in checkUsercacheStorage = storage.put is not a function. (In 'storage.put(key, true)', 'storage.put' is undefined)
2018-09-10 22:44:14.971782-0700 emTripLog[74123:7728962] DEBUG: error in checkUsercacheStorage = storage.put is not a function. (In 'storage.put(key, true)', 'storage.put' is undefined)
2018-09-10 22:44:15.042263-0700 emTripLog[74123:7728962] fetched current base path as /Users/shankari/Library/Developer/CoreSimulator/Devices/6C5761EE-50C4-4B08-8A8C-FB2C085E9E27/data/Containers/Data/Application/EBA306F4-C827-41EB-94CC-9DB48E878859/Library/NoCloud/ionic_built_snapshots/67f4d71a-e337-4a8b-99b2-eebf40b40c52

shankari · 2018-09-11T15:25:44Z

I'm going to replace those alerts with javascript alerts so that the javascript execution is paused and I can see the current state of the loaded app when the errors occur. I'll have to do that on a different channel, one that nobody is using. Let's use interscity which I think nobody is using right now.

shankari · 2018-09-11T15:34:16Z

Just tried the same steps as https://github.com/e-mission/e-mission-phone/issues/443#issuecomment-420290336

In step 2, app launched with custom channel. Dunno how/why this works sometimes and not other times.

shankari · 2018-09-20T00:44:04Z

I am an idiot. Given that I was able to reproduce this, I was able to figure out what happened. And suddenly it all became clear.

First, we cannot use localstorage reliably (at least on iOS) because the OS clears it if the phone is low on space.
https://cordova.apache.org/docs/en/8.x/cordova/storage/storage.html

iOS stores localStorage data in a location that may be cleaned out by the OS when space is required.

So the check for local storage versus native storage fails periodically and generates the first popup.

                    $ionicPopup.alert({template: "Local "+key+" not found, native "+key+" found, writing "+key+" to local"})

And then we call storage.put

                    logger.log("local intro_done false, remote intro_done "+uc_stored_val[key]+", setting local");
                    $ionicPopup.alert({template: "Local "+key+" not found, native "+key+" found, writing "+key+" to local"})
                    storage.put(key, true);

But the localStorage API doesn't have put! The correct call, as we can see here

    startprefs.setDefaultTheme = function(new_theme) {
      storage.set(DEFAULT_THEME_KEY, new_theme);
    }

is storage.set, so that is what generates the second error, the one I have been trying to track down, the one from pure javascript.

So basically, the iOS fix consists of two parts:

change storage.put -> storage.set (easy)
look at instances of angularLocalStorage and replace them all with native storage, at least on iOS. Or at least, don't popup an alert if the local storage is not present, because that is expected behavior.

shankari · 2018-09-20T07:11:53Z

I think that the long-term solution is to remove all instances of angularLocalStorage and replace them with native storage. However, sometimes native storage is flaky too (see the android plist issue) and I am reluctant to rely only on it. Note that I currently have double-backups - if there is local storage but not native storage, then I set the native storage and if there is native storage and not local storage, I set the local storage.

Arguments in favor of switching entirely to native storage:

simplifies the code
have not had a recent example of the native storage getting deleted

Arguments in favor of keeping the backup:

It must have happened before (which is why I added the checks)
Redundancy is good.

For now, I will refactor the storage into its own class that will have the interface of native storage, but will include the backup of checking against local storage.

This will simplify the interface, keep the backup in place, and make it easier to remove the backup if it turns out to not be necessary.

Back in the old days, we tried to move all storage to local storage so that we could read it without waiting for ionicPlatform.ready and startup quickly. Well, it turned out that sometimes local storage was deleted on iOS, so we ended up with the case where local intro_done was deleted, so we would reset the UI. Which looked stupid and like a bug (https://github.com/e-mission/e-mission-phone/issues/443). Re-writing to pull out the KV storage into a service, write primarily to native with local as backup, and read from both and unify. Everything was already promisified because we were reading consent from native code, so I am not sure that this actually affects performance. And it is a heck of a lot easier to understand. Testing done: - started app -> went to intro - restarted app after restarting emulator -> went to current screen. For fixes to the current screen, see 0ecfdae - cleared out intro_done and consent from local storage and restarted app -> went to current screen - cleared all native storage, generated popups for intro_done and consent, then crashed due to e-mission/e-mission-transition-notify#18 On relaunch, launched properly without any popups. I declare that this is done. ``` [phonegap] [console.log] DEBUG:uc_stored_val = null ls_stored_val = {"intro_done":"2018-09-25T13:14:51-07:00"} ```

shankari · 2018-09-26T18:03:13Z

ok so now that we have figured out the iOS issue, we need to look at the android issue.

I put a breakpoint in the code that displays the debug message (https://github.com/e-mission/e-mission-base/issues/3#issuecomment-410544856).

The backtrace is:

  public Object onMessage(String id, Object data) {
    boolean is_nothing = "file:///".equals(String.valueOf(data));
    boolean is_index = "file:///android_asset/www/index.html".equals(String.valueOf(data));
    boolean is_original = (is_nothing || is_index) ? true : false;

    if("onPageStarted".equals(id) && is_original) {
      final String uuid = this.getUUID();

      if(!IonicDeploy.NO_DEPLOY_AVAILABLE.equals(uuid)) {
        logMessage("LOAD", "Init Deploy Version");
        if (this.isDebug()) {
          this.showDebug();
        } else {
          this.redirect(uuid);
        }
      }
    }
    return null;
  }

The uuid is c99ee4a8-ed02-4736-963f-c2ee9398f972

This seems pretty correct. Is this just a debug issue and it works in release builds. Let's try to reproduce.

shankari · 2018-09-26T18:10:27Z

Yes, this is only a debug issue.
Installed the release apk, switched to the megumi-brt channel, rebooted, still on the megumi-brt channel.

Let's try to see if the upgrade issue still exists and fix it.

shankari · 2018-09-26T18:18:27Z

upgrade issue still exists. Let's see again what the underlying issue is.

shankari · 2018-09-27T13:06:39Z

Fixed by changing code in initVersionChecks to

      if(!this.version_label.equals(ionicdeploy_version_label)) {
        // this.ignore_deploy = true;
        this.updateVersionLabel(IonicDeploy.NOTHING_TO_IGNORE);
        // this.prefs.edit().remove("uuid").apply();
      }

shankari · 2018-09-27T13:31:07Z

There is similar code on iOS as well. Let's remove it and verify

    if(![ionicdeploy_version_label isEqualToString: NO_DEPLOY_LABEL]) {
        if(![self.version_label isEqualToString: ionicdeploy_version_label]) {
            self.ignore_deploy = true;
            [self updateVersionLabel:uuid];
            [prefs setObject: @"" forKey: @"uuid"];
            [prefs setBool:YES forKey:@"show_splash"];
            [prefs synchronize];
        }
    }

shankari · 2018-09-27T14:10:56Z

Verified that default code caused the reset.
There is no relevant documentation for the blame.
Removing it, though, causes the code to try to apply the update again, which then hangs on extract.
This may be the reason why this code was added, let's explore it further and see how to fix it.

shankari · 2018-09-27T14:44:46Z

Unable to reproduce the previous issue.
https://github.com/e-mission/e-mission-phone/issues/443#issuecomment-425107505

Went from version 6 -> version 10, updating at every stage and it loaded just fine.
Going to fork, test and submit for at least emTripLog this weekend.

shankari · 2018-10-19T18:59:28Z

@asiripanich I spent a whole day deploying all the fixes and I believe that this issue is fixed.

The fork of the deploy plugin is now in emTripLog (e-mission/e-mission-base#10, https://github.com/e-mission/e-mission-base/releases/tag/v1.1.0, e-mission (#481, https://github.com/e-mission/e-mission-phone/releases/tag/v2.7.1)and em-devapp (e-mission/e-mission-devapp#7, https://github.com/e-mission/e-mission-devapp/releases/tag/v2.0.2).

The changes to store state in native, rather than local storage (e-mission/e-mission-phone#478) are checked into master. I have also merged these changes from master (e-mission/e-mission-phone#484) and deployed a new version to your channel.

Can you keep an eye out and confirm that the UI does not get reset any more? Thanks!

shankari · 2018-11-01T15:13:26Z

@asiripanich can you confirm that this issue is now fixed?

asiripanich · 2018-11-04T23:35:44Z

@shankari Yes, we no longer experience this issue. But we are not sure if this is because the updates you made or us switched to the cci branch.

shankari · 2018-11-04T23:44:01Z

you didn't actually switch to the cci channel, right? You just merged from the channel?
If so, I don't think that is the reason.
At any rate, if you do see the issue again, please re-open :)

asiripanich · 2018-11-05T00:17:23Z

you didn't actually switch to the cci channel, right? You just merged from the channel?

Yes we merged. Sorry for the confusion. :)
Thanks Shanakri.

shankari closed this as completed Oct 19, 2018

shankari transferred this issue from e-mission/e-mission-phone Feb 11, 2019

Errors prompt re-login #276

Errors prompt re-login #276

Comments

asiripanich commented Jul 5, 2018

shankari commented Jul 5, 2018

shankari commented Jul 5, 2018

asiripanich commented Jul 5, 2018

shankari commented Aug 23, 2018 • edited Loading

shankari commented Aug 23, 2018

shankari commented Aug 23, 2018

shankari commented Aug 24, 2018

shankari commented Aug 24, 2018

shankari commented Aug 25, 2018

shankari commented Aug 25, 2018 • edited Loading

shankari commented Aug 25, 2018

shankari commented Aug 25, 2018

shankari commented Aug 25, 2018

asiripanich commented Aug 26, 2018

shankari commented Aug 26, 2018

asiripanich commented Aug 26, 2018

shankari commented Aug 27, 2018

asiripanich commented Aug 27, 2018

shankari commented Aug 27, 2018

shankari commented Sep 6, 2018 • edited Loading

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018 • edited Loading

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018 • edited Loading

shankari commented Sep 6, 2018

shankari commented Sep 6, 2018 • edited Loading

shankari commented Sep 6, 2018 • edited Loading

shankari commented Sep 6, 2018 • edited Loading

shankari commented Sep 7, 2018

shankari commented Sep 7, 2018

asiripanich commented Sep 9, 2018 • edited Loading

shankari commented Sep 11, 2018 • edited Loading

shankari commented Sep 11, 2018

shankari commented Sep 11, 2018 • edited Loading

shankari commented Sep 11, 2018 • edited Loading

shankari commented Sep 11, 2018

shankari commented Sep 20, 2018

shankari commented Sep 20, 2018

shankari commented Sep 26, 2018

shankari commented Sep 26, 2018

shankari commented Sep 26, 2018

shankari commented Sep 27, 2018

shankari commented Sep 27, 2018

shankari commented Sep 27, 2018

shankari commented Sep 27, 2018

shankari commented Oct 19, 2018

shankari commented Nov 1, 2018

asiripanich commented Nov 4, 2018

shankari commented Nov 4, 2018

asiripanich commented Nov 5, 2018 • edited Loading

shankari commented Aug 23, 2018 •

edited

Loading

shankari commented Aug 25, 2018 •

edited

Loading

shankari commented Sep 6, 2018 •

edited

Loading

shankari commented Sep 6, 2018 •

edited

Loading

shankari commented Sep 6, 2018 •

edited

Loading

shankari commented Sep 6, 2018 •

edited

Loading

shankari commented Sep 6, 2018 •

edited

Loading

shankari commented Sep 6, 2018 •

edited

Loading

asiripanich commented Sep 9, 2018 •

edited

Loading

shankari commented Sep 11, 2018 •

edited

Loading

shankari commented Sep 11, 2018 •

edited

Loading

shankari commented Sep 11, 2018 •

edited

Loading

asiripanich commented Nov 5, 2018 •

edited

Loading