Add support for the V3 scheduler; drop support for V1 #11808

dae · 2022-07-09T09:00:19Z

Will compile and pass legacy schema tests without a backend update, so this can be merged before updating. It does require backend changes to pass the tests when legacy_schema is false.

Users still on V1 will be prompted to update when they tap on a deck. The update uses the backend code so that learning cards are not lost.
By default, AnkiDroid will continue to use V2 with a V3 collection. The V3 path is only enabled when legacy_schema=false
Tests pass, and in my brief tests in the emulator, reviewing/undoing seems to work.

I think this closes #7127, due to the backend code being much faster.
Closes #10694
Closes #10411

mikehardy · 2022-07-09T10:00:00Z

🤯 😄

mikehardy · 2022-07-22T00:08:35Z

Ready for rebase and hopefully I'll have time for a review tomorrow (nearly dinner time here in New Orleans 😁)

dae · 2022-07-22T00:45:08Z

Have confirmed tests pass in each commit locally.
Have moved the commit from Use backend to calculate legacy timezone #11881 onto the tail of this one, so the final commit switches AD over to the latest backend. Can be kept in a separate PR if you'd prefer.

dae · 2022-07-22T04:56:41Z

(made a trivial change to re-trigger the flaky emulator test)

Edit: new failure appears to have been introduced in the move to the latest backend. Investigating. I didn't catch this in local tests because I had an older API emulator active.

Edit 2: should be fixed now (though this code will presumably be dropped in favour of the backend importer anyway)

mikehardy

I only had enough time to look at the first commit 😩 - but had a couple things worth posting about it now

AnkiDroid/src/main/java/com/ichi2/anki/DeckPicker.kt

mikehardy · 2022-07-22T21:45:45Z

AnkiDroid/src/main/java/com/ichi2/anki/CoroutineHelpers.kt

+    // TODO: add translatable string
+    message: String = "Processing...",


I'm concerned about both this string and relying on new backend's tr method below, we haven't (I haven't?) fully analyzed translation language sets to see how compatible they are etc, and I don't have the time to do that right now. I fear it'll hang up this PR unless we just use AnkiDroid-level translation strings for the moment or we (where "we" is someone other than me at the moment, and through mid-August) can harmonize the languages and update AnkiDroid docs to point at Anki's translation service as well for people that want to translate non-crowdin strings etc etc and kick off that whole process

For the "processing..." string, I figured you'd want to do that in a follow-up commit after an i18n sync.

I've gone through and checked the languages mappings, and they should all be ok now. For any languages that the backend doesn't currently support, it will just fall back on the English strings like if it hadn't been translated yet. New languages can be added when users follow the instructions on https://translating.ankiweb.net/

For the "processing..." string, I figured you'd want to do that in a follow-up commit after an i18n sync.

Historically, user-visible strings are always added into strings file at time of first usage, and I just label the PR with the strings label so I remember to do the sync action / PR after - we never let user-visible stuff through with hard-coded strings in PRs, or at least not intentionally

I've gone through and checked the languages mappings, and they should all be ok now.

Thank you!

For any languages that the backend doesn't currently support, it will just fall back on the English strings like if it hadn't been translated yet. New languages can be added when users follow the instructions on https://translating.ankiweb.net/

I think this is a pretty solid start - and I am biased towards action, so let's go with it. I guess all that's left is an edit to the translation page to give people a handle to the new-to-us system you've got with Pontoon - I just did that https://github.com/ankidroid/Anki-Android/wiki/Translating-AnkiDroid/_compare/7c0ba5c393040654691f8ed44d2d44c3617d845e...002935cd2bc07effc17b8f3a409a846bde79e2be

String added

mikehardy · 2022-07-24T22:14:54Z

AnkiDroid/src/main/java/com/ichi2/anki/CollectionHelper.java

        // Open collection
+        Context context = AnkiDroidApp.getInstance();


nice one - thanks - hopefully avoids the semi-false positives I was seeing with LeakCanary which were making me think things were crashing

AnkiDroid/src/androidTest/java/com/ichi2/anki/tests/ContentProviderTest.java

mikehardy

This all seems great to me - nothing seems to jump out and be obviously off. Just need the user-visible string extracted to something (03-dialogs maybe?) and I'm +1. V3 scheduler support, this is huge

mikehardy · 2022-07-24T22:18:55Z

Oh, and it took a conflict but I am assuming (hopefully not naively) that it'll be trivial

mikehardy · 2022-07-24T22:20:59Z

All other reviewers: I really want to maintain momentum in this connected chain of anki/Anki-Android-Backend/Anki-Android work, and it threatens to stall all the time because of my poor availability and the pushing it needs between the components. So I'd like to set a 1-2 day timer on this one before I merge by default. If you want to review and need more time just say so! No problem. Otherwise if it's quiet on this PR I'm going to merge it as soon as the string + de-conflict are in

Studying with the V1 scheduler enabled is no longer possible on recent Anki, AnkiMobile or AnkiWeb versions, so AnkiDroid is the last one still supporting it. Pushing users to upgrade will save them from some of the footguns V1 had, and will allow AnkiDroid to cut out some code and tests. Many users have likely already upgraded due to the use of the other clients. This commit adds support for the backend upgrade code, so that learning cards will not be reset on upgrade. To make use of this, users will be automatically updated to the latest schema version, the scheduler upgrade will be performed, and then they're moved back to the legacy schema. I've also added a helper to more ergonomically deal with schema changes.

- Mirrors the code layout of the desktop version, where routines like buryCard(), that are not specific to V2 or V3, are stored in a separate file. This leaves AbstractSched containing only the methods that will need to be implemented separately for V3. - Moves some methods from SchedV2 into BaseSched, as V3 will want to use them as well for now. - Some of the routines in BaseSched will only work with a schema upgrade. For now, SchedV2 overrides these so they continue to work with the legacy schema.

Like the desktop, this works by using a single set of tests that alter some of the checks depending on the active scheduler version, so that code duplication is avoided.

I expect this fixes ankidroid#9368 Bumps the backend version, since the API changed. Closes ankidroid#11881

- Move logic from backend into frontend, so a new backend release is not required - Fix mapping of Chinese languages and Hindi; check others - Don't require a restart after changing language in prefs

After discarding the backend, Preferences was calling getCol and passing itself in as the context. This caused the preferences screen to be retained as the backend lives on. By switching to AnkiDroidApp, a leak can be avoided.

Some were irrelevant; the ones in ContentProviderTest need investigating.

mikehardy

timer expired + string extracted == let's go!

SnipeAT · 2022-08-02T07:18:18Z

This is a feature I've been checking daily for. Will it will show up in the next alpha?

dae · 2022-08-02T08:01:14Z

AFAIK it's already in the latest alpha, but it requires opting in to the new backend, which is a big change that has not received any user testing so far, so you may be exposing yourself to bugs in unrelated functionality. If you're happy to live life on the edge and make lots of backups, you can opt in for a session by tapping on the V16 option in the developer options in the preferences screen. Please report any issues you encounter.

mikehardy · 2022-08-02T15:24:33Z

@dae unfortunately this is not 100% accurate:

the V16 option in the developer options in the preferences screen.

That "developer options" screen is dynamically added only to "debug" builds, that is when you build from source. So it's not generally accessible to users yet.

The next step in progression of V16+ schema testing is to make that specific option a toggle with state stored/used in/from preferences, and to move it to advanced menu. I'd love to see that soon as folks like @SnipeAT are (assuming they understand the risk) a really really really valuable source of testing since they are motivated to try it. Whatever we can do to get to this state where end users can get a warning but then try V16 would be a big step forward IMHO

BrayanDSO · 2022-08-06T00:07:25Z

@mikehardy, actually with the new About screen, users on RELEASE build can use developer options as well.

Tap the logo 6 times and a big warning will show asking if the user wants to enable it or not.
(6 is the number of letters of my name, now you won't forget it 😉)

And FWIW, I'm in favor of out adding the V16 option to the alpha/beta releases on Advanced, so anyone can find them.

SnipeAT · 2022-08-16T21:44:14Z

Just wanted to report that I've been using the new backend and the v3 scheduler for a week or so and it's been working out pretty great. The app might crash in a couple situations but reports seem to be being sent back home. Just updated to the alpha released today and will continue using (after making another backup of my collection).

dae · 2022-08-16T22:21:13Z

Thanks for letting us know! Please note that there's currently a bug where review times are being saved incorrectly. Until the next alpha, I recommend you set your maximum answer time in the deck options to your average answer time, and/or adjust the previous review records using something like this in the debug console of the desktop version (5000 is 5 seconds), which will reset 60s answers in the last month:

mw.col.modSchema(check=True)
mw.col.db.execute("update revlog set time = 5000 where time = 60000 and id > 1658096427000")

https://docs.ankiweb.net/misc.html#debug-console

#12024

Also the bug reports are kind of hard to search, so if you are able to reproduce a bug, it would be helpful if you could describe the steps you're taking on a ticket here, and optionally include the acra ID if it's not easy to reproduce.

BrayanDSO · 2022-08-16T23:01:40Z

complementing: You get the Acra ID together with the debug info on Settings > About > Copy Debug Info

mikehardy · 2022-08-23T18:44:21Z

@dae follow-up on this one: I attempted to document it here for early testers, did I get all the information correct? #10411 (comment)

Secondly, seems we may need an "enable v3 scheduler" switch? Or do we have one already and I missed it? That's my main doubt but I could be wrong on any of it of course

BrayanDSO · 2022-08-23T20:14:21Z

We don't have a v3 Scheduler switch. I have a draft branch for it here https://github.com/BrayanDSO/Anki-Android/tree/v3pref, but it has a bug that the preference is always re-enabled, both here and Anki desktop, after sync

mikehardy · 2022-08-23T22:26:00Z

No worries - it will be needed in the future yes - but right now as a way to feed new testers into new backend / v3, having it be a requirement that the collection is v3-enabled on desktop perhaps feeds them in at a measured pace. Not a bad thing - I'm not looking for ease of use, just the possibility of use, and real testing. I think our current state fits that

dae · 2022-08-24T08:48:08Z

LGTM. There's a bug where the scheduler version is not updated after a sync (will fix), so recommend you tell people to restart the app after they have synced the setting from the desktop.

mikehardy · 2022-08-24T15:07:57Z

@dae thanks for the review, I updated the comment with a link the PR you just posted for the same. I see a fair bit of traffic today on V3/new-backend related items but it doesn't seem overwhelming? Do you get the same feeling? Like, there are issues but it is at capacity and not over-capacity? I ask because I'm gradually widening my audience where I publicize this and right now it feels like "no more new testers for a few days" but maybe by end of week I can start mentioning it's available again. Perhaps in a week or so we can make a real call for testers but that's not a decision now

dae · 2022-08-25T06:13:53Z

#12157 should address the crashes I'm aware of. No particular preference on timing on my end.

dae force-pushed the v3 branch from d1e19af to accb53b Compare July 9, 2022 09:17

dae force-pushed the v3 branch from accb53b to 3523d3d Compare July 11, 2022 05:49

This was referenced Jul 13, 2022

Like AnkiDesktop, do not reset the interval when rescheduling a card. #11275

Closed

Introduce CollectionManager for preventing concurrent access #11849

Merged

Use backend to calculate legacy timezone #11881

Closed

mikehardy mentioned this pull request Jul 19, 2022

More backend integration work (undo, progress in GUI, cancellation, refactoring) #11740

Merged

dae force-pushed the v3 branch from 3523d3d to 4075b5b Compare July 20, 2022 01:59

dae marked this pull request as ready for review July 20, 2022 02:01

dae force-pushed the v3 branch from 4075b5b to 092df5c Compare July 22, 2022 00:43

dae force-pushed the v3 branch from 092df5c to 94d0a4d Compare July 22, 2022 04:56

dae force-pushed the v3 branch from 94d0a4d to 2859c06 Compare July 22, 2022 06:32

mikehardy mentioned this pull request Jul 22, 2022

Counts in background #7127

Closed

5 tasks

mikehardy added Review High Priority Request for high priority review Needs Review labels Jul 22, 2022

mikehardy reviewed Jul 22, 2022

View reviewed changes

dae force-pushed the v3 branch from 2859c06 to f969c0e Compare July 23, 2022 05:50

mikehardy reviewed Jul 24, 2022

View reviewed changes

AnkiDroid/src/androidTest/java/com/ichi2/anki/tests/ContentProviderTest.java Show resolved Hide resolved

mikehardy requested changes Jul 24, 2022

View reviewed changes

mikehardy added Needs Author Reply Waiting for a reply from the original author Needs Second Approval Has one approval, one more approval to merge and removed Needs Review labels Jul 24, 2022

dae force-pushed the v3 branch from f969c0e to db67460 Compare July 24, 2022 23:24

dae added 7 commits July 25, 2022 09:28

Add support for the V3 scheduler

84cee17

Add V3 scheduler tests

7f44dc4

Like the desktop, this works by using a single set of tests that alter some of the checks depending on the active scheduler version, so that code duplication is avoided.

Use backend to calculate legacy timezone

87de70e

I expect this fixes ankidroid#9368 Bumps the backend version, since the API changed. Closes ankidroid#11881

Backend language tweaks

79732a9

- Move logic from backend into frontend, so a new backend release is not required - Fix mapping of Chinese languages and Hindi; check others - Don't require a restart after changing language in prefs

Fix LeakCanary warning when changing language/v16 schema

40063f8

After discarding the backend, Preferences was calling getCol and passing itself in as the context. This caused the preferences screen to be retained as the backend lives on. By switching to AnkiDroidApp, a leak can be avoided.

Fix some connected tests failing when legacy_schema=false

a590936

Some were irrelevant; the ones in ContentProviderTest need investigating.

dae force-pushed the v3 branch from db67460 to a590936 Compare July 24, 2022 23:28

mikehardy approved these changes Jul 27, 2022

View reviewed changes

mikehardy merged commit d02093d into ankidroid:main Jul 27, 2022

github-actions bot added this to the 2.16 release milestone Jul 27, 2022

github-actions bot removed Review High Priority Request for high priority review Needs Author Reply Waiting for a reply from the original author Needs Second Approval Has one approval, one more approval to merge labels Jul 27, 2022

dae deleted the v3 branch July 28, 2022 04:20

dae mentioned this pull request Aug 5, 2022

[GSoC] Add Use new backend (persistent) preference #11977

Merged

7 tasks

dae mentioned this pull request Sep 4, 2022

refactor: Convert Sched to Kotlin #12301

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for the V3 scheduler; drop support for V1 #11808

Add support for the V3 scheduler; drop support for V1 #11808

dae commented Jul 9, 2022 •

edited

Loading

mikehardy commented Jul 9, 2022

mikehardy commented Jul 22, 2022

dae commented Jul 22, 2022

dae commented Jul 22, 2022 •

edited

Loading

mikehardy left a comment

mikehardy Jul 22, 2022

dae Jul 23, 2022 •

edited

Loading

mikehardy Jul 24, 2022

dae Jul 24, 2022

mikehardy Jul 24, 2022

mikehardy left a comment

mikehardy commented Jul 24, 2022

mikehardy commented Jul 24, 2022

mikehardy left a comment

SnipeAT commented Aug 2, 2022

dae commented Aug 2, 2022

mikehardy commented Aug 2, 2022

BrayanDSO commented Aug 6, 2022 •

edited

Loading

SnipeAT commented Aug 16, 2022

dae commented Aug 16, 2022 •

edited

Loading

BrayanDSO commented Aug 16, 2022

mikehardy commented Aug 23, 2022

BrayanDSO commented Aug 23, 2022

mikehardy commented Aug 23, 2022

dae commented Aug 24, 2022

mikehardy commented Aug 24, 2022

dae commented Aug 25, 2022

		// TODO: add translatable string
		message: String = "Processing...",

		// Open collection
		Context context = AnkiDroidApp.getInstance();

Add support for the V3 scheduler; drop support for V1 #11808

Add support for the V3 scheduler; drop support for V1 #11808

Conversation

dae commented Jul 9, 2022 • edited Loading

mikehardy commented Jul 9, 2022

mikehardy commented Jul 22, 2022

dae commented Jul 22, 2022

dae commented Jul 22, 2022 • edited Loading

mikehardy left a comment

Choose a reason for hiding this comment

mikehardy Jul 22, 2022

Choose a reason for hiding this comment

dae Jul 23, 2022 • edited Loading

Choose a reason for hiding this comment

mikehardy Jul 24, 2022

Choose a reason for hiding this comment

dae Jul 24, 2022

Choose a reason for hiding this comment

mikehardy Jul 24, 2022

Choose a reason for hiding this comment

mikehardy left a comment

Choose a reason for hiding this comment

mikehardy commented Jul 24, 2022

mikehardy commented Jul 24, 2022

mikehardy left a comment

Choose a reason for hiding this comment

SnipeAT commented Aug 2, 2022

dae commented Aug 2, 2022

mikehardy commented Aug 2, 2022

BrayanDSO commented Aug 6, 2022 • edited Loading

SnipeAT commented Aug 16, 2022

dae commented Aug 16, 2022 • edited Loading

BrayanDSO commented Aug 16, 2022

mikehardy commented Aug 23, 2022

BrayanDSO commented Aug 23, 2022

mikehardy commented Aug 23, 2022

dae commented Aug 24, 2022

mikehardy commented Aug 24, 2022

dae commented Aug 25, 2022

dae commented Jul 9, 2022 •

edited

Loading

dae commented Jul 22, 2022 •

edited

Loading

dae Jul 23, 2022 •

edited

Loading

BrayanDSO commented Aug 6, 2022 •

edited

Loading

dae commented Aug 16, 2022 •

edited

Loading