-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Performance] JSI Storage implementation POC #4492
Comments
Triggered auto assignment to @yuwenmemon ( |
Exciting!
…On Sat, Aug 7, 2021, 3:50 AM MelvinBot ***@***.***> wrote:
Triggered auto assignment to @yuwenmemon <https://github.com/yuwenmemon> (
Engineering), see https://stackoverflow.com/c/expensify/questions/4319
for more details.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#4492 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEMNUSOZCAWDVL5IOT7TO3T3UFXHANCNFSM5BXJLI2Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
Not to be a party pooper, but this feels a bit complicated. Can we maybe try to do this the simplest way possible and create a version of |
I think you should just take a look at the PR: Expensify/react-native-onyx#95 Picking a provider like this storage/index.js// index.native uses MMKV_Provider
import StorageProvider from './providers/AsyncStorageProvider';
const instance = new StorageProvider();
export default instance; And then in Onyx.js use
How is that the simplest way? Implementing my proposal allows switching to Onyx to any storage library in a matter of minutes |
If the point of this exercise is to prove that MMKV is valuable then we don't need the provider stuff for that. We could have just made
Edit: Just seeing the comment now (maybe add it to the description?) |
We've captured benchmark data, did manual tests and it seems we're not getting any benefit from the JSI MMKV storage implementation ATM In terms of "time to interactive" and "chat switch time" there's no obvious improvement What we can continue with:
|
Does everyone agree that it feels like we are missing a bottleneck that has nothing to do with storage and until we solve that we won't see the benefits of *improved storage? Proving that there is some benefit in isolation seems like it will tell us that things could be better, but we still need to fix "something" else about our Onyx design. I'm not 100% sure what @quinthar had in mind here but it sounds like maybe we want to benchmark various parts of Onyx in isolation with different scenarios to see if there are any places where MMKV pulls out ahead of |
Yep, that's exactly right. Let's start with a bare-bones benchmark that
proves Tencent is faster than AsyncStorage in *anything*, and then bit by
bit add Onyx step by step until we figure out where Onyx introduces the
bottleneck.
…On Tue, Aug 10, 2021 at 6:08 PM Marc Glasser ***@***.***> wrote:
Does everyone agree that it feels like we are missing a bottleneck that
has nothing to do with storage and until we solve that we won't see the
benefits of storage?
Proving that there is some benefit in isolation seems like it will tell us
that things *could* be better, but we still need to fix "something" else
about our Onyx design.
I'm not 100% sure what @quinthar <https://github.com/quinthar> had in
mind here
<Expensify/react-native-onyx#95 (comment)>
but it sounds like maybe we want to benchmark various parts of Onyx in
isolation with different scenarios to see if there are any places where
MMKV pulls out ahead of AsyncStorage and where there's no difference at
all. And maybe in the latter cases we can rethink the design so that it can
take advantage if MMKV's "faster storage".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4492 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEMNURYCJOFB76KQADJXMDT4HERVANCNFSM5BXJLI2Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
Yes, We've accumulated multiple clues about this. The problem doesn't seem to be storage access speed, but:
1. Relaying updatesWe have plans like fill initial state sync from cache 2. Too much scriptingThe infrastructure that Onyx creates to manage the pub/sub might be taking too much main thread time or doing something that causes React to be slow. 3. Something elseThe JSI storage had pretty much the same 2+ sec calls on the same places where AsyncStorage had those, but MMKV and JSI work differently
Ok I'm down for that, but it's still undecided how exactly we should go about this, I've already proposed the standalone app, but no one commented on that and so it's probably not the way to go
If Onyx is used in a isolated environment - at a small scale - it does not exhibit any problems. One of my past proposals was to strip everything from the Authenticated app and start to put it back piece by piece and see how components affect TTI and interactions |
The only thing related to the current ticket is standalone benchmarking MMKV |
Just speculating - but I wonder if the At the moment, a key change updates a value immediately instead of when it's needed. There are probably numerous places (mainly in the non React code but also in React components) where a store getter could replace a subscriber that updates a local variable. e.g. if we were working with redux there are cases for Not too sure how to go about testing this. I think it might require an audit of all
I agree this does seem unnecessary. I'd also kind of expect it would happen very fast unless there were thousands of connections, but not too sure.
This feels really likely. Especially since we are not deferring any writes either. We try to read, write, and display UI all at once when the app inits. I wonder if it would help to take a less aggressive strategy when it comes to writing to storage? We've looked at batching reads - but what about deferring any writes until we're done reading and building critical UI? Seems like the work of writing could be batched/throttled by some amount and perhaps that will free the main thread up to do other things? |
I've captured more data regarding |
Some conversation has been happening here. If you want to create a new ticket with what you are thinking we can link it to that one. |
Did a bit of basic testing with MMKV. It seems quite a bit faster than AsyncStorage for simple operations but only somewhat better (100ms or so improvement) for when you need to do a lot of stuff (e.g. read and write a ton of keys at once like we do when the app inits). Here's the rough test I tried... I exported my entire The results were kind of surprising in that while Not sure how relevant this is for us since we are using |
@kidroca Eep! 4 days overdue now. Issues have feelings too... |
Yes, I've also observed that writing with AsyncStorage tends to be slower, merge and multiset as well Something to consider with MMKV are the sync methods - these block the main thread, though it might not be an issue as we're writing to memory A pro with MMKV is we would be able to drop some or all of the caching logic
I've tracked I've applied refactoring, where we store connections mapped by key, which allows to iterate less items, but it didn't seem to help much I've also explored whether the setState calls in
So far it looks like we know MMKV is faster general, but Onyx with MMKV didn't result in any noticeable improvement One thing left is to explore is rewriting |
Kind of crazy idea, what if we take the opposite approach and make a fake
storage engine that is entirely in javascript, avoiding the bridge totally.
It would just have a large Json object that it is initialized with, and
then when requests are made to it it just looks up in response with that.
This would eliminate the bridge and the storage engine entirely from
testing, and would simply evaluate the overhead of Onyx itself, and all of
the promises and so forth.
It could be that all of the time lost is actually just in the use of our
promises, react components, etc.
…On Wed, Aug 18, 2021, 4:43 PM Peter Velkov ***@***.***> wrote:
Yes, I've also observed that writing with AsyncStorage tends to be slower,
merge and multiset as well
Something to note is that even though a write can be taking more time, the
call is not blocking the UI
Something to consider with MMKV are the sync methods - these block the
main thread, though it might not be an issue as we're writing to memory
A pro with MMKV is we would be able to drop some or all of the caching
logic
- we would no longer need to capture tasks as calls resolve immediately
- we might not capture cache for the same reason, though then we'd
need to JSON.parse every time we read a key
------------------------------
We've already discovered and addressed the issue with "too many
connections for the same key", but another thing that happens often is the
keyChanged call (might actually be the most frequently called func) - it
iterates the full list of connections instead of just the connections for
the key that changed - that seems so plausible and unnecessary that I'll
just open a ticket
I've tracked keyChanged calls during app init and report switch
It turned out they aren't happening as often as I thought - about 20-25
for init and 8-10 for chat switching
The ones during init averaged a slower time of about 16ms per call,
there's always 1 slow call of about 150ms (on a DEV build though), while
for chat switches time never raised above 5ms
I've applied refactoring, where we store connections mapped by key, which
allows to iterate less items, but it didn't seem to help much
I've also explored whether the setState calls in key(s)Changed aren't
delaying things as they use the callback version, but they aren't
------------------------------
Yep, that's exactly right. Let's start with a bare-bones benchmark that
proves Tencent is faster than AsyncStorage in *anything*, and then bit by
bit add Onyx step by step until we figure out where Onyx introduces the
bottleneck.
So far it looks like we know MMKV is faster general, but Onyx with MMKV
didn't result in any noticeable improvement
If Onyx has a problem it's not tied to the underlying storage
implementation but how it's used and how updates are translated to React
One thing left is to explore is rewriting withOnyx to use an aggregated
storage provider
Another option is identifying other bottlenecks during app init
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4492 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEMNUT2375NND6454RGRTLT5RAQ7ANCNFSM5BXJLI2Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
|
This is pretty much the cache object that we currently have This experiment we would need to initialize cache from storage. And this would fall to the batching and the multi-get category optimizations that we wanted to avoid in the first place (for the current issue scope) I don't think the experiment would reveal anything that we don't already know.
|
We've made some improvements to App and Onyx and we've also have a new more consistent benchmark process so it might be worth it to redo the JSI benchmark |
Nothing new to report here, I need some internal feedback to proceed |
@kidroca Uh oh! This issue is overdue by 2 days. Don't forget to update your issues! |
@kidroca 6 days overdue. This is scarier than being forced to listen to Vogon poetry! |
@marcaaron In the mean time I've worked on file handling in Onyx which uses the same storage provider concept to deliver separate handling between native and web - that work will allow us to add the MMKV storage whenever we want if we decide to |
I like the sound of this. Having a synchronous I/O sounds neat, but also seems like a big change to Onyx for unestablished benefit. Feel free to get more other opinions by pitching these ideas in the Slack channel. Perhaps some others will have ideas on whether this issue is worth keeping open or how to proceed. |
@kidroca Eep! 4 days overdue now. Issues have feelings too... |
@kidroca 6 days overdue. This is scarier than being forced to listen to Vogon poetry! |
@kidroca 8 days overdue is a lot. Should this be a Weekly issue? If so, feel free to change it! |
Dropping this to weekly as it doesn't seem like there is a clear consensus how to proceed or close. |
I've posted on slack about this ticket, but it didn't get any attention. |
Yes can you please bump that thread? Android performance is light years
beyond what it was before, which was entirely unusable. But it's still not
necessarily fast.
…On Sat, Sep 11, 2021, 10:26 AM Peter Velkov ***@***.***> wrote:
I've posted on slack about this ticket, but it didn't get any attention.
I'll try again next week
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4492 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEMNUUW5FDPVM6QHMGKVZ3UBOGNRANCNFSM5BXJLI2Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
I thought the idea was to test how much MMKV and JSI would improve things. The work done here is handy as it allows us to easily swap and try different storage solutions and we're already applying the concept for file handling #3867. I thought the bottleneck was storage and particularly the react-native bridge, but even when we skipped it entirely the performance is just slightly faster. Overall my opinion is that the solution to Android performance is not in the storage library that we use. We have other paths to explore let's do that and come back here It's seems we don't have a ticket dedicated on Android performance, we have a few [Performance] tickets that cover all platforms (e.g. multiget and batching). I can go thought the suggestions that we didn't try here and post them on slack so we can create tickets for the ones that do pickup ...
|
I like the idea of summarizing what outstanding ideas we haven't explored.
Also, what's the latest on the big core react native upgrade to (I think?)
deprecate the bridge and go to JSI for everything?
…On Mon, Sep 13, 2021, 7:08 AM Peter Velkov ***@***.***> wrote:
I thought the idea was to test how much MMKV and JSI would improve things.
In terms of TTI there were no improvements, no noticeable improvement for
chat switches as well.
So we started to look elsewhere
Since the benchmark here, there were some updates that improved the app as
you've noticed, IMO the ticket can continue to explore another storage
library or re-run the benchmarks to see if something changed on that front
The work done here is handy as it allows us to easily swap and try
different storage solutions and we're applying the concept for file
handling #3867 <#3867>.
We can relatively easy try different native storage libraries if we want
to explore that path.
------------------------------
I thought the bottleneck was storage and particularly the react-native
*bridge*, but even when we skipped it entirely the performance is just
slightly faster.
Overall my opinion is that the solution to Android performance is not in
the storage library that we use. We have other paths to explore let's do
that and come back here
------------------------------
It's seems we don't have a ticket dedicated on Android performance, we
have a few [Performance] tickets that cover all platforms (e.g. multiget
and batching).
We can open a separate ticket for Android, but wouldn't it be better to
handle "leads" in slack and then open specific tickets (Pretty much like
how this ticket came to existence)
I can go thought the suggestions that we didn't try here and post them on
slack so we can create tickets for the ones that do pickup
...
Yes can you please bump that thread? Android performance is light years
beyond what it was before, which was entirely unusable. But it's still not
necessarily fast.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4492 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEMNUR7XSZJWHH44SEOB73UBYAVHANCNFSM5BXJLI2Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
This issue has not been updated in over 15 days. @kidroca eroding to Monthly issue. P.S. Is everyone reading this sure this is really a near-term priority? Be brave: if you disagree, go ahead and close it out. If someone disagrees, they'll reopen it, and if they don't: one less thing to do! |
@kidroca, this Monthly task hasn't been acted upon in 6 weeks; closing. If you disagree, feel encouraged to reopen it -- but pick your least important issue to close instead. |
If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!
This issue was created as a result of the conversation here: https://expensify.slack.com/archives/C01GTK53T8Q/p1628181643026200?thread_ts=1628106063.148700&cid=C01GTK53T8Q
What performance issue do we need to solve?
What is the impact of this on end-users?
List any benchmarks that show the severity of the issue
Setup
Flows
Init - relaunch the app and use
Onyx.printMetrics
after everything is renderedChat switch - go back to LHN, use
Onyx.resetMetrics
now open a different Chat. UseOnyx.printMetrics
after everything is renderedResults Android (dev)
Proposed solution (if any)
Create a Proof Of Concept by switching Onyx to use a JSI storage library like
1. Extract a storage provider interface from the existing AsyncStorage usages in Onyx
Onyx interface with the native storage library is pretty simple get, set ,merge etc
We can make Onyx work with different providers of this interface
2. Work with different storage providers
To make different storage providers like AsyncStorage or MMKV work with Onyx we make a thin wrapper
implementing the storage interface in
provider.native.js
orprovider.web.js
- whatever is neededThen in Onyx we
import provider from './provider'
and use the methods from the interface - get, set, mergeList any benchmarks after implementing the changes to show impacts of the proposed solution (if any)
Note: These should be the same as the benchmarks collected before any changes.
Platform:
Where is this issue occurring?
Version Number: 1.0.83-0
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
Expensify/Expensify Issue URL:
View all open jobs on Upwork
The text was updated successfully, but these errors were encountered: