Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce initial memory usage per thread #8

Open
poef opened this issue Aug 8, 2024 · 0 comments
Open

Reduce initial memory usage per thread #8

poef opened this issue Aug 8, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@poef
Copy link
Member

poef commented Aug 8, 2024

Right now each thread gets a copy of meta.index.id, which can get quite large. This cannot be avoided, unless we make a single indexing thread, and use that for access to index information. This seem tricky and a potential bottleneck. The only other option is to try to find a more memory-friendly way to store the index.

In addition when parsing the sharedArrayBuffer, for each object a Proxy is created, which will parse the object when accessed.
It should be possible to avoid making those Proxies, untill they are accessed in most cases. E.g. when an object is in an array, the array can be turned into a proxy that only creates a Proxy for each object accessed, when it is accessed itself. Each object has itself a Proxy, which can do the same for direct links to other objects.
The main problem is that the root object contains links to most if not all other objects, so here is where the array proxy comes in. The meta.resultArray is another place where we need a proxy for the array. This proxy must have access to the position information for each specific object. So some information still gets copied around.

@poef poef added the enhancement New feature or request label Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant