Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accelerate the file based storage classes #30

Open
bdezonia opened this issue May 23, 2020 · 1 comment
Open

Accelerate the file based storage classes #30

bdezonia opened this issue May 23, 2020 · 1 comment

Comments

@bdezonia
Copy link
Owner

The current file based storage is a second or 3rd generation implementation. It is much faster than it used to be. But I think it can be even faster. The code right now has one buffer it uses. We should make the code configurable so you can specify number of buffers and buffer size and then write a standalone program that tries a few inline algos (like set all values to random numbers and then sum all values), a few random access algos (like Shuffle) and at least one pathological algo (like Reverse.compute(a,a)). It should time those handful of tests and then write a driver that varies buffer size and number of buffers predictably and maybe can converge on the best combination of these things.

@bdezonia
Copy link
Owner Author

Just a note that the file based storage is now using two buffers and that greatly improved speed. But it can be much faster I think with more buffers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant