-
-
Notifications
You must be signed in to change notification settings - Fork 307
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
r.neighbors: implement parallelization with OpenMP #1724
Conversation
343afee
to
6abf3af
Compare
I was running the following test in NC SPM sample location (2,025,000 cells):
I repeated each run 30 times and used different machines with up to 32 cores (on HPC). Here are tests for 8 and 32 cores (results were the same or similar for all the machines and max nprocs I used): Memory is the version in this PR. Segment and File are the two previous versions of parallel r.neighbors using the segment library and r.mfilter-like temporary files, respectively. We discussed these results and the advantages and disadvantages of each approach during our GSoC meetings, so this just for the record. |
17a7469
to
91c07ee
Compare
91c07ee
to
d76faa5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good. Just some smaller issues.
for (row = start - ncb.dist; row < start + ncb.dist; row++)
at 512 could be for (row = 0; row < 2 * ncb.dist; row++)
, but not needed.
This is a rework upon the revision to the first implementation of r.neighbors (#1654). It avoids using temporary file buffer to reduce unnecessary disk I/O which is not ideal in some systems. Instead, it allows the users to specify a parameter "memory" to indicate the amount of memory to be used for the output buffer. Furthermore, it exposes a "nprocs" parameter to allow users the amount of processing units to be used for computation. A benchmark script is also included under r.neighbors/benchmark to showcase how the performance scales with "nprocs", using 4 generated raster maps of size 50M, 100M, 200M and 400M respectively.
Checklists before merging: