Skip to content
This repository has been archived by the owner on Jun 18, 2020. It is now read-only.

Fix multinode export. #4

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

wickman
Copy link

@wickman wickman commented Jul 6, 2016

Multi-node export using --section/--totalSections is currently broken. ParallelScanExecutor creates a BitSet to track which segments are completed, and terminates when finished.cardinality() == workers.length which can only be true if totalSections is 1. If totalSections > 1, then finished.cardinality() will always be less than workers.length. While the code does eventually copy the entire table, it never terminates.

This change is sort of a hacky way to unblock me. Probably the right thing is to have ParallelScanExecutor have an expected bitset (the segments that should be completed) and a finished bitset of which ones have completed, and compares the two.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant