Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client assignment can miss a (small) number of docs #344

Closed
danielmitterdorfer opened this issue Nov 7, 2017 · 0 comments
Closed

Client assignment can miss a (small) number of docs #344

danielmitterdorfer opened this issue Nov 7, 2017 · 0 comments
Labels
bug Something's wrong :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc.
Milestone

Comments

@danielmitterdorfer
Copy link
Member

If the total document count is not a multiple of the number of clients, Rally currently misses a few documents. This will not affect the correctness of a benchmark as the number of documents should usually be very high (in the millions) but it is incorrect nevertheless.

Consider, the following scenario: We have 16000 documents among 12 clients. In that case, each client needs to index 1333.3333 documents. Currently, each client will index 1333 documents and the last one will index 1332 (due to another bug). This gives: 1333*11 + 1332 = 15595 documents which means we miss 5 documents.

Instead, we need to assign documents per client a little bit differently: two out of three clients will need to index 1333 documents and the third one will index one document more, i.e. 1334 documents. 8 (clients) * 1333 (documents) + 4 (clients) * 1334 (documents) = 16000 documents.

This new assignment rule will ensure that we index all documents but load is still spread almost evenly across all clients.

@danielmitterdorfer danielmitterdorfer added :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc. bug Something's wrong labels Nov 7, 2017
@danielmitterdorfer danielmitterdorfer added this to the 0.7.4 milestone Nov 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something's wrong :Load Driver Changes that affect the core of the load driver such as scheduling, the measurement approach etc.
Projects
None yet
Development

No branches or pull requests

1 participant