Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use JobConf input files list for input size computation used by ReducersEstimators #1644

Merged
merged 1 commit into from
Feb 23, 2017

Conversation

dieu
Copy link
Contributor

@dieu dieu commented Feb 22, 2017

Fix for bug in InputSizeReduceEstimator when he calculates a size of all versions for VersionedKeyValSource instead using only one version.

adopted approach from #1428

@dieu dieu force-pushed the apanasenko/reduce_estimator_vkvs branch from 5e41f89 to 54820db Compare February 22, 2017 21:40
@dieu dieu requested a review from piyushnarang February 22, 2017 21:41
@dieu dieu force-pushed the apanasenko/reduce_estimator_vkvs branch from 54820db to 1a42b5a Compare February 23, 2017 01:02
@dieu dieu changed the title Checking size in tap first and then calculate by ourselves Use JobConf input files list for input size computation used by ReducersEstimators Feb 23, 2017
Copy link
Collaborator

@johnynek johnynek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@dieu dieu merged commit 34cc46c into twitter:develop Feb 23, 2017
Copy link
Contributor

@gerashegalov gerashegalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants