Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Issues with Sampling Start and Cap #431

Open
westernguy2 opened this issue Nov 10, 2021 · 0 comments
Open

[BUG] Issues with Sampling Start and Cap #431

westernguy2 opened this issue Nov 10, 2021 · 0 comments

Comments

@westernguy2
Copy link
Contributor

Describe the bug
DataFrames that are of size slightly below the sampling cap will end up having a larger sample of the data than DataFrames that are slightly above the sampling cap. This shouldn't be happening, as the size of the sample of the DataFrame should grow with the size of the DataFrame.
To Reproduce
The description of the bug is a clear explanation to reproduce the bug.

Expected behavior
This is also described in the bug description above.
westernguy2 added a commit to westernguy2/lux that referenced this issue Nov 10, 2021
@westernguy2
FIX-lux-org#431: implement sampling threshold and edit tests and docs
f4441c5 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
@westernguy2 westernguy2 mentioned this issue Nov 10, 2021
Open
westernguy2 added a commit to westernguy2/lux that referenced this issue Nov 28, 2021
@westernguy2
FIX-lux-org#431: small cleanup changes
6c82a88 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Nov 28, 2021
@westernguy2
FIX-lux-org#431: move sampling to after the filtering
75ab18f 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Nov 29, 2021
@westernguy2
FIX-lux-org#431: add sampling for metadata statistics computation
6b224a0 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Nov 29, 2021
@westernguy2
FIX-lux-org#431: fix cardinality bug
6ea34a7 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Nov 30, 2021
@westernguy2
FIX-lux-org#431: remove print statement
c26295d 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Dec 1, 2021
@westernguy2
FIX-lux-org#431: fix bug in check_id_like
caf1fe1 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Dec 2, 2021
@westernguy2
FIX-lux-org#431: remove filter_executed dictionary implementation
222037a 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Jan 31, 2022
@westernguy2
FIX-lux-org#431: fix bug with caching sampled df
c696b2b 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Feb 2, 2022
@westernguy2
FIX-lux-org#431: Move column filtering after sampling
35bd2d2 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
westernguy2 added a commit to westernguy2/lux that referenced this issue Mar 8, 2022
@westernguy2
FIX-lux-org#431: changed sampling method to removing rows
5252d91 
Signed-off-by: Kunal Agarwal <kagarwal2@berkeley.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@westernguy2