Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync #1 #4

Merged
merged 172 commits into from
Feb 18, 2021
Merged

Sync #1 #4

merged 172 commits into from
Feb 18, 2021

Conversation

aregm
Copy link
Owner

@aregm aregm commented Feb 18, 2021

What do these changes do?

  • commit message follows format outlined here
  • passes flake8 modin
  • passes black --check modin
  • signed commit with git commit -s
  • Resolves #?
  • tests added and passing

anmyachev and others added 30 commits October 20, 2020 16:36
@anmyachev
FIX-#2195: fix describe error for datasets with datetimes (#2272)
3fd16a9 
* FIX-#2195: fix describe error for datasets with datetimes

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* FIX-#2195: add test

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* FIX-#2195: enable fix

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* FIX-#2195: Update modin/pandas/test/dataframe/test_reduction.py

Co-authored-by: Dmitry Chigarev <62142979+dchigarev@users.noreply.github.com>

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@dchigarev
FIX-#1906: fixed incorrect behaviour of 'groupby.__getattr' (#2276)
c680077 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
FIX-#2277: applied Title Case to the names of DATASET_SIZE_DICT keys (#…
39d69ab 
…2278)

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@anmyachev
FIX-#2280: use 32 bytes in secrets.token_hex (#2286)
544de0d 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
TEST-#2260: use recommended pandas testing api (#2273)
b514d6f 
* TEST-#2260: use recommended pandas testing api

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2260: replace getSeriesData with test_data

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2260: remove assert_categories_equal

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@dchigarev
FIX-#2254: handling dict functions at groupby.agg improved (#2267)
64b94f5 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@ienkovich
FEAT-#2282: support DataFrame.[count|max|min|sum] for OmniSci backend (…
6697e05 
…#2283)

Signed-off-by: ienkovich <ilya.enkovich@intel.com>
@dchigarev
FIX-#1976: indices matching at reduction functions fixed (#2270)
55e3459 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@ienkovich
FEAT-#2299: support value_counts in OmniSci backend. (#2300)
60d4956 
Signed-off-by: ienkovich <ilya.enkovich@intel.com>
@prutskov
FIX-#1765: Fix support of s3 in read_parquet (#2287)
a0a3ebb 
Signed-off-by: Alexey Prutskov <alexey.prutskov@intel.com>
@dchigarev
FIX-#2285: Default to pandas warning message improved (#2302)
e63c042 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@ienkovich
FEAT-#2303: fix OmniSci aggregates and add mean (#2304)
e685b8d 
Signed-off-by: ienkovich <ilya.enkovich@intel.com>
@anmyachev
FIX-#2258: return 'Commit Message formatting' topic (#2306)
a571e10 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@prutskov
FIX-#2133 #2265: Fix binary operations for modin frames in case when …
8866ca8 
…partitioning isn't aligned (#2256)

Signed-off-by: Alexey Prutskov <alexey.prutskov@intel.com>
@devin-petersohn
FIX-#2239: Compute row index start using pandas (#2240)
a7d3093 
* FIX-#2239: Compute row index start using pandas

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>

* FIX-#2239: Documentation

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>

* FIX-#2239: Improve testing for case

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
@dchigarev
FIX-#2253: loc assignment fixed in case of (1, 1) shape frame (#2316)
5c5a8e0 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
FIX-#2311: fixed performance bottleneck at reduction operations (#2314)
5cabeb9 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@amyskov
TEST-#2288: Cover by tests delimiters parameters of read_csv (#2310)
ed69517 
Signed-off-by: Alexander Myskov <alexander.myskov@intel.com>
@anmyachev
FIX-#2234: update dask_deps in setup.py (#2325)
23b4408 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
FIX-#2326: move s3fs import in _read function (#2327)
e2f628c 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
FIX-#2329: TypeError while creating cluster (#2330)
9d3ead8 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@devin-petersohn
FIX-#0000: Indexing regression (#2333)
e08386b 
* FIX-#0000: Indexing regression

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>

* FIX-#0000: Fix `loc`

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>

* FIX-#0000: Fix DatetimeIndex

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>

* FIX-#0000: Fix Datetime and checks

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
@devin-petersohn
DOCS-#2334: Add tutorials to main repo (#2335)
f9cd119 
Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
@anmyachev
DOCS-#2193: Add contributing doc in checklist (#2216)
a11e7c9 
* DOCS-#2193: update contributing doc

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
REFACTOR-#2343: refactor offset, _read_rows, partitioned_file (#2344)
737ec34 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@YarShev
FIX-#1927: Fix performance issue related to sparse attribute access (…
c86422a 
…#2318)

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>
@YarShev @gshimansky @dchigarev
FIX-#2269: Move default_to_pandas logic from API layer to backend (#…
f1f82ee 
…2332)

* FIX-#2269: Move `default_to_pandas` logic from API layer to backend

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>

* FIX-#2269: Added a test which calls _apply_agg_function

Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

* FIX-#2269: Added required arguments for groupby_agg

Moved wrap_udf_function into backend because omnisci doesn't support
executing lambdas.

Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

* FIX-#2269: Use correct default_to_pandas for groupby in backend,
refactor default to pandas functions in BaseQC

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>

* FIX-#2269: Renamed new default_to_pandas_groupby function

into private function of Pandas backend because it is not used anywhere
else.

Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

* FIX-#2269: Fixed specification of backend

now it is possible to specify --backend=PandasOnDask,
--backend=PandasOnRay or --backend=PandasOnPython, not just
--backend=BaseOnPython.

Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

* FIX-#2269: Fix BaseOnPython tests

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>

* FIX-#2269: Remove default_to_pandas_groupby

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>

* FIX-#2269: logic of dropping 'by' moved back to API level

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>

Co-authored-by: Gregory Shimansky <gregory.shimansky@intel.com>
Co-authored-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@amyskov
TEST-#2292: Cover by tests Datetime Handling parameters of read_csv (#…
067b8ac 
…2336)

Signed-off-by: Alexander Myskov <alexander.myskov@intel.com>
@prutskov
FEAT-#2271: Add implementation of groupby.shift (#2323)
776d8e2 
Signed-off-by: Alexey Prutskov <alexey.prutskov@intel.com>
@YarShev
FIX-#2348: Fix default to pandas warnings (#2349)
5251135 
Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>
anmyachev and others added 28 commits February 4, 2021 10:36
@anmyachev
TEST-#2670: some updates to improve asv tests stability (#2671)
77d40ce 
* TEST-#2670: some updates to improve asv tests stability

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: fixes

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: data_size -> shape

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: use dict approach

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: use CpuCount when Npartitions isn't defined

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: fix ASV_DATASET_SIZE

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: update TimeSortValues

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: modify asv tests for using with old modin version

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: reply to review comments

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2670: use env variables for default values

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
TEST-#2686: add fillna benchmark (#2687)
16fa188 
* TEST-#2686: add fillna benchmark

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2686: reply to review comments

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2686: add inplace parameter

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
TEST-#2692: add drop benchmark (#2693)
0f4ce8a 
* TEST-#2692: add drop benchmark

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2692: add one column case

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@devin-petersohn
FIX-#2688: Update ray.ObjectID to ray.ObjectRef for Ray 2.0 (#2695)
5990e06 
* FIX-#2688: Update ray.ObjectID to ray.ObjectRef for Ray 2.0

Resovles #2688

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>

* FIX-#2688: Address comments

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
@dchigarev
TEST-#2707: add lint check for ASV benchmarks (#2708)
3a31683 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@anmyachev
TEST-#2699: add append benchmark (#2700)
7525f05 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@prutskov
FIX-#2684: Add method level docs for Modin XGBoost (#2685)
c637d89 
Signed-off-by: Alexey Prutskov <alexey.prutskov@intel.com>
@anmyachev
TEST-#2694: add head benchmark (#2696)
b93b879 
* TEST-#2694: add head benchmark

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2694: add small number for head op

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@dchigarev
TEST-#2705: add 'value_counts' benchmarks (#2706)
e74e012 
* TEST-#2705: add 'value_counts' benchmarks

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>

* TEST-#2705: apply suggestions from review

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
FIX-#2709: fixed typo in '_copartition' (#2710)
8a50c4a 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@YarShev @prutskov
@devin-petersohn @dchigarev
FIX-#2596: Update pandas version to 1.2.1 (#2600)
5cb3283 
Co-authored-by: Alexey Prutskov <alexey.prutskov@intel.com>
Co-authored-by: Devin Petersohn <devin.petersohn@gmail.com>
Co-authored-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
Co-authored-by: Devin Petersohn <devin-petersohn@users.noreply.github.com>
Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>
@anmyachev
TEST-#2690: add astype benchmark (#2691)
6f1fe69 
* TEST-#2690: add astype benchmark

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2690: add category dtype; use df.types

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2690: add case with one column

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
TEST-#2702: add loc/iloc benchmark (#2703)
a54875a 
* TEST-#2702: add loc/iloc benchmark

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2702: add multiindex loc bench

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* TEST-#2702: add row_loc check

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
TEST-#2716: add describe bench (#2718)
569337b 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@prutskov
DOCS-#2717: Fix version of Modin for building latest docs (#2719)
a1238a0 
Signed-off-by: Alexey Prutskov <alexey.prutskov@intel.com>
@abykovsk
FEAT-#1611: Add mod operation (#2726)
ba82360 
Signed-off-by: Alina <alina.bykovskaya@intel.com>
@anmyachev
TEST-#2725: add index, columns, shape benchmarks (#2727)
f14a3c1 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@dchigarev
FIX-#2305: fix handling of renaming aggregation (#2732)
04cd912 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
FIX-#2362: fix key handling in 'Series.__setitem__' (#2731)
a2ecf31 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@amyskov @anmyachev
TEST-#2722: add ASV read_csv skiprows benchmark (#2724)
33a57e3 
* TEST-#2722: add ASV read_csv skiprows benchmark

Co-authored-by: Anatoly Myachev <45976948+anmyachev@users.noreply.github.com>
Signed-off-by: Alexander Myskov <alexander.myskov@intel.com>
@dchigarev
FIX-#2735: move '.reindex' logic about axis dispatching from the base…
7935c59 
… class (#2736)

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
TEST-#1496: add tests for setting new column with different from fram…
aa818f5 
…e length (#2733)

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@amyskov
REFACTOR-#2739: io tests refactoring (#2740)
8ebbad9 
Signed-off-by: Alexander Myskov <alexander.myskov@intel.com>
@dchigarev
TEST-#2753: add GroupBy benchmarsk with huge amount of groups (#2754)
61c6e99 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
FIX-#2362: fix handling slices in 'DataFrame.__setitem__' (#2741)
1f3b514 
Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@dchigarev
FIX-#2742: fix performance degradation for dictionary GroupBy aggrega…
ad0bcab 
…tion (#2743)

* FIX-#2742: changed callable functions to its names in dict aggregation

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>

* FIX-#2742: commends added

Signed-off-by: Dmitry Chigarev <dmitry.chigarev@intel.com>
@amyskov @anmyachev
FIX-#2737: fix handling of dates for read_csv with OmniSci backend (#…
84c9ab5 
…2738)

Co-authored-by: Anatoly Myachev <45976948+anmyachev@users.noreply.github.com>
Signed-off-by: Alexander Myskov <alexander.myskov@intel.com>
@devin-petersohn
DOCS-#2584: Add CODEOWNERS file (#2759)
d1321f5 
* Resolves #2584

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
@aregm
Copy link
Owner Author

aregm commented Feb 18, 2021

Sync

@aregm aregm merged commit 62bbe39 into aregm:master Feb 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.