Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.10.1 & turn into multi-output recipe to account for optional installs #14

Merged
merged 16 commits into from
Jul 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .azure-pipelines/azure-pipelines-linux.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions .azure-pipelines/azure-pipelines-osx.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions .azure-pipelines/azure-pipelines-win.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 19 additions & 0 deletions .ci_support/linux_64_python3.6.____cpython.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
cdt_name:
- cos6
channel_sources:
- conda-forge,defaults
channel_targets:
- conda-forge main
docker_image:
- quay.io/condaforge/linux-anvil-comp7
pin_run_as_build:
python:
min_pin: x.x
max_pin: x.x
python:
- 3.6.* *_cpython
target_platform:
- linux-64
zip_keys:
- - cdt_name
- docker_image
16 changes: 16 additions & 0 deletions .ci_support/osx_64_python3.6.____cpython.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
MACOSX_DEPLOYMENT_TARGET:
- '10.9'
channel_sources:
- conda-forge,defaults
channel_targets:
- conda-forge main
macos_machine:
- x86_64-apple-darwin13.4.0
pin_run_as_build:
python:
min_pin: x.x
max_pin: x.x
python:
- 3.6.* *_cpython
target_platform:
- osx-64
12 changes: 12 additions & 0 deletions .ci_support/win_64_python3.6.____cpython.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
channel_sources:
- conda-forge,defaults
channel_targets:
- conda-forge main
pin_run_as_build:
python:
min_pin: x.x
max_pin: x.x
python:
- 3.6.* *_cpython
target_platform:
- win-64
32 changes: 30 additions & 2 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

195 changes: 173 additions & 22 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,37 +1,187 @@
{% set name = "modin" %}
{% set version = "0.10.0" %}
{% set version = "0.10.1" %}

package:
name: modin
name: modin-packages
version: {{ version }}

source:
url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz
sha256: 88ce4def4a1145d31d207be55b9c3ba914e084ccfebace1d52b03507df258fd2
url: https://github.com/modin-project/modin/archive/refs/tags/{{ version }}.tar.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be worth adding a comment explaining why we take github instead of pypi

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can if you want. It might just be this release where the sources are different - for the feedstocks I maintain, I consider PyPI & GH sources to be completely equivalent (actually with a slight preference to GH), and not really worth commenting about. But YMMV.

sha256: a0ac09d6ef94185c5dd949a9cec047e5b922e7dd5884abda1e455c76d8958cd4
patches:
# Remove all requirements from setup.py - they should be handled here.
# The point is to create a "patch failed" conflict when dependencies change
# in upstream so this doesn't go unnoticed.
- patches/0001-Remove-all-dependencies-from-setup.py.patch

build:
number: 0
script: '{{ PYTHON }} -m pip install . --no-deps --ignore-installed -vv '
# noarch disabled because the recipe now has selector for Ray, see below
# noarch: python
skip: true # [py<37]

requirements:
host:
- pip
- python
run:
- python
- pandas ==1.2.3
- dask >=2.12
- distributed >=2.12
# there is no Ray in conda-forge for MacOS yet, see https://github.com/conda-forge/ray-packages-feedstock/issues/2
- ray-core >=1.0 # [win or linux]

test:
imports:
- modin
- modin.pandas
# the outputs map the modin options on PyPI (see upstream definition
# https://github.com/modin-project/modin/blob/0.10.1/setup.py#L40-L45)
# from "modin[<option>]" to "modin-<option>" as a conda-forge package
outputs:
- name: modin-all
requirements:
host:
run:
- python
- {{ pin_subpackage('modin-core', exact=True) }}
- {{ pin_subpackage('modin-dask', exact=True) }}
# ray does not have OSX builds yet
- {{ pin_subpackage('modin-ray', exact=True) }} # [not osx]
# Note: modin-omnisci is conda-only, its dependencies were never published on PyPI
- {{ pin_subpackage('modin-omnisci', exact=True) }} # [linux]
#- {{ pin_subpackage('modin-remote', exact=True) }} # turned off for now, see below
#- {{ pin_subpackage('modin-spreadsheet', exact=True) }} # turned off for now, see below
#- {{ pin_subpackage('modin-sql', exact=True) }} # turned off for now, see below
test:
imports:
# dummy test; actual tests are in subpackages
- modin

# compatibility with packagename before split into options
- name: modin
requirements:
host:
run:
- python
# modin needs at least one engine; use dask as it is more light-weight than ray
- {{ pin_subpackage('modin-dask', exact=True) }}
test:
imports:
- modin
- modin.pandas

- name: modin-core
build:
script: 'python -m pip install . --no-deps --ignore-installed -vv'
requirements:
host:
- pip
- python
run:
- python
- packaging
- pandas ==1.3.0
- numpy >=1.16.5
test:
imports:
- modin
- modin.pandas

- name: modin-dask
build:
requirements:
host:
- python
run:
- python
- {{ pin_subpackage('modin-core', exact=True) }}
- dask >=2.22.0
- distributed >=2.22.0
test:
imports:
- modin
- modin.pandas
commands:
- python -c "import modin.pandas as pd, modin.config as cfg; cfg.Engine.put('Dask'); df = pd.DataFrame([])"

- name: modin-ray
build:
# there is no Ray in conda-forge for MacOS yet, see conda-forge/ray-packages-feedstock#2
skip: true # [osx]
requirements:
host:
- python
run:
- python
- {{ pin_subpackage('modin-core', exact=True) }}
- ray-core >=1.4.0
- pyarrow >=1.0
test:
imports:
- modin
- modin.pandas
commands:
- python -c "import modin.pandas as pd, modin.config as cfg; cfg.Engine.put('Ray'); df = pd.DataFrame([])"

- name: modin-omnisci
build:
# there is no OmniSci on anything but Linux for now
skip: true # [not linux]
requirements:
host:
- python
run:
- python
- {{ pin_subpackage('modin-ray', exact=True) }}
- pyomniscidbe
test:
imports:
- modin
- modin.pandas
commands:
- python -c "import modin.experimental.pandas as pd, modin.config as cfg; cfg.Engine.put('Ray'); cfg.Backend.put('OmniSci'); df = pd.DataFrame([])"

- name: modin-sql
build:
# Dependency dfsql is not in conda-forge yet.
skip: true
requirements:
host:
- python
run:
- python
- {{ pin_subpackage('modin-core', exact=True) }}
- dfsql >=0.2.6
test:
imports:
# are there more specific imports?!
- modin

- name: modin-remote
build:
# Dependency rpyc is not in conda-forge yet.
# Also requires ray-autoscaler which is now absent because
# its dependencies on conda-forge are too ancient.
Comment on lines +148 to +149
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vnlitvinov, I don't see the dependency on ray[autoscaler] in setup.py for modin[remote]. Where does this come from? Again a conda-only dependency?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, just knowledge of how things are implemented inside (as I was the one who implemented them 😄).
That said, ray does not have a ray[autoscale] target yet, ref: ray-project/ray#15725

skip: true
requirements:
host:
- python
run:
- python
- {{ pin_subpackage('modin-core', exact=True) }}
- boto3 ==1.4.8
- cloudpickle
- rpyc ==4.1.5
test:
imports:
# are there more specific imports?!
- modin

# TODO: bring https://pypi.org/project/modin-spreadsheet/ to conda-forge
# as modin-spreadsheet-core, leave modin-spreadsheet to be a subpackage here;
# see discussion with upstream developers here:
# https://github.com/conda-forge/modin-feedstock/pull/14/files#r595534632
- name: modin-spreadsheet
build:
# dependency modin-spreadsheet-core is not in conda-forge yet
skip: true
requirements:
host:
- python
run:
- python
- {{ pin_subpackage('modin-core', exact=True) }}
- modin-spreadsheet >=0.1.0
test:
imports:
# are there more specific imports?!
- modin

about:
home: https://github.com/modin-project/modin
license: Apache-2.0
Expand All @@ -45,3 +195,4 @@ extra:
recipe-maintainers:
- devin-petersohn
- h-vetinari
feedstock-name: modin
Loading