update scikit to 0.20.0 #100

k-dominik · 2023-03-16T16:53:37Z

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

conda-forge-webservices · 2023-03-16T16:53:42Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

k-dominik · 2023-03-16T17:15:38Z

meh, ok for some reason https://pypi.io/packages/source/s/scikit-image/scikit-image-0.20.0.tar.gz doesn't point to the source dist...

k-dominik · 2023-03-16T17:19:55Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2023.03.16.14.46.37

jni · 2023-03-17T05:54:53Z

Thanks for the update @k-dominik! I don't know why the conda-forge bot went on strike. 🤔

I expect the tests are failing because the dependencies in the recipe need to be updated. Here's the current deps:

https://github.com/scikit-image/scikit-image/blob/907bfb71d2c8e4d58c2dffc6056760c1e00d070b/pyproject.toml#L70-L79

ref: https://github.com/scikit-image/scikit-image/blob/907bfb71d2c8e4d58c2dffc6056760c1e00d070b/pyproject.toml#L70-L79 also removed Python 3.7 special pins - Python 3.7 is not longer supported by skimage as far as I can tell.

k-dominik · 2023-03-17T09:15:29Z

Thank you for the quick feedback @jni,

I updated the dependencies as suggested, but there seems to be something off with pypi - the tests fail with:

Downloading https://pypi.io/packages/source/s/scikit-image/scikit-image-0.20.0.tar.gz
INFO:conda_build.source:Source cache directory is: /home/conda/feedstock_root/build_artifacts/src_cache
INFO:conda_build.source:Downloading source to cache: scikit-image-0.20.0_e1076d7dc2.tar.gz
INFO:conda_build.source:Downloading https://pypi.io/packages/source/s/scikit-image/scikit-image-0.20.0.tar.gz
Error: HTTP 404 NOT FOUND for url <[https://pypi.io/packages/source/s/scikit-image/scikit-image-0.20.0.tar.gz>](https://pypi.io/packages/source/s/scikit-image/scikit-image-0.20.0.tar.gz%3E)
Elapsed: 00:00.056455

okay, after looking at this, I realized that the artifact on pypi has scikit_image (-> scikit_image-0.20.0.tar.gz) with an underscore instead of a hyphen. Is this expected?

k-dominik · 2023-03-17T10:05:56Z

not sure the required version of meson-python can currently be satisfied as it's an rc... I assume meson-python 0.12.1 won't do?

Is it possible that all the cross-compilation issues in CI are related to that?
Also I'm not quite sure about the numpy 1.20 builds as I diligently specified numpy >=1.21 in the recipe 🤷

mkcor · 2023-03-17T14:57:18Z

not sure the required version of meson-python can currently be satisfied as it's an rc... I assume meson-python 0.12.1 won't do?

I guess @jarrodmillman would know. This minimum version requirement comes from scikit-image/scikit-image#6757.

stefanv · 2023-03-17T17:19:49Z

okay, after looking at this, I realized that the artifact on pypi has scikit_image (-> scikit_image-0.20.0.tar.gz) with an underscore instead of a hyphen. Is this expected?

This is a known change. Meson, our new build system, follows the pypa rules for cleaning up sdist names, which setuptools (ironically) never did.

jni · 2023-03-22T22:51:43Z

@k-dominik

not sure the required version of meson-python can currently be satisfied as it's an rc... I assume meson-python 0.12.1 won't do?

This PR to the meson-python-feedstock suggests that those issues maybe should be fixed, but I see that in the recipe you didn't constrain meson-python, so I don't think that's where they come from. Hopefully @stefanv and @jarrodmillman can have a look at the build soon...

stefanv · 2023-03-23T00:05:18Z

I think Meson needs the cross files to cross-compile on arm64? See, e.g., the SciPy patch and the Meson docs.

stefanv · 2023-03-23T00:11:01Z

@k-dominik I am going to try to get the correct CPU version of ninja in place; if pushing to your PR in conda-forge is considered bad practice, I do apologize, and feel free to remove those patches.

stefanv · 2023-03-28T18:52:08Z

Build machine cpu family: aarch64
Build machine cpu: aarch64
Host machine cpu family: aarch64
Host machine cpu: arm64
Target machine cpu family: aarch64
Target machine cpu: arm64

stefanv · 2023-03-28T18:58:59Z

@hmaarrfk what does re-rendering do exactly? Looks like it updates to the most recent version of all the forge CI pipelines?

Rerendering is conda-forge’s way to update the files common to all feedstocks (e.g. README, CI configuration, pinned dependencies).

hmaarrfk · 2023-03-29T00:33:27Z

@stefanv rerendering does 2 things:

"pins" compile time dependencies.
- You can build the same package with multiple different compile time dependencies. For example, we build for different versions of python, each version is "pinned" in the associated yaml file.
Generates the scripts required for the CIs (there can be multiple providers) to do the work.

In order to not step on each other's toes, I'm going to make small attempts in
#101

Sorry for the noise here.

hmaarrfk · 2023-03-29T00:35:09Z

I thought scipy had already trailblazed meson + scientific python? is that not true? I thought we were following the build procedure of an other major package? Can we use their recipe as a model?

hmaarrfk · 2023-03-29T02:09:00Z

I haven't really kept up with how you all managed to build OSX on CIs with meson python. Maybe that can be extended to work on conda-forge? If you point me to it I can likely help with that.

I noticed this bug report opened by isuruf (conda-forge core member) mesonbuild/meson-python#321

To get windows to work you will need: 7390e30
You may want to skip the tests found in 4c115c9 on pypy. When we do, we should likely open an issue in the scikit-image repo too.

stefanv · 2023-03-29T04:34:11Z

That's a decent set of passes. Now, most of the remaining ones are of the form:

skimage/meson.build:33:0: ERROR: Command `$PREFIX/bin/python -c 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'` failed with status 1.

Still no aarch64 / arm64 builds.

stefanv · 2023-03-29T04:39:22Z

I don't think there's much more I can do to help here. I'll follow along, just in case some knowledge of the build system is helpful.

@hmaarrfk We don't cross-compile for macosx, we just build natively: https://github.com/scikit-image/scikit-image/blob/main/.github/workflows/wheel_tests_and_release.yml

The one tricky bit with that build was a bug in meson-python where they look at MACOS_DEPLOYMENT_TARGET instead of at MACOSX_DEPLOYMENT_TARGET.

stefanv · 2023-03-29T04:46:14Z

I thought scipy had already trailblazed meson + scientific python? is that not true? I thought we were following the build procedure of an other major package? Can we use their recipe as a model?

Not that I could find at https://github.com/conda-forge/scipy-feedstock or https://github.com/conda-forge/scipy-feedstock/pulls

stefanv · 2023-03-29T05:07:20Z

I thought scipy had already trailblazed meson + scientific python? is that not true? I thought we were following the build procedure of an other major package? Can we use their recipe as a model?

Not that I could find at https://github.com/conda-forge/scipy-feedstock or https://github.com/conda-forge/scipy-feedstock/pulls

OK, I found the work that SciPy's done that can (maybe?) get us our macosx cross-compiled binaries!

conda-forge/scipy-feedstock#205

This reverts commit 2ef040a.

hmaarrfk · 2023-03-29T12:34:00Z

Still no aarch64 / arm64 builds.

See builds in
#101

aarch seems to run and pass tests on linux.

stefanv · 2023-03-29T17:20:57Z

@isuruf I apologize for tagging you personally, but I think we are quite close on this build here, and wondered whether you had any insight on how to get the arm64 cross-compilation going?

isuruf · 2023-03-29T17:33:56Z

#101 seems to be going in the correct direction

k-dominik · 2023-03-29T19:59:56Z

closed in favor of #101

k-dominik requested review from stefanv, emmanuelle, grlee77, hmaarrfk, jakirkham, jni, msarahan, ocefpaf and soupault as code owners March 16, 2023 16:53

k-dominik force-pushed the patch-1 branch from b204d4c to 6c15d32 Compare March 16, 2023 17:18

update scikit to 0.20.0

b35fa31

k-dominik force-pushed the patch-1 branch from 6c15d32 to b35fa31 Compare March 16, 2023 17:19

MNT: Re-rendered with conda-build 3.23.3, conda-smithy 3.23.0, and co…

d38370c

…nda-forge-pinning 2023.03.16.14.46.37

updated dependencies

6887f3a

ref: https://github.com/scikit-image/scikit-image/blob/907bfb71d2c8e4d58c2dffc6056760c1e00d070b/pyproject.toml#L70-L79 also removed Python 3.7 special pins - Python 3.7 is not longer supported by skimage as far as I can tell.

scikit_image with underscore in url

2b2a8b2

k-dominik force-pushed the patch-1 branch from 205cb84 to 2b2a8b2 Compare March 17, 2023 09:20

add/update missing build deps

b1cec3d

k-dominik force-pushed the patch-1 branch from d67eee9 to b1cec3d Compare March 17, 2023 10:03

Add ninja for cross-platform builds

4e8e835

Try single category specification of dependencies

c17a8ac

stefanv force-pushed the patch-1 branch from 8d80eb3 to c17a8ac Compare March 28, 2023 19:09

stefanv added 3 commits March 28, 2023 15:08

Match NumPy to build machine architecture

f5ee923

Use clang instead of clang-cl to build on Windows

20aebdc

Encourage Windows to use clang

e4c85d2

stefanv force-pushed the patch-1 branch from af78843 to e4c85d2 Compare March 28, 2023 23:44

stefanv added 2 commits March 28, 2023 17:04

Just install NumPy already

e2d1350

Host install Python, otherwise it's not always found

2965be3

Try Mark's suggestions

dd3678f

stefanv force-pushed the patch-1 branch 3 times, most recently from 4461633 to ebab4db Compare March 29, 2023 06:09

stefanv mentioned this pull request Mar 29, 2023

Unable to install conda environment for development scikit-image/scikit-image#6494

Open

Invoke correct Python when cross-compiling

2ef040a

stefanv force-pushed the patch-1 branch from ebab4db to 2ef040a Compare March 29, 2023 06:21

Revert "Invoke correct Python when cross-compiling"

5b55d51

This reverts commit 2ef040a.

k-dominik closed this Mar 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update scikit to 0.20.0 #100

update scikit to 0.20.0 #100

k-dominik commented Mar 16, 2023 •

edited

Loading

conda-forge-webservices bot commented Mar 16, 2023

k-dominik commented Mar 16, 2023

k-dominik commented Mar 16, 2023

jni commented Mar 17, 2023

k-dominik commented Mar 17, 2023 •

edited

Loading

k-dominik commented Mar 17, 2023 •

edited

Loading

mkcor commented Mar 17, 2023

stefanv commented Mar 17, 2023

jni commented Mar 22, 2023

stefanv commented Mar 23, 2023 •

edited

Loading

stefanv commented Mar 23, 2023

stefanv commented Mar 28, 2023

stefanv commented Mar 28, 2023

hmaarrfk commented Mar 29, 2023

hmaarrfk commented Mar 29, 2023

hmaarrfk commented Mar 29, 2023

stefanv commented Mar 29, 2023 •

edited

Loading

stefanv commented Mar 29, 2023

stefanv commented Mar 29, 2023

stefanv commented Mar 29, 2023 •

edited

Loading

hmaarrfk commented Mar 29, 2023

stefanv commented Mar 29, 2023

isuruf commented Mar 29, 2023

k-dominik commented Mar 29, 2023

update scikit to 0.20.0 #100

update scikit to 0.20.0 #100

Conversation

k-dominik commented Mar 16, 2023 • edited Loading

conda-forge-webservices bot commented Mar 16, 2023

k-dominik commented Mar 16, 2023

k-dominik commented Mar 16, 2023

jni commented Mar 17, 2023

k-dominik commented Mar 17, 2023 • edited Loading

k-dominik commented Mar 17, 2023 • edited Loading

mkcor commented Mar 17, 2023

stefanv commented Mar 17, 2023

jni commented Mar 22, 2023

stefanv commented Mar 23, 2023 • edited Loading

stefanv commented Mar 23, 2023

stefanv commented Mar 28, 2023

stefanv commented Mar 28, 2023

hmaarrfk commented Mar 29, 2023

hmaarrfk commented Mar 29, 2023

hmaarrfk commented Mar 29, 2023

stefanv commented Mar 29, 2023 • edited Loading

stefanv commented Mar 29, 2023

stefanv commented Mar 29, 2023

stefanv commented Mar 29, 2023 • edited Loading

hmaarrfk commented Mar 29, 2023

stefanv commented Mar 29, 2023

isuruf commented Mar 29, 2023

k-dominik commented Mar 29, 2023

k-dominik commented Mar 16, 2023 •

edited

Loading

k-dominik commented Mar 17, 2023 •

edited

Loading

k-dominik commented Mar 17, 2023 •

edited

Loading

stefanv commented Mar 23, 2023 •

edited

Loading

stefanv commented Mar 29, 2023 •

edited

Loading

stefanv commented Mar 29, 2023 •

edited

Loading