Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish Python library on PyPi? #161

Open
ataber opened this issue Jul 5, 2017 · 9 comments
Open

Publish Python library on PyPi? #161

ataber opened this issue Jul 5, 2017 · 9 comments

Comments

@ataber
Copy link

ataber commented Jul 5, 2017

Hello! I'm very excited about using OpenVDB, but I'm looking for a cross-platform, easy-to-install way of interacting with it. I have successfully compiled but it would be great to be able to install the python library like pip install pyopenvdb. I'm not sure of the challenges to do that, but I thought I would bring it up because having python packages available via pip decreases the startup cost by an order of magnitude, easily. Thoughts?

@peytondmurray
Copy link

Hey all, sorry to dig up such an old issue, but I'm interested in being able to install openvdb's python bindings as a package from PYPI as well. I'd be happy to submit a PR with the packaging necessary to publish on PYPI, if the maintainers would accept it. I can also write the github workflows to automate building and publishing the releases. Would this be something the maintainers be okay with?

@Idclip
Copy link
Contributor

Idclip commented Apr 13, 2022

Hi @peytondmurray - I think this would be great to see! I'm just not sure what exactly this looks like as I haven't worked too much with PYPI. My understanding is that you would be contributing a bunch of scripts to allow us to automate and test the publishing process of a PYPI package from within the OpenVDB repo? I see a number of pyopenvdb packages already published by the community - do you see any issues with us publishing our own with the same name?

I'll speak to the other maintainers to confirm we'd be happy to support this.

@peytondmurray
Copy link

My understanding is that you would be contributing a bunch of scripts to allow us to automate and test the publishing process of a PYPI package from within the OpenVDB repo?

Yeah, that's right. Actually, I think there's a way to set up Github actions to automatically publish (not just test) a new version of pyopenvdb on PYPI whenever you make a new release. If we can get that working then the process of publishing to PYPI won't require any extra effort at all when it comes time to make new releases of OpenVDB.

I see a number of pyopenvdb packages already published by the community - do you see any issues with us publishing our own with the same name?

There are indeed issues with publishing our own with the same name, but I think the effort in overcoming them will be worth it. For reference, there are currently two separate projects on PYPI which show up if you search "pyopenvdb", but they're both really the same:

  1. pyopenvdb 0.1.4 is a project for building and packaging pyopenvdb in a docker container. There's a github page which has commits from January 2021, but no PYPI release has been made in 2 years.
  2. pyopenvdb-3.8 0.1.5 has a link to the same github page as the first result.

I'm guessing the author made both projects in the course of figuring out how to publish packages to PYPI (definitely done this myself before...). In any case, the rules governing PYPI package name transfers are clear: the first thing to do is just try to talk to the owner of the packages. There haven't been any releases on either of these projects in two years; if we email the author three times, and we don't get a response within six weeks, then the projects already meet all the requirements for being considered "abandoned." At that point I think we have a good case for transferring ownership of those PYPI packages to the official OpenVDB project maintainers.

Of course, all of that won't be necessary if we can get a hold of the owner of the projects. I hope they'll be willing to give up those projects, especially if we publish the officially supported python packages in their place.

I'll speak to the other maintainers to confirm we'd be happy to support this.

Thank you! I'd also be happy to contact the owner of the existing packages that we'd need to remove on behalf of the OpenVDB project to get the process started, but I'll leave the decision to do that up to you. Let me know how you'd like me to proceed.

@Idclip
Copy link
Contributor

Idclip commented Apr 26, 2022

Hey @peytondmurray - sorry for the delayed reply. Thanks a lot for outlining the process, this all sounds good to me! I confirmed today with the other maintainers that it would be great to pursue this and we'd happily support your proposal of an automated/CI based system for building and publishing pyopenvdb.

If you're still interested in looking at this please feel free to start the process and e-mail the author of those other published packages on our behalf. We have a public e-mail/lists openvdb-dev@lists.aswf.io which you can include us on (all messages available here https://lists.aswf.io/g/openvdb-dev/topics) but please let me know if you need anything else!

@peytondmurray
Copy link

Awesome, I'll get started on this. Thank you! 🎉

@Korijn
Copy link

Korijn commented May 3, 2022

If you need any help I have lots of experience setting this type of thing up, hit me up! I'm also keen on getting my hands on openvdb in the python ecosystem. :)

@peytondmurray
Copy link

peytondmurray commented May 7, 2022

An update on progress so far, since I know some other people have expressed interest here:

  • Contacted the current owner of the pyopenvdb package on PyPI (Alex). Alex was super nice, very responsive, and excited that there are other people interested in python packaging for OpenVDB. He agreed to transfer ownership of the pyopenvdb package name to the OpenVDB maintainers, on the proviso that he is credited with writing the original packaging for the python bindings. I said that I thought that this would be fine - we can credit him in the README.md for the python bindings or some other appropriate place (CONTRIBUTORS.md or something, maybe?), as long as the maintainers agree
  • Fork the repo and started a new branch for python packaging development
  • Add README.md explaining installation, dependencies, and build toolchain
  • Add pyproject.toml: specifies python build-time dependencies and options for builds
    triggered by CI
  • Add setup.py, which makes openvdb/openvdb/python a python package that can be installed with pip. I've configured it to use scikit-build as a way of running cmake automatically during the build process, and setuptools_scm to set the appropriate package version string based on the current state of the git index
  • Set up github CI to build and publish an sdist for the package. I'm still in the process of testing this by uploading a package to https://test.pypi.org/legacy, but I think this is basically complete. I'm still trying to get wheels to build as part of the same CI pipeline, so I've left this unchecked for now
  • Set up cibuildwheel to automatically build wheels for python 3.8, 3.9, and 3.10 with manylinux images. This is in progress, and a bit fiddly to get right. Unfortunately I think I'll have to build Boost 1.70 inside the container (which will be slow; there is a github action to do this, but it doesn't support Boost 1.70, our minimum version) before building OpenVDB. Only then will I be able to build pyopenvdb. If wheels are too much work to generate, we can always just distribute an sdist which should cover pretty much everyone's use case - the only advantage of distributing wheels is that they're smaller and faster to install.

Currently the github workflow on the fork is configured to trigger in two ways:

  1. By a release, in which case it will build wheels and a source distribution for the released tag before uploading to PyPI
  2. Manually, in which case the person who triggers the build can specify a git ref for the build

As it stands, you can go to my fork, go to the python directory and run pip install . and it should build as long as you have the correct dependencies - including Boost, OpenVDB, CMake, and the python build dependencies specified in pyproject.toml.

@Alessi42
Copy link

Alessi42 commented Jun 6, 2024

Hello, Has there been an update on this? Very interested in being able to install OpenVDB bindings with pip

@peytondmurray
Copy link

No, sorry. I've been out of the loop and have passed off my previous work on this. Please see the associated PR for additional context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants