Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Editable jaxlib build #15205

Merged
merged 1 commit into from
Mar 27, 2023
Merged

Conversation

yhtang
Copy link
Collaborator

@yhtang yhtang commented Mar 24, 2023

This PR adds an --editable option to the build/build.py script. When set, the build script will

  1. Upon compilation completion, copy the jaxlib source folder and the compiled .so files from the Bazel sandbox to the output path (dist by default)
  2. print a message prompting users to use pip install -e to install jaxlib in-place at the output path.

This way, we only need to run pip install -e once, and subsequent Bazel builds with --editable will automatically update the installed jaxlib package (e.g., in a virtual environment).

Things worth consideration:

  • Would it be better to copy the compiled .so files directly back into the jaxlib source folder and run pip install -e there? This way, rebuilding is not needed if changes only apply to Python files.

This PR is adapted from the Alpa project as part of a rebasing effort, with contributions from @merrymercy, @zhisbug, @ZYHowell, @zhuohan123 among many others.

@google-cla
Copy link

google-cla bot commented Mar 24, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@yhtang
Copy link
Collaborator Author

yhtang commented Mar 24, 2023

@mjsML @nouiz for viz.

Co-authored-by: Yonghao Zhuang <zhuangyh@sjtu.edu.cn>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
@yhtang yhtang force-pushed the editable-jaxlib-build branch from 60d4d39 to caaa0a2 Compare March 24, 2023 21:25
@yhtang
Copy link
Collaborator Author

yhtang commented Mar 24, 2023

Updated commit message's co-author list after communicating with the original author @merrymercy

@google-ml-butler google-ml-butler bot added kokoro:force-run pull ready Ready for copybara import and testing labels Mar 27, 2023
@hawkinsp
Copy link
Collaborator

Things worth consideration:

  • Would it be better to copy the compiled .so files directly back into the jaxlib source folder and run pip install -e there? This way, rebuilding is not needed if changes only apply to Python files.

Perhaps, but I think not. In general it's nice to keep built artifacts out of the source tree.

@hawkinsp hawkinsp added pull ready Ready for copybara import and testing and removed pull ready Ready for copybara import and testing labels Mar 27, 2023
@hawkinsp hawkinsp closed this Mar 27, 2023
@hawkinsp hawkinsp reopened this Mar 27, 2023
@copybara-service copybara-service bot merged commit 6715736 into jax-ml:main Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pull ready Ready for copybara import and testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants