[SPARK-49097][INFRA] Add Python3 environment detection for the build_error_docs method in build_api_decs.rb
#47590
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yaooqinn
changed the title
build_orror_docs method in build_api_decs.rbbuild_error_docs method in build_api_decs.rb
wayneguow
commented
Aug 2, 2024
| raise("Missing python3 in your path, stopping error doc generation") | ||
| end | ||
|
|
||
| system("python3 '#{SPARK_PROJECT_ROOT}/docs/util/build-error-docs.py'") \ |
There was a problem hiding this comment.
Since users may not use python venv, I suggest changing to python3 here.
There was a problem hiding this comment.
It seems the failure is relevant to this change.
Could you make CI happy, @wayneguow ?
wayneguow
changed the title
build_error_docs method in build_api_decs.rbbuild_error_docs method in build_api_decs.rb
wayneguow
commented
Aug 4, 2024
| @@ -872,6 +872,8 @@ jobs: | |||
| run: | | |||
| # We need this link because the jekyll build calls `python`. | |||
| ln -s "$(which python3.9)" "/usr/local/bin/python" | |||
| # We need this link to make sure `python3` points to `python3.9` which contains the prerequisite packages. | |||
| ln -s "$(which python3.9)" "/usr/local/bin/python3" | |||
There was a problem hiding this comment.
Currently, there are multiple pythons in the ci env：
- Python 3.9.19
- Python 3.10.12
And python3 points toPython 3.10.12.
wayneguow
changed the title
build_error_docs method in build_api_decs.rbbuild_error_docs method in build_api_decs.rb
Well, I found out the reason, because python3 currently does not point to the python3.9 with the installation packages we need. |
dongjoon-hyun
requested changes
Aug 4, 2024
.github/workflows/build_and_test.yml
Outdated
| @@ -872,6 +872,8 @@ jobs: | |||
| run: | | |||
| # We need this link because the jekyll build calls `python`. | |||
| ln -s "$(which python3.9)" "/usr/local/bin/python" | |||
There was a problem hiding this comment.
Can we remove the above two lines from SPARK-46935, @wayneguow ?
There was a problem hiding this comment.
Yes, good advice, this link is indeed introduced in this PR #44971 and only used in this place.
HyukjinKwon
approved these changes
Aug 5, 2024
dongjoon-hyun
approved these changes
Aug 5, 2024
There was a problem hiding this comment.
+1, LGTM. Thank you, @wayneguow and @HyukjinKwon .
Merged to master.
fusheng-rd
pushed a commit
to fusheng-rd/spark
that referenced
this pull request
Aug 6, 2024
…_error_docs` method in `build_api_decs.rb` ### What changes were proposed in this pull request? This PR aims to add Python3 environment detection for the `build_orror_docs` method in `build_api_decs.rb`. ### Why are the changes needed? Make the environment exception prompts more friendly for developers when generating documents. Before: <img width="1322" alt="image" src="https://github.com/user-attachments/assets/9f31c951-e63a-479a-9600-2b62e8ad9ddd"> After: <img width="1379" alt="image" src="https://github.com/user-attachments/assets/b0841f42-237a-429e-8673-5254328c6dd2"> ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#47590 from wayneguow/inf. Authored-by: Wei Guo <guow93@gmail.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
attilapiros
pushed a commit
to attilapiros/spark
that referenced
this pull request
Oct 4, 2024
…_error_docs` method in `build_api_decs.rb` ### What changes were proposed in this pull request? This PR aims to add Python3 environment detection for the `build_orror_docs` method in `build_api_decs.rb`. ### Why are the changes needed? Make the environment exception prompts more friendly for developers when generating documents. Before: <img width="1322" alt="image" src="https://github.com/user-attachments/assets/9f31c951-e63a-479a-9600-2b62e8ad9ddd"> After: <img width="1379" alt="image" src="https://github.com/user-attachments/assets/b0841f42-237a-429e-8673-5254328c6dd2"> ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#47590 from wayneguow/inf. Authored-by: Wei Guo <guow93@gmail.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR aims to add Python3 environment detection for the
build_orror_docsmethod inbuild_api_decs.rb.Why are the changes needed?
Make the environment exception prompts more friendly for developers when generating documents.
Before:
After:
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GA.
Was this patch authored or co-authored using generative AI tooling?
No.