-
Notifications
You must be signed in to change notification settings - Fork 28.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-46919][BUILD][CONNECT] Upgrade grpcio*
and grpc-java
to 1.62.x
#44929
Conversation
@@ -188,6 +188,17 @@ | |||
<artifactId>grpc-protobuf</artifactId> | |||
<version>${io.grpc.version}</version> | |||
</dependency> | |||
<dependency> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"io.grpc.protobuf.lite.ProtoLiteUtils
has been moved to the grpc-protobuf-lite
module, but protobuf-javalite
and protobuf-java
cannot be used simultaneously because there would be a conflict with com.google.protobuf.Any
."
grpcio*
1.60.0grpcio*
to 1.60.0 and grpc-java
to 1.61.0
.github/workflows/build_and_test.yml
Outdated
@@ -704,7 +704,7 @@ jobs: | |||
python3.9 -m pip install 'sphinx==4.5.0' mkdocs 'pydata_sphinx_theme>=0.13' sphinx-copybutton nbsphinx numpydoc jinja2 markupsafe 'pyzmq<24.0.0' \ | |||
ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.20.0' pyarrow pandas 'plotly>=4.8' 'docutils<0.18.0' \ | |||
'flake8==3.9.0' 'mypy==0.982' 'pytest==7.1.3' 'pytest-mypy-plugins==1.9.3' 'black==23.9.1' \ | |||
'pandas-stubs==1.2.0.53' 'grpcio==1.59.3' 'grpc-stubs==1.24.11' 'googleapis-common-protos-stubs==2.2.0' \ | |||
'pandas-stubs==1.2.0.53' 'grpcio==1.60.0' 'grpc-stubs==1.24.11' 'googleapis-common-protos-stubs==2.2.0' \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, there is no 1.61.0 on the python side yet?
I think this is OK, but should we stick to 1.61.0 on the JVM side for now then too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that they are compatible, and grpc-java 1.61.0 brings an additional optimization related to memory use:
so this PR has upgraded grpc-java to version 1.61.0.
1.61.0 released, mark this one to draft first |
grpcio*
to 1.60.0 and grpc-java
to 1.61.0grpcio*
and grpc-java
to 1.62.0
grpcio*
and grpc-java
to 1.62.0grpcio*
and grpc-java
to 1.62
grpcio*
and grpc-java
to 1.62grpcio*
and grpc-java
to 1.62.x
grpcio*
and grpc-java
to 1.62.xgrpcio*
and grpc-java
to 1.62.x
The version has been unified to the 1.62 series now |
grpcio*
and grpc-java
to 1.62.xgrpcio*
and grpc-java
to 1.62.x
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM. Thank you, @LuciferYang .
Merged to master for Apache Spark 4.0.0.
…62.x ### What changes were proposed in this pull request? This PR aims to upgrade `grpcio*` from 1.59.3 to [1.62.0](https://pypi.org/project/grpcio/1.62.0/)and `grpc-java` from 1.59.0 to 1.62.2 for Apache Spark 4.0.0. ### Why are the changes needed? grpc 1.60.0 start to support dualstack IPv4 and IPv6 backend support: - Implemented dualstack IPv4 and IPv6 backend support, as per draft gRFC A61. xDS support currently guarded by GRPC_EXPERIMENTAL_XDS_DUALSTACK_ENDPOINTS env var. Note that in `grpc-java` 1.61.0, since the dependency scope of `grpc-protobuf` on `grpc-protobuf-lite` has been changed from `compile` to `runtime`, we need to manually configure the dependency of the `connect` module on `grpc-protobuf-lite` and explicitly exclude the dependency on `protobuf-javalite` because `SparkConnectService` uses `io.grpc.protobuf.lite.ProtoLiteUtils` - https://github.com/grpc/grpc-java/pull/10756/files The relevant release notes are as follows: - https://github.com/grpc/grpc/releases/tag/v1.60.0 - https://github.com/grpc/grpc/releases/tag/v1.60.1 - https://github.com/grpc/grpc/releases/tag/v1.61.0 - https://github.com/grpc/grpc/releases/tag/v1.61.1 - https://github.com/grpc/grpc/releases/tag/v1.62.0 - https://github.com/grpc/grpc-java/releases/tag/v1.60.0 - https://github.com/grpc/grpc-java/releases/tag/v1.60.1 - https://github.com/grpc/grpc-java/releases/tag/v1.61.0 - https://github.com/grpc/grpc-java/releases/tag/v1.61.1 - https://github.com/grpc/grpc-java/releases/tag/v1.62.2 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#44929 from LuciferYang/grpc-16. Lead-authored-by: yangjie01 <yangjie01@baidu.com> Co-authored-by: YangJie <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Thanks @dongjoon-hyun and @srowen |
…62.x ### What changes were proposed in this pull request? This PR aims to upgrade `grpcio*` from 1.59.3 to [1.62.0](https://pypi.org/project/grpcio/1.62.0/)and `grpc-java` from 1.59.0 to 1.62.2 for Apache Spark 4.0.0. ### Why are the changes needed? grpc 1.60.0 start to support dualstack IPv4 and IPv6 backend support: - Implemented dualstack IPv4 and IPv6 backend support, as per draft gRFC A61. xDS support currently guarded by GRPC_EXPERIMENTAL_XDS_DUALSTACK_ENDPOINTS env var. Note that in `grpc-java` 1.61.0, since the dependency scope of `grpc-protobuf` on `grpc-protobuf-lite` has been changed from `compile` to `runtime`, we need to manually configure the dependency of the `connect` module on `grpc-protobuf-lite` and explicitly exclude the dependency on `protobuf-javalite` because `SparkConnectService` uses `io.grpc.protobuf.lite.ProtoLiteUtils` - https://github.com/grpc/grpc-java/pull/10756/files The relevant release notes are as follows: - https://github.com/grpc/grpc/releases/tag/v1.60.0 - https://github.com/grpc/grpc/releases/tag/v1.60.1 - https://github.com/grpc/grpc/releases/tag/v1.61.0 - https://github.com/grpc/grpc/releases/tag/v1.61.1 - https://github.com/grpc/grpc/releases/tag/v1.62.0 - https://github.com/grpc/grpc-java/releases/tag/v1.60.0 - https://github.com/grpc/grpc-java/releases/tag/v1.60.1 - https://github.com/grpc/grpc-java/releases/tag/v1.61.0 - https://github.com/grpc/grpc-java/releases/tag/v1.61.1 - https://github.com/grpc/grpc-java/releases/tag/v1.62.2 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#44929 from LuciferYang/grpc-16. Lead-authored-by: yangjie01 <yangjie01@baidu.com> Co-authored-by: YangJie <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
This PR aims to upgrade
grpcio*
from 1.59.3 to 1.62.0andgrpc-java
from 1.59.0 to 1.62.2 for Apache Spark 4.0.0.Why are the changes needed?
grpc 1.60.0 start to support dualstack IPv4 and IPv6 backend support:
Note that in
grpc-java
1.61.0, since the dependency scope ofgrpc-protobuf
ongrpc-protobuf-lite
has been changed fromcompile
toruntime
, we need to manually configure the dependency of theconnect
module ongrpc-protobuf-lite
and explicitly exclude the dependency onprotobuf-javalite
becauseSparkConnectService
usesio.grpc.protobuf.lite.ProtoLiteUtils
The relevant release notes are as follows:
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass GitHub Actions
Was this patch authored or co-authored using generative AI tooling?
No