[Task]: Support custom Python SDK for sdks/java/extensions/python PythonService

[Abacn]

What needs to happen?

Currently, PythonService determines the Python SDK version as in

beam/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/PythonService.java

Line 87 in f64aec2

+ getMatchingStablePythonSDKVersion(ReleaseInfo.getReleaseInfo().getSdkVersion()));

and

beam/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/PythonService.java

Line 140 in f64aec2

static String getMatchingStablePythonSDKVersion(String javaSDKVersion) {

based on the java SDK info.

This has a few problems.

1. For release candidates, Java artifacts does not use "rc" suffices while Python SDK does. This will cause PythonService trying to install the not-yet-released pypi package, failing the initialization.

2. For dev version, it always pin to "latest" version in pypi. One may not be able to run an old dev version.

We should expose the interface to support install custom Python SDK, either assign a version number (like 2.57.0RC1) or provide a path of the tar ball.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner