Support accessing OSS with proxy #18139
Conversation
|
Automated checks report:
Some checks failed. Please fix the reported issues and reply 'alluxio-bot, check this please' to re-run checks. |
Jackson-Wang-7
changed the title
|
Automated checks report:
All checks passed! |
| public static final PropertyKey UNDERFS_OSS_PROXY_PORT = | ||
| intBuilder(Name.UNDERFS_OSS_PROXY_PORT) | ||
| .setAlias("alluxio.underfs.oss.proxy.port") | ||
| .setDefaultValue(0) |
There was a problem hiding this comment.
does it need a default? port 0 means assign dynamically right? So I'm afraid it's not the best default to use? I think you will need to configure this anyway (there's an existing port the proxy uses, instead of for the alluxio proc to allocate). So maybe no need for a default?
There was a problem hiding this comment.
if the port is 0(or not > 0), we think the proxy configuration is not enabled, so we don't set proxy in the oss config.
| if (isProxyEnabled(alluxioConf)) { | ||
| ossClientConf.setProxyHost(getProxyHost(alluxioConf)); | ||
| ossClientConf.setProxyPort(getProxyPort(alluxioConf)); | ||
| ossClientConf.setProtocol(getProtocol()); |
There was a problem hiding this comment.
add a log here if proxy is used (so you know)
There was a problem hiding this comment.
good idea, done
| * returns value of http.proxyPort. Defaults to {@link this.proxyPort} | ||
| * if the system property is not set with a valid port number. | ||
| */ | ||
| private static int getProxyPortProperty() { |
There was a problem hiding this comment.
| private static int getProxyPortProperty() { | |
| private static int getProxyPortFromSystemProperty() { |
|
@jiacheliu3 thanks for your comment, I have fixed the description of the properties and added the log if proxy is enabled. |
|
alluxio-bot, merge this please |
What changes are proposed in this pull request?
Support accessing OSS through proxy by configuring alluxio properties or system properties.
Why are the changes needed?
When accessing OSS through a proxy, the OSS client cannot recognize the proxy configuration in system property and environment variables. So it has to proactively set proxy-related configurations in the configuration.
Does this PR introduce any user facing changes?
Please list the user-facing changes introduced by your change, including