Update building-with-maven.md

[loachli]

Support building spark under http proxy environment

[AmplabJenkins]

Can one of the admins verify this patch?

[loachli]

I have create SPARK-3191(https://issues.apache.org/jira/browse/SPARK-3191) to describe this issues detailly

[srowen]

I'm not sure this resolves proxy problems. It may resolve a particular issue wherein your network proxy is breaking SSL connections, but is not how you configure proxies. See http://maven.apache.org/guides/mini/guide-proxies.html

Attacks are very rare, but if someone were trying to inject a bad binary into your build, this would invite users to explicitly ignore that warning sign. So I disagree that this is something all users should use when using a proxy.

The yarn profile and so on are not related, just the two maven.wagon settings. The error message you quote does not contain the type of failure you would see, which is a "could not resolve dependencies" error. Finally, there are punctuation and capitalization problems, like "http".

I don't think this should be added.

[pwendell]

Hey @loachli - thanks for looking into this. I don't think we can advise users to disable security settings for their maven build. Does your proxy support HTTPS?

[loachli]

hey @pwendell , thanks for your comments- Yes, my proxy support https.

I had used no-proxy open environment before. In order to support more people using spark, I have to move spark environment into my company's inner environment. For security reason, I have to use http-proxy provided by my company to access network.
When I used spark in my company's inner environment, I could not compile spark successfully.
Because the maven's error hint was not obvious, I spent much time solving this problem.

You can find the definition of these two parameters in http://maven.apache.org/wagon/wagon-providers/wagon-http/
"maven.wagon.http.ssl.insecure = true/false (default false), enable/disable use of relaxed ssl check for user generated certificates.
maven.wagon.http.ssl.allowall = true/false (default false), enable/disable match of the server's X.509 certificate with hostname. If disabled, a browser like check will be used."

I believe, this issue will be met by others in the future when they use http-proxy environment.

I still think we could add this hint to the document because the attack is very rare when someone use pom.xml provided by spark. One optional way is that I could add risk warnig when using these parameters. Do you agree?

[loachli]

Hey @srowen, thanks a lot for your detailed comment.
this is my configuration

lzb
true
http
user
password
proxy.company.com
8080
*.company.com

This configuration is same as the content in the link provided by you
I agree with you that the attack is very rare, especially when I use pom.xml provided by spark

If my suggestion could be accepted, I can reorganized my documents to meet requirements

[AmplabJenkins]

Can one of the admins verify this patch?

[loachli]

I have reorganized my documents to meet requriments. If you don't set this, make-distribution.sh cann't be run properly too

[pwendell]

Can you create a JIRA for this issue? I'm going to re-word this a bit when I merge it, I think it's fine to say that certain users have reported issues building behind proxies without disabling security - and have a clear warning that this, indeed, disables security. Anyways I can fix-up the wording if you create a JIRA for it.

[loachli]

I have created SPARK-3191(https://issues.apache.org/jira/browse/SPARK-3191) for it. Do you think it is enough for this PR?

[SparkQA]

Can one of the admins verify this patch?

[pwendell]

Hi there - I thought a bit more about this and I think we probably shouldn't explicitly tell users to disable security settings. I've never heard of a user report this issue before - so it doesn't seem super common, even for users building behind proxies (which I know many people do). However, let's do this. If we have other users report this issue again, we can add something to the docs and we'll use this patch as a starting point. So let's close this issue for now and re-open it later if necessary.