Skip to content

Commit

Permalink
[docs] [SPARK-4820] Spark build encounters "File name too long" on so…
Browse files Browse the repository at this point in the history
…me encrypted filesystems

Added a note instructing users how to build Spark in an encrypted file system.

Author: Theodore Vasiloudis <tvas@sics.se>

Closes apache#5041 from thvasilo/patch-2 and squashes the following commits:

09d890b [Theodore Vasiloudis] Workaroung for buiding in an encrypted filesystem
  • Loading branch information
Theodore Vasiloudis authored and srowen committed Mar 17, 2015
1 parent 005d1c5 commit e26db9b
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions docs/building-spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,18 @@ build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package

Other build examples can be found below.

**Note:** When building on an encrypted filesystem (if your home directory is encrypted, for example), then the Spark build might fail with a "Filename too long" error. As a workaround, add the following in the configuration args of the `scala-maven-plugin` in the project `pom.xml`:

<arg>-Xmax-classfile-name</arg>
<arg>128</arg>

and in `project/SparkBuild.scala` add:

scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"),

to the `sharedSettings` val. See also [this PR](https://github.com/apache/spark/pull/2883/files) if you are unsure of where to add these lines.


# Setting up Maven's Memory Usage

You'll need to configure Maven to use more memory than usual by setting `MAVEN_OPTS`. We recommend the following settings:
Expand Down

0 comments on commit e26db9b

Please sign in to comment.