Skip to content

Commit

Permalink
Revise disableOutputSpecValidation() comment to not refer to Spark St…
Browse files Browse the repository at this point in the history
…reaming.
  • Loading branch information
JoshRosen committed Jan 3, 2015
1 parent e581d17 commit bf9094d
Showing 1 changed file with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1130,10 +1130,10 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

private[spark] object PairRDDFunctions {
val RECORDS_BETWEEN_BYTES_WRITTEN_METRIC_UPDATES = 256

/**
* Used by Spark Streaming in order to bypass the `spark.hadoop.validateOutputSpecs` checks
* for save actions launched by Spark Streaming, since the validation may break Spark Streaming's
* ability to recover from checkpoints. See SPARK-4835 for more details.
* Allows for the `spark.hadoop.validateOutputSpecs` checks to be disabled on a case-by-case
* basis; see SPARK-4835 for more details.
*/
val disableOutputSpecValidation: DynamicVariable[Boolean] = new DynamicVariable[Boolean](false)
}

0 comments on commit bf9094d

Please sign in to comment.