Starting

Starting

You can use Ruby Spark via interactive shell

$ ruby-spark pry

Or on existing project

require 'ruby-spark'
Spark.start

If you want configure Spark first. See configurations page for more details.

require 'ruby-spark'

Spark.load_lib(spark_home)
Spark.config do
  set_app_name 'RubySpark'
  set 'spark.ruby.batch_size', 100
  set 'spark.ruby.serializer', 'oj'
end
Spark.start

Context

Spark.context
Spark.sc

Stopping

Context is automatically stopped when program is ended but you can also use this

# Stop context, clear config and kill all workers
Spark.stop