Skip to content

Remote shuffle service for Apache Spark to store shuffle data on remote servers.

License

Notifications You must be signed in to change notification settings

datapunchorg/RemoteShuffleService

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This project is a fork of Uber Remote Shuffle Service, with extra enhancement to make it work well in Kuberentes environment.

The master branch of this project is for Spark 2.4.x and is not recommended to use.

Recommend using k8s-spark-3.1 branch and k8s-spark-3.2 branch.

See README in k8s-spark-3.2 branch for details.

About

Remote shuffle service for Apache Spark to store shuffle data on remote servers.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Languages

  • Java 84.8%
  • Scala 15.2%