Skip to content

Write Hadoop MapReduce in high-level API. Inspired by Apache Spark

Notifications You must be signed in to change notification settings

fanx-dev/EasyMapReduce

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

EasyMapReduce

Write Hadoop MapReduce in high-level API. Inspired by Apache Spark

Word Count

  Void main() {
    init
    ds := c.load(input).map{ it.split(' ').join("\n") }.reduce(0)|Int r,v|{ r + 1 }
    ds.save(output)
  }

Run

cd demo
fan wordCount.fan

Config

set 'HADOOP_HOME' env vars

Debug

Running in a local simulation env.

fan wordCount.fan -debug

How it works

Show more detail by '-debug_show' args.

fan wordCount.fan -debug_show

Passing Hadoop Args

fan wordCount.fan -hadoop '-files ./dict,./data'

More Hadoop Streaming Args

About

Write Hadoop MapReduce in high-level API. Inspired by Apache Spark

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages