Sunday 13 December 2015

How will you write a custom partitioner for a Hadoop job?

To have Hadoop use a custom partitioner you will have to do minimum the following three:
 - Create a new class that extends Partitioner Class
 - Override method getPartition
 - In the wrapper that runs the Mapreduce, either
 - Add the custom partitioner to the job programmatically using method set Partitioner Class or – add the custom partitioner to the job as a config file (if your wrapper reads from config file or oozie)

2 comments: