Example usage for org.apache.hadoop.mapreduce Partitioner subclass-usage

Introduction

In this page you can find the example usage for org.apache.hadoop.mapreduce Partitioner subclass-usage.

Usage

From source file com.synerzip.analytics.commoncrawl.googleads.counter.GoogleAdsCounterPartitioner.java

/**
 * Google Ad Partitioner, which Partitions the map emitted values based on ad type alone.
 * @author Rohit Ghatol
 *
 */
public class GoogleAdsCounterPartitioner extends Partitioner<GoogleAdsInfo, IntWritable> {

From source file com.twitter.algebra.matrix.format.RowPartitioner.java

/**
 * We enforce total order in MapReudce output via {@link RowPartitioner} and
 * {@link MatrixOutputFormat}. Each matrix job needs to derive its partitioner
 * from {@link RowPartitioner} and set its output format from
 * {@link MatrixOutputFormat}. The output of each reducer must be also locally
 * sorted.

From source file com.twitter.algebra.nmf.RowColPartitioner.java

/**
 * An example of how to extend {@link RowColPartitioner} is
 * {@link ElementRowColPartitioner} class.
 * 
 * @author myabandeh
 *

From source file com.veera.secondarysort.demo2.NaturalKeyPartitioner.java

/**
 * Partitions key based on "natural" key of {@link StockKey} (which
 * is the symbol).
 * @author Jee Vang
 *
 */

From source file com.vinod.hadoop.mapreduce.example.secondarysort.StockKeyPartitioner.java

/**
 * Partitions key based on "natural" key of {@link StockKey} (which
 * is the symbol).
 * @author Jee Vang
 *
 */

From source file com.yahoo.ycsb.bulk.hbase.RangePartitioner.java

/**
 * Hadoop partitioner that uses ranges, and optionally sub-bins based on
 * hashing.
 */
public class RangePartitioner extends Partitioner<Text, Writable> implements Configurable {
    private static final String PREFIX = RangePartitioner.class.getName();

From source file crunch.MaxTemperature.java

    public class StationPartitioner extends Partitioner<LongWritable, Text> {

        private NcdcRecordParser parser = new NcdcRecordParser();

        @Override
        public int getPartition(LongWritable key, Text value, int numPartitions) {

From source file csc555.ebratt.depaul.edu.GroupByCountPairPartitioner.java

/**
 * GroupByCountPairPartitioner is a custom partitioner used in a hadoop job to
 * ensure that GroupByCountPair objects are sent to the same reducer based on
 * their groupBy values. This is so that each reducer works on the same set of
 * groupBy key values and can perform the sorting at that level.
 *

From source file datafu.hourglass.jobs.TimePartitioner.java

/**
 * A partitioner used by {@link AbstractPartitionPreservingIncrementalJob} to limit the number of named outputs
 * used by each reducer.
 * 
 * <p>
 * The purpose of this partitioner is to prevent a proliferation of small files created by {@link AbstractPartitionPreservingIncrementalJob}.

From source file de.tudarmstadt.ukp.dkpro.bigdata.collocations.GramKeyPartitioner.java

/**
 * Partition GramKeys based on their Gram, ignoring the secondary sort key so that all GramKeys with the same
 * gram are sent to the same partition.
 */
public final class GramKeyPartitioner extends Partitioner<GramKey, Gram> {