|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.mapred.lib.InputSampler<K,V>
public class InputSampler<K,V>
Utility for collecting samples and writing a partition file for
TotalOrderPartitioner
.
This class is copied from Hadoop 0.20.x release to make Mobius compatible in both Hadoop 0.20.X and 0.22. (see https://issues.apache.org/jira/browse/MAPREDUCE-4251)
This product is licensed under the Apache License, Version 2.0, available at http://www.apache.org/licenses/LICENSE-2.0. This product contains portions derived from Apache hadoop which is licensed under the Apache License, Version 2.0, available at http://hadoop.apache.org. © 2007 – 2012 eBay Inc., Evan Chiu, Woody Zhou, Neel Sundaresan
Nested Class Summary | |
---|---|
static class |
InputSampler.IntervalSampler<K,V>
Sample from s splits at regular intervals. |
static class |
InputSampler.RandomSampler<K,V>
Sample from random points in the input. |
static interface |
InputSampler.Sampler<K,V>
Interface to sample using an InputFormat . |
static class |
InputSampler.SplitSampler<K,V>
Samples the first n records from s splits. |
Constructor Summary | |
---|---|
InputSampler(org.apache.hadoop.mapred.JobConf conf)
|
Method Summary | ||
---|---|---|
org.apache.hadoop.conf.Configuration |
getConf()
|
|
int |
run(java.lang.String[] args)
Driver for InputSampler from the command line. |
|
void |
setConf(org.apache.hadoop.conf.Configuration conf)
|
|
static
|
writePartitionFile(org.apache.hadoop.mapred.JobConf job,
InputSampler.Sampler<K,V> sampler)
Write a partition file for the given job, using the Sampler provided. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public InputSampler(org.apache.hadoop.mapred.JobConf conf)
Method Detail |
---|
public org.apache.hadoop.conf.Configuration getConf()
getConf
in interface org.apache.hadoop.conf.Configurable
public void setConf(org.apache.hadoop.conf.Configuration conf)
setConf
in interface org.apache.hadoop.conf.Configurable
public static <K,V> void writePartitionFile(org.apache.hadoop.mapred.JobConf job, InputSampler.Sampler<K,V> sampler) throws java.io.IOException
TotalOrderPartitioner.getPartitionFile(org.apache.hadoop.mapred.JobConf)
.
java.io.IOException
public int run(java.lang.String[] args) throws java.lang.Exception
writePartitionFile(org.apache.hadoop.mapred.JobConf, org.apache.hadoop.mapred.lib.InputSampler.Sampler)
.
run
in interface org.apache.hadoop.util.Tool
java.lang.Exception
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |