com.ebay.erl.mobius.core.mapred
Class MobiusMultiInputs
java.lang.Object
com.ebay.erl.mobius.core.mapred.MobiusMultiInputs
public class MobiusMultiInputs
- extends java.lang.Object
MultipleInputs
uses the input path to find the corresponding
Map class. Mobius need to find the TupleCriterion
,
schema, or projection column for a given Dataset
,
and two different Dataset
elements in a Mobius job
might be processed by same type of AbstractMobiusMapper
. For example, joining two
different CSV dataset, both Dataset
are processed by
TSVMapper
.
This class is used to build the path to dataset mapping, or vice versa, so
the mapper can use ConfigureConstants.CURRENT_DATASET_ID
to
determine the projection columns, filters, and so on.
This class is used along with MobiusDelegatingInputFormat
This product is licensed under the Apache License, Version 2.0,
available at http://www.apache.org/licenses/LICENSE-2.0.
This product contains portions derived from Apache hadoop which is
licensed under the Apache License, Version 2.0, available at
http://hadoop.apache.org.
© 2007 – 2012 eBay Inc., Evan Chiu, Woody Zhou, Neel Sundaresan
- See Also:
MobiusDelegatingInputFormat
Method Summary |
static void |
addInputPath(org.apache.hadoop.mapred.JobConf conf,
org.apache.hadoop.fs.Path anInput,
java.lang.Class<? extends org.apache.hadoop.mapred.InputFormat> inputFormatClass,
java.lang.Class<? extends AbstractMobiusMapper> mapperClass,
java.lang.String datasetID,
org.apache.hadoop.fs.FileSystem fs)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
MobiusMultiInputs
public MobiusMultiInputs()
addInputPath
public static void addInputPath(org.apache.hadoop.mapred.JobConf conf,
org.apache.hadoop.fs.Path anInput,
java.lang.Class<? extends org.apache.hadoop.mapred.InputFormat> inputFormatClass,
java.lang.Class<? extends AbstractMobiusMapper> mapperClass,
java.lang.String datasetID,
org.apache.hadoop.fs.FileSystem fs)
throws java.io.IOException
- Throws:
java.io.IOException