|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectcom.intel.hadoop.graphbuilder.preprocess.mapreduce.EdgeTransformMR
public class EdgeTransformMR
This MapReduce Class applies user defined "Reduce" and "Apply" functional on
edges that share the same source or target vertex, determined by the
reduceEndPoint
parameter. The Reduce functional computes a sufficient
statistics on a list of edges, and the Apply functional applies it back to
each edge in the list.
For example, the tf*idf http://en.wikipedia.org/wiki/Tf*idf
transformation can be done with two passes on a doc-word count graph.
First pass computes the term frequency (tf):
Second pass computes the final tfidf:
Input directory: list of edges. Output directory: $outputdir contains list of transformed edges.
TransformToTFIDF
Field Summary | |
---|---|
static boolean |
SOURCE
|
static boolean |
TARGET
|
Constructor Summary | |
---|---|
EdgeTransformMR(boolean reduceEndPoint,
GraphParser graphparser,
FieldParser vidparser,
FieldParser edataparser)
Create a EdgeTransform Job with reduceEndPoint and parsers. |
Method Summary | |
---|---|
org.apache.hadoop.mapred.JobConf |
getConf()
|
void |
run(java.lang.String inputpath,
java.lang.String outputpath)
|
void |
setFunctionClass(java.lang.Class reducefunc,
java.lang.Class applyfunc)
Set the reduce and apply function. |
void |
setKeyValueClass(java.lang.Class keyClass,
java.lang.Class valClass)
Set the intermediate key value class. |
void |
setUserOptions(java.util.HashMap<java.lang.String,java.lang.String> userOpts)
Set the user defined options. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final boolean SOURCE
public static final boolean TARGET
Constructor Detail |
---|
public EdgeTransformMR(boolean reduceEndPoint, GraphParser graphparser, FieldParser vidparser, FieldParser edataparser)
reduceEndPoint
and parsers.
reduceEndPoint
- {SOURCE, TARGET} the edge end point to reduce on.graphparser
- vidparser
- edataparser
- Method Detail |
---|
public void setFunctionClass(java.lang.Class reducefunc, java.lang.Class applyfunc)
reducefunc
- applyfunc
- public void setKeyValueClass(java.lang.Class keyClass, java.lang.Class valClass)
keyClass
- valClass
- public void setUserOptions(java.util.HashMap<java.lang.String,java.lang.String> userOpts)
userOpts
- a Map of option key value pairs.public org.apache.hadoop.mapred.JobConf getConf()
public void run(java.lang.String inputpath, java.lang.String outputpath) throws java.io.IOException
java.io.IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |