com.intel.hadoop.graphbuilder.partition.mapreduce.edge
Class EdgeIngressMapper<VidType extends org.apache.hadoop.io.WritableComparable<VidType>,VertexData extends org.apache.hadoop.io.Writable,EdgeData extends org.apache.hadoop.io.Writable,KeyType extends IngressKeyType<VidType>,ValueType extends IngressValueType<VidType,VertexData,EdgeData>>
java.lang.Object
org.apache.hadoop.mapred.MapReduceBase
com.intel.hadoop.graphbuilder.partition.mapreduce.edge.EdgeIngressMapper<VidType,VertexData,EdgeData,KeyType,ValueType>
- Type Parameters:
VidType
- VertexData
- EdgeData
- KeyType
- ValueType
-
- All Implemented Interfaces:
- java.io.Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,KeyType,ValueType>
public class EdgeIngressMapper<VidType extends org.apache.hadoop.io.WritableComparable<VidType>,VertexData extends org.apache.hadoop.io.Writable,EdgeData extends org.apache.hadoop.io.Writable,KeyType extends IngressKeyType<VidType>,ValueType extends IngressValueType<VidType,VertexData,EdgeData>>
- extends org.apache.hadoop.mapred.MapReduceBase
- implements org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,KeyType,ValueType>
This mapper class maps edge and vertex list into intermediate key, value
pairs. Because mapper takes in edge and vertex at the same time, "!" is used
as a special character in the beginning of each vertex data to distinguish it
from edge data.
For each edge e = (u,v,edata) it assigns its partition using the ingress
algorithm defined by the JobConf
. After computing the partition id,
it generates 3 records:
- EdgeType value:
(pid, (u,v,edata))
- VertexType value:
(u, ( pid
, 0, 1))}
- VertexType value:
(v, ( pid
, 1, 0))}
For each vertex v = (v, vdata) it generates a VertexType value:
(v, vdata)
.
All EdgeType value with the same partition id is reduced into a local graph
partition. And all VertexType value with the same vertex id is reduced into a
vertex record.
Method Summary |
void |
configure(org.apache.hadoop.mapred.JobConf job)
|
void |
map(org.apache.hadoop.io.LongWritable key,
org.apache.hadoop.io.Text value,
org.apache.hadoop.mapred.OutputCollector<KeyType,ValueType> out,
org.apache.hadoop.mapred.Reporter reporter)
|
Methods inherited from class org.apache.hadoop.mapred.MapReduceBase |
close |
Methods inherited from class java.lang.Object |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface java.io.Closeable |
close |
EdgeIngressMapper
public EdgeIngressMapper()
configure
public void configure(org.apache.hadoop.mapred.JobConf job)
- Specified by:
configure
in interface org.apache.hadoop.mapred.JobConfigurable
- Overrides:
configure
in class org.apache.hadoop.mapred.MapReduceBase
map
public void map(org.apache.hadoop.io.LongWritable key,
org.apache.hadoop.io.Text value,
org.apache.hadoop.mapred.OutputCollector<KeyType,ValueType> out,
org.apache.hadoop.mapred.Reporter reporter)
throws java.io.IOException
- Specified by:
map
in interface org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,KeyType extends IngressKeyType<VidType>,ValueType extends IngressValueType<VidType,VertexData,EdgeData>>
- Throws:
java.io.IOException