Example usage for org.apache.hadoop.mapred MultiFileSplit getNumPaths

Introduction

In this page you can find the example usage for org.apache.hadoop.mapred MultiFileSplit getNumPaths.

Prototype

public int getNumPaths() 

Source Link

Document

Returns the number of Paths in the split

Usage

From source file:hitune.analysis.mapreduce.MultiSequenceFileReader.java

License:Apache License

/**
 * //from  w w w  .  j ava  2 s. c  om
 */
public MultiSequenceFileReader(Configuration conf, MultiFileSplit split) throws IOException {
    // TODO Auto-generated constructor stub
    this.split = split;
    this.conf = conf;
    paths = this.split.getPaths();
    fs = FileSystem.get(conf);
    totalLength = split.getLength();
    pos = 0;
    log.debug("total split number:" + split.getNumPaths());
    log.debug("open split:" + paths[0].toString());
    FileSplit filesplit = new FileSplit(paths[0], 0, split.getLength(0), (JobConf) conf);
    reader = new SequenceFileRecordReader<ChukwaRecordKey, ChukwaRecord>(conf, filesplit);
    if (reader == null) {
        log.warn("open split failed!");
    }
}

---