Example usage for org.apache.hadoop.mapreduce.lib.output FileOutputCommitter subclass-usage

Introduction

In this page you can find the example usage for org.apache.hadoop.mapreduce.lib.output FileOutputCommitter subclass-usage.

Usage

From source file alluxio.hadoop.mapreduce.KeyValueOutputCommitter.java

/**
 * Extension of {@link FileOutputCommitter} where creating, completing, or deleting a {@link
 * KeyValueSystem} in different phases of a job's or task's lifecycle is considered.
 * <p>
 * This committer must be used along with {@link KeyValueOutputFormat} to merge the key-value stores
 * created by each Reducer into one key-value store under the MapReduce output directory.

From source file boa.io.BoaOutputCommitter.java

/**
 * A {@link FileOutputCommitter} that stores the job results into a database.
 * 
 * @author rdyer
 */
public class BoaOutputCommitter extends FileOutputCommitter {

From source file co.cask.cdap.internal.app.runtime.batch.dataset.partitioned.DynamicPartitioningOutputCommitter.java

/**
 * An OutputCommitter which creates partitions in a configured PartitionedFileSet dataset for all of the partitions
 * that were written to by a DynamicPartitioningOutputFormat
 * It enables this by having each job write to a job-specific temporary path within that output directory.
 * Then, upon commitJob, it moves the files to the final, parent directory if the final output directories do not
 * already exist.

From source file com.github.sadikovi.hadoop.riff.RiffOutputCommitter.java

/**
 * Riff output committer.
 * Provides methods to save metadata for the output folder.
 *
 * Note: Spark SQL does not use custom output committer when appending data to the existing table,
 * in our case it is okay, since we store only static information that does not change with writing

From source file com.inmobi.conduit.distcp.tools.mapred.CopyCommitter.java

public class CopyCommitter extends FileOutputCommitter {
    private static final Log LOG = LogFactory.getLog(CopyCommitter.class);

    private final TaskAttemptContext taskAttemptContext;

    /**

From source file com.practicalHadoop.outputformat.MultpleDirectories.FileOutputCommitter.java

public class FileOutputCommitter extends org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter {

    private static final Log LOG = LogFactory.getLog(FileOutputCommitter.class);
    private static Map<String, FileOutputCommitter> committers = new HashMap<String, FileOutputCommitter>();

    /**

From source file gobblin.compaction.mapreduce.avro.AvroKeyCompactorOutputCommitter.java

/**
 * Class used with {@link MRCompactorAvroKeyDedupJobRunner} to rename files as they
 * are being committed. In addition to moving files from their working directory to
 * the commit output directory, the files are named to include a timestamp and a
 * count of how many records the file contains, in the format
 * {recordCount}.{timestamp}.avro.

From source file org.apache.carbondata.hadoop.api.CarbonOutputCommitter.java

/**
 * Outputcommitter which manages the segments during loading.It commits segment information to the
 * tablestatus file upon success or fail.
 */
public class CarbonOutputCommitter extends FileOutputCommitter {

From source file org.apache.falcon.hive.mapreduce.CopyCommitter.java

/**
 * Copy committer class.
 */
public class CopyCommitter extends FileOutputCommitter {

    private static final Logger LOG = LoggerFactory.getLogger(CopyCommitter.class);

From source file org.apache.gobblin.compaction.mapreduce.avro.AvroKeyCompactorOutputCommitter.java

/**
 * Class used with {@link MRCompactorAvroKeyDedupJobRunner} to rename files as they
 * are being committed. In addition to moving files from their working directory to
 * the commit output directory, the files are named to include a timestamp and a
 * count of how many records the file contains, in the format
 * {recordCount}.{timestamp}.avro.

• «
• 1
• 2
• »