Example usage for org.apache.hadoop.mapred FixedLengthInputFormat FIXED_RECORD

Introduction

In this page you can find the example usage for org.apache.hadoop.mapred FixedLengthInputFormat FIXED_RECORD_LENGTH.

Prototype

String FIXED_RECORD_LENGTH

To view the source code for org.apache.hadoop.mapred FixedLengthInputFormat FIXED_RECORD_LENGTH.

Click Source Link

Usage

From source file:ca.sparkera.adapters.mainframe.CobolSerdeUtils.java

License:Apache License

/**
 * Determine the layout to that's been provided for cobol serde work.
 * //from  w  w  w.  ja va 2  s .co  m
 * @param properties
 *            containing a key pointing to the layout, one way or another
 * @return layout to use while serdeing the avro file
 * @throws IOException
 *             if error while trying to read the layout from another
 *             location
 * @throws CobolSerdeException
 *             if unable to find a layout or pointer to it in the properties
 */
public static String determineLayoutOrThrowException(Configuration conf, Properties properties)
        throws IOException, CobolSerdeException {

    //For fixed length record get length of the file 
    String fixedRecordLength = properties.getProperty(CobolTableProperties.FB_LENGTH.getPropName());
    if (fixedRecordLength != null) {
        conf.setInt(FixedLengthInputFormat.FIXED_RECORD_LENGTH, Integer.parseInt(fixedRecordLength));
    }

    String layoutString = properties.getProperty(CobolTableProperties.LAYOUT_LITERAL.getPropName());
    if (layoutString != null && !layoutString.equals(LAYOUT_NONE))
        return CobolSerdeUtils.getLayoutFor(layoutString);

    //For testing purpose
    layoutString = properties.getProperty(CobolTableProperties.LAYOUT_TEST.getPropName());
    if (layoutString != null) {
        return readFile(layoutString, Charset.defaultCharset());
    }

    // Try pulling directly from URL
    layoutString = properties.getProperty(CobolTableProperties.LAYOUT_URL.getPropName());
    if (layoutString == null || layoutString.equals(LAYOUT_NONE))
        throw new CobolSerdeException(EXCEPTION_MESSAGE);

    try {
        String s = getLayoutFromFS(layoutString, conf);
        if (s == null) {
            // in case layout is not a file system

            return CobolSerdeUtils.getLayoutFor(new URL(layoutString).openStream());
        }
        return s;
    } catch (IOException ioe) {
        throw new CobolSerdeException("Unable to read layout from given path: " + layoutString, ioe);
    } catch (URISyntaxException urie) {
        throw new CobolSerdeException("Unable to read layout from given path: " + layoutString, urie);
    }
}

Example usage for org.apache.hadoop.mapred FixedLengthInputFormat FIXED_RECORD_LENGTH

Introduction

Prototype

Usage