Example usage for org.apache.hadoop.io.compress BZip2Codec createInputStream

Introduction

In this page you can find the example usage for org.apache.hadoop.io.compress BZip2Codec createInputStream.

Prototype

@Override
public CompressionInputStream createInputStream(InputStream in) throws IOException 

Source Link

Document

Create a CompressionInputStream that will read from the given input stream and return a stream for uncompressed data.

Usage

From source file:org.hadoop.tdg.TestPseudoHadoop.java

License:Apache License

@Test
public void writeAndReadBzipCompressed() throws IOException {
    BZip2Codec codec = new BZip2Codec();
    String ext = codec.getDefaultExtension();
    Path p = new Path(DST_FILE + ext);
    File f1 = new File(HOME_FILE);
    File f2 = new File(HOME_FILE + ext);
    //writing compressed to hdfs
    CompressionOutputStream cout = codec.createOutputStream(fs.create(p));
    IOUtils.copyBytes(new FileInputStream(f1), cout, 4096, false);
    Assert.assertTrue(/*from   w w  w . j  a v a 2  s .  co m*/
            fs.getFileStatus(p).getPath().equals(new Path(fs.getUri().toString(), p.toUri().toString())));

    //reading and checking if it's the same
    FSDataInputStream dis = fs.open(p);
    //doesn't work don't know why
    CompressionInputStream cin = codec.createInputStream(dis);
    IOUtils.copyBytes(dis, new FileOutputStream(f2), 4096, false);
    Files.equal(f1, f2);
}

---

From source file:org.icgc.dcc.submission.validation.first.io.FPVFileSystem.java

License:Open Source License

public void attemptBzip2Read(String fileName) throws IOException {
    // check the bzip2 header

    BZip2Codec codec = new BZip2Codec();

    @Cleanup/* w ww  . j a v  a  2  s. c  o m*/
    CompressionInputStream in = codec.createInputStream(submissionDirectory.open(fileName));

    // see if it can be read through
    byte[] buf = new byte[BUFFER_SIZE];
    while (in.read(buf) > 0) {
    }
}

---