Example usage for org.antlr.v4.runtime CommonTokenStream CommonTokenStream

Introduction

In this page you can find the example usage for org.antlr.v4.runtime CommonTokenStream CommonTokenStream.

Prototype

public CommonTokenStream(TokenSource tokenSource, int channel) 

Source Link

Document

Constructs a new CommonTokenStream using the specified token source and filtering tokens to the specified channel.

Usage

From source file:de.fuerstenau.gradle.commentstripper.CommentStripper.java

License:Open Source License

private void doStrip(Path fin, Path fout, Charset cs, DocType docType) throws IOException {
    try (Reader in = Files.newBufferedReader(fin, cs);
            OutputStream out = new BufferedOutputStream(Files.newOutputStream(fout))) {
        final ANTLRInputStream stream = new ANTLRInputStream(in);

        final CommonTokenStream tokens;
        if (docType == DocType.JAVA) {
            tokens = new CommonTokenStream(new GroovyLexer(stream), GroovyLexer.COMMENT);
        } else {//from w  w  w . j a v  a  2s  .  c o m
            tokens = new CommonTokenStream(new Java8Lexer(stream), Java8Lexer.COMMENTS);
        }

        TokenStreamRewriter rew = new TokenStreamRewriter(tokens);

        while (tokens.LA(1) != EOF) {
            final Token t = tokens.LT(1);
            if (cond.shouldStrip(t.getText())) {
                rew.delete(t);
            }
            tokens.consume();
        }

        out.write(rew.getText().getBytes(cs));
        out.flush();
    }
}

---