Example usage for org.apache.lucene.analysis.pattern PatternTokenizer PatternTokenizer

Introduction

In this page you can find the example usage for org.apache.lucene.analysis.pattern PatternTokenizer PatternTokenizer.

Prototype

public PatternTokenizer(Pattern pattern, int group) 

Source Link

Document

creates a new PatternTokenizer returning tokens from group (-1 for split functionality)

Usage

From source file:org.codelibs.elasticsearch.index.analysis.PatternAnalyzer.java

License:Apache License

@Override
protected TokenStreamComponents createComponents(String s) {
    final Tokenizer tokenizer = new PatternTokenizer(pattern, -1);
    TokenStream stream = tokenizer;//w w  w  .  j  av a  2s  .c  om
    if (lowercase) {
        stream = new LowerCaseFilter(stream);
    }
    if (stopWords != null) {
        stream = new StopFilter(stream, stopWords);
    }
    return new TokenStreamComponents(tokenizer, stream);
}

---

From source file:org.elasticsearch.analysis.common.PatternTokenizerFactory.java

License:Apache License

@Override
public Tokenizer create() {
    return new PatternTokenizer(pattern, group);
}

---