Example usage for org.apache.lucene.analysis.core TypeTokenFilter TypeTokenFilter

Introduction

In this page you can find the example usage for org.apache.lucene.analysis.core TypeTokenFilter TypeTokenFilter.

Prototype

public TypeTokenFilter(TokenStream input, Set<String> stopTypes, boolean useWhiteList) 

Source Link

Document

Create a new TypeTokenFilter .

Usage

From source file:org.elasticsearch.analysis.common.KeepTypesFilterFactory.java

License:Apache License

@Override
public TokenStream create(TokenStream tokenStream) {
    return new TypeTokenFilter(tokenStream, keepTypes, true);
}

---

From source file:org.fastcatsearch.plugin.analysis.jp.JapaneseAnalyzer.java

License:Apache License

@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
    // TODO Auto-generated method stub

    Set<String> emptyStop = new HashSet<String>();
    JapaneseTokenizer tokenizer = new JapaneseTokenizer(reader, this.japDictionary, null, Mode.NORMAL, true);
    TokenStream result = new TypeTokenFilter(true, tokenizer, emptyStop);
    return new TokenStreamComponents(tokenizer, result);
}

---