Example usage for org.apache.lucene.analysis.cjk CJKBigramFilter CJKBigramFilter

Introduction

In this page you can find the example usage for org.apache.lucene.analysis.cjk CJKBigramFilter CJKBigramFilter.

Prototype

public CJKBigramFilter(TokenStream in) 

Source Link

Document

Calls CJKBigramFilter#CJKBigramFilter(TokenStream,int) CJKBigramFilter(in, HAN | HIRAGANA | KATAKANA | HANGUL)

Usage

From source file:com.github.buzztaiki.lucene.lastuni.CJKLastUniGramAnalyzer.java

License:Apache License

@Override
protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
    final Tokenizer source = new StandardTokenizer(matchVersion, reader);
    TokenStream result = new CJKWidthFilter(source);
    result = new LowerCaseFilter(matchVersion, result);
    result = new CJKBigramFilter(result);
    result = new CJKLastUniGramFilter(result, tokenizeLastUni);
    return new TokenStreamComponents(source, new StopFilter(matchVersion, result, stopwords));
}

---

From source file:com.github.buzztaiki.lucene.lastuni.CJKLastUniGramFilterForSearchTest.java

License:Apache License

private static CJKLastUniGramFilter newFilter(String str) {
    TokenStream ts = new StandardTokenizer(TEST_VERSION_CURRENT, new StringReader(str));
    ts = new CJKBigramFilter(ts);
    return new CJKLastUniGramFilter(ts, false);
}

---

From source file:com.github.buzztaiki.lucene.lastuni.CJKLastUniGramFilterTest.java

License:Apache License

private static CJKLastUniGramFilter newFilter(String str) {
    TokenStream ts = new StandardTokenizer(TEST_VERSION_CURRENT, new StringReader(str));
    ts = new CJKBigramFilter(ts);
    return new CJKLastUniGramFilter(ts, true);
}

---