Example usage for org.apache.lucene.analysis.shingle ShingleFilter setTokenSeparator

Introduction

In this page you can find the example usage for org.apache.lucene.analysis.shingle ShingleFilter setTokenSeparator.

Prototype

public void setTokenSeparator(String tokenSeparator) 

Source Link

Document

Sets the string to use when joining adjacent tokens to form a shingle

Usage

From source file:org.apache.solr.analysis.ShingleFilterFactory.java

License:Apache License

public ShingleFilter create(TokenStream input) {
    ShingleFilter r = new ShingleFilter(input, minShingleSize, maxShingleSize);
    r.setOutputUnigrams(outputUnigrams);
    r.setOutputUnigramsIfNoShingles(outputUnigramsIfNoShingles);
    r.setTokenSeparator(tokenSeparator);
    return r;//from   w w  w. j ava2 s .  c o m
}

---