Example usage for org.apache.lucene.analysis.cn.smart WordType FULLWIDTH_NUMBER

Introduction

In this page you can find the example usage for org.apache.lucene.analysis.cn.smart WordType FULLWIDTH_NUMBER.

Prototype

int FULLWIDTH_NUMBER

To view the source code for org.apache.lucene.analysis.cn.smart WordType FULLWIDTH_NUMBER.

Click Source Link

Document

Full-Width Alphanumeric

Usage

From source file:com.churvey.graduate.chinese.WordSegmenter.java

License:Apache License

/**
 * Process a {@link SegToken} so that it is ready for indexing.
 * /*  w w  w. ja  va2  s . com*/
 * This method calculates offsets and normalizes the token with {@link SegTokenFilter}.
 * 
 * @param st input {@link SegToken}
 * @param sentence associated Sentence
 * @param sentenceStartOffset offset into sentence
 * @return Lucene {@link SegToken}
 */
public SegToken convertSegToken(SegToken st, String sentence, int sentenceStartOffset) {

    switch (st.wordType) {
    case WordType.STRING:
    case WordType.NUMBER:
    case WordType.FULLWIDTH_NUMBER:
    case WordType.FULLWIDTH_STRING:
        st.charArray = sentence.substring(st.startOffset, st.endOffset).toCharArray();
        break;
    default:
        break;
    }

    st = tokenFilter.filter(st);
    st.startOffset += sentenceStartOffset;
    st.endOffset += sentenceStartOffset;
    return st;
}

---