Java String Tokenize tokenize(String s, char separator, int maxTokens)

Description

Given a string, return an array of tokens.

License

Open Source License

Parameter

Parameter	Description
s	the string to tokenize.
separator	the separator char.
maxTokens	the maxmimum number of tokens returned. If the max is reached, the remaining part of s is appended to the end of the last token.

Return

an array of tokens.

Declaration

public static String[] tokenize(String s, char separator, int maxTokens) 

Method Source Code

//package com.java2s;
/*/*from w  w  w.  j a v a  2s. c o m*/
 * @(#)Util.java   1.62 04/06/27
 *
 * Copyright 2004 Sun Microsystems, Inc. All rights reserved.
 * SUN PROPRIETARY/CONFIDENTIAL. Use is subject to license terms.
 */

import java.util.*;

public class Main {
    /**
     * Given a string, return an array of tokens.  The separator can be escaped 
     * with the '\' character.  The '\' character may also be escaped by the
     * '\' character.
     * 
     * @param s         the string to tokenize.
     * @param separator the separator char.
     * @param maxTokens the maxmimum number of tokens returned.  If the
     *                  max is reached, the remaining part of s is appended
     *                  to the end of the last token.
     * 
     * @return an array of tokens.
     */
    public static String[] tokenize(String s, char separator, int maxTokens) {
        List tokens = new ArrayList();
        StringBuilder token = new StringBuilder();
        boolean prevIsEscapeChar = false;
        for (int i = 0; i < s.length(); i += Character.charCount(i)) {
            int currentChar = s.codePointAt(i);
            if (prevIsEscapeChar) {
                // Case 1:  escaped character
                token.appendCodePoint(currentChar);
                prevIsEscapeChar = false;
            } else if (currentChar == separator && tokens.size() < maxTokens - 1) {
                // Case 2:  separator
                tokens.add(token.toString());
                token = new StringBuilder();
            } else if (currentChar == '\\') {
                // Case 3:  escape character
                prevIsEscapeChar = true;
            } else {
                // Case 4:  regular character
                token.appendCodePoint(currentChar);
            }
        }
        if (token.length() > 0) {
            tokens.add(token.toString());
        }
        return (String[]) tokens.toArray(new String[] {});
    }
}

Related

1. tokenize(String s)
2. tokenize(String s)
3. tokenize(String s)
4. tokenize(String s, char separator)
5. tokenize(String s, char separator, int maxTokens)
6. tokenize(String s, String delimiters)
7. tokenize(String s1)
8. Tokenize(String sent)
9. tokenize(String sentence)