Java String Tokenize parseNMTokens(String nmTokens)

Here you can find the source of parseNMTokens(String nmTokens)

Description

parse NM Tokens

License

Open Source License

Declaration

public static List parseNMTokens(String nmTokens) 

Method Source Code

//package com.java2s;

import java.util.*;

public class Main {
    public static List parseNMTokens(String nmTokens) {
        StringTokenizer strTok = new StringTokenizer(nmTokens, " ");
        List tokens = new Vector();

        while (strTok.hasMoreTokens()) {
            tokens.add(strTok.nextToken());
        }//from   w w  w .j a v  a  2  s . c  om

        return tokens;
    }
}

Related

  1. hasSuffix(String _token)
  2. isPOSTag(String token)
  3. isStringFunction(String token)
  4. isToken(String sentence, String searchWord)
  5. maxTokenLength(String s)
  6. parseString(String in, String token)
  7. readTokens(String line)
  8. removeEmptyStrings(String[] tokens)
  9. removeMatchingRegex(String regex, String replacement, String[] tokens, boolean removeEmpty)