Java String Extract extract_tokens(String in_string, String delimiters)

Here you can find the source of extract_tokens(String in_string, String delimiters)

Description

Produce an array of string "tokens" that are present in the specified string and separated by the specified delimiters.

License

Open Source License

Parameter

Parameter Description
in_string the string from which the tokens are to be extracted
delimiters the string containing the delimiters, eg: " ,;:\t\n\r\f"

Return

An array of strings containing the tokens

Declaration

public static String[] extract_tokens(String in_string, String delimiters) 

Method Source Code

//package com.java2s;
/*//from w ww . ja  v  a2s  .  c om
 * File: StringUtil.java
 *
 * Copyright (C) 2000, Dennis Mikkelson
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * as published by the Free Software Foundation; either version 2
 * of the License, or (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this library; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307, USA.
 *
 * Contact : Dennis Mikkelson <mikkelsond@uwstout.edu>
 *           Department of Mathematics, Statistics and Computer Science
 *           University of Wisconsin-Stout
 *           Menomonie, WI 54751, USA
 *
 * This work was supported by the Intense Pulsed Neutron Source Division
 * of Argonne National Laboratory, Argonne, IL 60439-4845, USA.
 *
 * For further information, see <http://www.pns.anl.gov/ISAW/>
 *
 * Modified:
 *
 *  $Log$
 *  Revision 1.30  2004/03/11 22:13:15  millermi
 *  - Changed package names and replaced SharedData with
 *    SharedMessages class.
 *
 *  Revision 1.29  2004/01/30 02:43:04  bouzekc
 *  Added method to pad a String on the left with spaces.
 *
 *  Revision 1.28  2004/01/22 01:21:55  bouzekc
 *  Reinserted check for null String argument.
 *
 *  Revision 1.27  2004/01/21 17:48:32  bouzekc
 *  Backwards-compatible with previous version; if the splitter for split()
 *  is null, it is treated as a space separator.
 *
 *  Revision 1.26  2004/01/21 17:43:24  bouzekc
 *  Removed unused local variables and unused imports.
 *
 *  Revision 1.25  2004/01/21 17:42:06  bouzekc
 *  Now uses Java 1.4's split() method for this class's split() method.
 *
 *  Revision 1.24  2003/12/18 13:09:30  rmikk
 *  Added a new method, toString( object, boolean), which displays ALL the
 *    members of an array or Collection if the second argument is true.
 *
 *  Revision 1.23  2003/10/22 20:28:46  rmikk
 *  Fixed javadoc error
 *
 *  Revision 1.22  2003/10/16 00:35:41  dennis
 *  Partly fixed javadocs to build cleanly with jdk 1.4.2
 *  Line 433 @link MAX_ARRAY_DEPTH reference still not found
 *
 *  Revision 1.21  2003/10/14 16:22:12  dennis
 *  Fixed bug in trim( StringBuffer ) method.  Now checks that string is
 *  still non-empty while removing characters.
 *
 *  Revision 1.20  2003/07/07 14:10:17  bouzekc
 *  Smarter parsing in getNumOccurrences().
 *
 *  Revision 1.19  2003/07/07 14:05:34  bouzekc
 *  Added method to find the number of occurrences of one
 *  String within another.
 *
 *  Revision 1.18  2003/06/18 19:32:40  pfpeterson
 *  Implemented toString(Object) which will create a string containing
 *  the value of an object.
 *
 *  Revision 1.17  2003/06/17 15:28:58  pfpeterson
 *  Added split method for breaking a string into a string[].
 *
 *  Revision 1.16  2003/02/13 20:58:11  pfpeterson
 *  Deprecated fixSeparator and renamed the method to setFileSeparator.
 *
 *  Revision 1.15  2003/02/06 16:20:46  pfpeterson
 *  Fixed small bug in getBoolean(StringBuffer) which did not remove
 *  the parsed portion of the StringBuffer.
 *
 *  Revision 1.14  2003/02/06 15:22:04  pfpeterson
 *  Pulled out the functionality to determine the next space for StringBuffers
 *  into a private method. Also now understands tabs as spaces.
 *
 *  Revision 1.13  2003/02/05 19:30:26  pfpeterson
 *  Added methods to get a boolean value from a StringBuffer, updated
 *  some documentation, and made getFloat and getInt explicitly throw
 *  NumberFormatException (which does not need to be caught).
 *
 *  Revision 1.12  2002/11/27 23:23:49  pfpeterson
 *  standardized header
 *
 *  Revision 1.11  2002/09/25 16:46:58  pfpeterson
 *  Fixed a bug in getFloat(sb) and getInt(sb) where negative numbers
 *  where not considered numbers.
 *
 *  Revision 1.10  2002/08/15 18:45:09  pfpeterson
 *  Now replaces "//" in fixSeparator as well.
 *
 *  Revision 1.9  2002/08/12 20:23:59  pfpeterson
 *  Returned the fixSeparator method to its original purpose. Last
 *  version caused a bug that made runfiles not load under windows.
 *
 *  Revision 1.8  2002/08/12 18:51:47  pfpeterson
 *  fixSeparator now points at FilenameUtil.fixSeparator. Also, updated
 *  the documentation to reflect what fixSeparator actually does.
 *
 *  Revision 1.7  2002/08/06 21:22:31  pfpeterson
 *  Added methods to get fun things like floats, ints and strings
 *  out of a StringBuffer.
 *
 *  Revision 1.6  2002/04/17 21:33:10  dennis
 *  Made replace() method more robust.  Now works even if the
 *  replaced string is contained in the new string.
 *  Added method replace_token() that only makes replacements if
 *  the string is bordered by non-alphanumeric characters.
 *  Added method extract_tokens() that places separate tokens that
 *  occur in a String into an array of Strings.
 *
 */

import java.util.*;

public class Main {
    /**
     *  Produce an array of string "tokens" that are present in the specified
     *  string and separated by the specified delimiters.
     *
     *  @param   in_string   the string from which the tokens are to be extracted
     *
     *  @param   delimiters  the string containing the delimiters, 
     *                       eg: " ,;:\t\n\r\f"
     *
     *  @return  An array of strings containing the tokens 
     */
    public static String[] extract_tokens(String in_string, String delimiters) {
        StringTokenizer tokenizer = new StringTokenizer(in_string, delimiters);
        Vector tokens = new Vector();

        while (tokenizer.hasMoreTokens())
            tokens.add(tokenizer.nextToken());

        String list[] = new String[tokens.size()];
        for (int i = 0; i < list.length; i++)
            list[i] = (String) tokens.elementAt(i);

        return list;
    }
}

Related

  1. extract(final String grammarString, final char separator)
  2. extract(Properties p, String prefix)
  3. extractAggregationPrimKeysFromSql( final String sql)
  4. extractArrayData(String input, StringBuilder output)
  5. extractArrayData(String input, StringBuilder output)
  6. extractCharNgram(String content, int n)