Learn StringTokenizer In Java With its Constructors & Methods

After the Java StringBuffer, you must be curious about StringTokenizer. According to the Oracle, StringTokenizer in Java is a class that allows an application to break a string into tokens.

With the help of different StringTokenizer constructors and methods, we can break a string into tokens.

So, what are you waiting for? Let’s start with the introduction.

What is StringTokenizer in Java?

StringTokenizer class is used for creating tokens in Java. It allows an application to break or split into small parts. Each split string part is called Token.

A StringTokennizer in Java, object keeps the string in the present position as it is to be tokenized. By taking a substring of the string a token can return that utilize to make the StringTokenizer protest.

Wait, do you know String vs StringBuffer vs StringBuilder in Java?

StringTokenizer Constructors

There are 3 types of Constructors available in Java StringTokenizer, let’s discuss them with their examples-

• (String str)

str is a string to be tokenized and it considers default delimiters like newline, space, tab, carriage return and form feed which can be further tokenized.

• StringTokenizer(String str, String delim)

delim is set of delimiters that are used to tokenize the given string.

• StringTokenizer(String str, String delim, boolean flag)

Since the first two parameters have the same meaning. The flag serves the following purpose. If the flag is false, delimiter characters serve to separate tokens and if the flag is true, delimiter characters are considered to be tokens.

Example 

package com.DataFlair.StringTokenizer;
import java.util.*;
public class StringTokenizerDemo {
  public static void main(String args[])
  {
    System.out.println("StringTokenizer Constructor 1 - ");
    StringTokenizer st1 =
        new StringTokenizer("Hello Readers, Welcome to DataFlair", " ");
    while (st1.hasMoreTokens())
      System.out.println(st1.nextToken());
    System.out.println("StringTokenizer Constructor 2 - ");
    StringTokenizer st2 =
        new StringTokenizer("JAVA : Code : String", " :");
    while (st2.hasMoreTokens())
      System.out.println(st2.nextToken());
    System.out.println("StringTokenizer Constructor 3 - ");
    StringTokenizer st3 =
        new StringTokenizer("JAVA Code String", " : ",  true);
    while (st3.hasMoreTokens())
      System.out.println(st3.nextToken());
  }
}

Output 

StringTokenizer Methods

Following are 5 types of Methods available in Java StringTokenizer:

1. hasMoreTokens()

The method java.util.StringTokenizer.hasmoreTokens() plays a role in testing, if tokens are present for the StringTokenizer’s string.

Basically, those characters that are considered to be delimiters by the StringTokenizer object are changed to characters in the string delimiter. Then the next token to the current position in the string is returned.

Syntax

public boolean hasMoreTokens()

Returns: True if and only if next token to the current position in the string exists, else false.

2. nextToken()

The method java.util.StringTokenizer.nextToken() returns next token from the given StringTokenizer.

Syntax

public String nextToken()

• Return: The next token from the given StringTokenizer if present.
• Throws: NoSuchElementException – if no more token are left.

3. countTokens()

This method java.util.StringTokenizer.countTokens() returns  total number of tokens which are present.

Hence, the number further use nextToken() method before it gives an exception and use it.

Syntax

public int countTokens()

Return: The number of tokens remaining in the string using the current delimiter set.
Example 

package com.DataFlair.StringTokenizer;
import java.util.*;
public class StringTokenizerMethods {
   public static void main(String args[])
     {
         String mydelim = " : ";
         String mystr = "JAVA : Code : String : Tokenizer : Dataflair";
         StringTokenizer flair3 =
                           new StringTokenizer(mystr, mydelim);
         int count = flair3.countTokens();
         for (int i = 0; i <count; i++)
             System.out.println("token at [" + i + "] : "
                                + flair3.nextToken());
         StringTokenizer string1 = null;
    while (flair3.hasMoreTokens())
             System.out.println(string1.nextToken());
     }
  }

Output

4. nextElement()

This method returns Object rather than String and java.util.StringTokenizer.nextElements() works similar to nextToken exists so that this class can implement the Enumeration interface.

Let’s talk about Interface in Java

Syntax 

public Object nextElement()

Return: The next token from the given StringTokenizer.
Throws: NoSuchElementException – if there are no more tokens left.

5. hasMoreElements()

Next, in this method java.util.StringTokenizer.hasMoreElements() returns same value as hasMoreToken.

Syntax 

public boolean hasMoreElements()

Return: True if tokens are present in the string, else false.

Example 

package com.DataFlair.StringTokenizer;
import java.util.*;
public class StringTokenizerMethod
{
   public static void main(String args[])
   {
       String mydelim = " : ";
       String mystr = "JAVA : Code : String : Tokenizer : Flair";
       StringTokenizer stringName =
                     new StringTokenizer(mystr, mydelim);
       int count = stringName.countTokens();
       System.out.println("Number of tokens : " + count);
       while (stringName.hasMoreElements())
           System.out.println(stringName.nextElement());
   }
}

Output-

StringTokenizer vs. String.split()

While StringTokenizer is a legacy class and its use is discouraged in new code, it’s still helpful to understand its functionality. However, for most modern Java programming, the String.split() method is the preferred approach for splitting strings into tokens. It offers several advantages over StringTokenizer:

1. More flexibility: String.split() allows you to specify a regular expression as the delimiter, providing more control over how the string is divided. This can be useful for handling complex delimiters or patterns within the string.

2. Immutable behavior: String.split() creates a new array of tokens from the original string, leaving the original string unmodified. This immutability can improve code readability and avoid unintended side effects.

3. Conciseness: String.split() is generally more concise and easier to use compared to StringTokenizer’s constructors and methods.

Summary

So, now you know why we use StringTokenizer. Ther are various constructors and methods for breaking the string into tokens. Hope, you understood everything with examples.

If you face any query or doubt. Feel free to ask in the comment section.

Do you know about Java Comparator Interface?