Java School Assignment? Project Due Tomorrow? Chat LIVE With A Programming Expert!

Programming Help

Web Development



What's Here?

  • Members: 307,022
  • Replies: 841,511
  • Topics: 140,618
  • Snippets: 4,466
  • Tutorials: 1,166
  • Total Online: 2,177
  • Members: 119
  • Guests: 2,058

Welcome to Dream.In.Code
Become a Java Expert!

Join 307,022 Java Programmers for FREE! Get instant access to thousands of Java experts, tutorials, code snippets, and more! There are 2,177 people online right now. Registration is fast and FREE... Join Now!




Java Stringtokenizer

 

Java Stringtokenizer, nextToken() returns the previous token's value
mikeash

27 Jun, 2009 - 01:11 PM
Post #1

New D.I.C Head
*

Joined: 27 Jun, 2009
Posts: 4


My Contributions
Can anyone help, please? I'm trying to extract data elements from a delimited string. The delimiter is '~' (a tilde). The problem arises when there are 2 tildes with nothing between them.

Here's a sample of my data:
100643421~GB~ACTIVE~0~1~~true~5.0~1246132975801~~N
You'll see there are two instances of blank fields: immediately before the word 'true', and before the 'N'.

Using '~' as a tokenizer, my code works OK, extracting
100643421
GB
ACTIVE
0
1
but then, on the next invocation of nextToken(), I get the previous value (i.e. '1') returned.
A call to countTokens() gives a result of 9, not the 11 I would have expected.

The relevant parts of my code are as follows:

CODE
String aStr, workStr;
StringTokenizer st1 = new StringTokenizer(workStr, "~");
long theID;
String region;
...
aStr = st1.nextToken();
theID = Long.parseLong(aStr);
aStr = st1.nextToken();
region = aStr.trim();


Any suggestions will be greatly appreciated!
User is offlineProfile CardPM
+Quote Post


nick2price

RE: Java Stringtokenizer

27 Jun, 2009 - 01:32 PM
Post #2

D.I.C Lover
*****

Joined: 23 Nov, 2007
Posts: 1,345



Thanked: 121 times
My Contributions
To make it easy on yourself, instead of trying to figure out how to handle different amount of spaces between your words, why dont you make sure that the spacing is consistant? You only passing the tokenizer a String, and not a text file, so making the String consistant beforehand is problably your easiest option. How do you create the String anyways, by user input or somthing?
User is offlineProfile CardPM
+Quote Post

mikeash

RE: Java Stringtokenizer

27 Jun, 2009 - 01:50 PM
Post #3

New D.I.C Head
*

Joined: 27 Jun, 2009
Posts: 4


My Contributions
QUOTE(nick2price @ 27 Jun, 2009 - 01:32 PM) *

To make it easy on yourself, instead of trying to figure out how to handle different amount of spaces between your words, why dont you make sure that the spacing is consistant? You only passing the tokenizer a String, and not a text file, so making the String consistant beforehand is problably your easiest option. How do you create the String anyways, by user input or somthing?


I don't create the string, I get it from a source over which I have no control.

If I put the string into a file with other lines of similar data, MS Excel would have no trouble in opening the file and creating a worksheet with 11 columns, 2 of which would be blank. I don't like paying compliments to MS, but at least they can handle delimited files!
User is offlineProfile CardPM
+Quote Post

nick2price

RE: Java Stringtokenizer

27 Jun, 2009 - 02:07 PM
Post #4

D.I.C Lover
*****

Joined: 23 Nov, 2007
Posts: 1,345



Thanked: 121 times
My Contributions
I cant remember exactly, but the second parameter of the method is a String of all deliminater types. So you could in theory do
CODE
StringTokenizer st1 = new StringTokenizer(workStr, "~  ~   ~");

See if somthing like that will work
User is offlineProfile CardPM
+Quote Post

mikeash

RE: Java Stringtokenizer

29 Jun, 2009 - 01:45 AM
Post #5

New D.I.C Head
*

Joined: 27 Jun, 2009
Posts: 4


My Contributions
QUOTE(nick2price @ 27 Jun, 2009 - 02:07 PM) *

I cant remember exactly, but the second parameter of the method is a String of all deliminater types. So you could in theory do
CODE
StringTokenizer st1 = new StringTokenizer(workStr, "~  ~   ~");

See if somthing like that will work

There are 3 constructors for StringTokenizer: the simplest assumes "white space" delimiters (space, tab, form-feed, etc), so you just specify the target string; the next requires you to specify the target string and a delimiter string; and the third takes a boolean parameter. In the third constructor, if the boolean is set to 'true', the tokens are returned with the delimiter string attached.

I believe it's possible to set a delimiter and then temporarily override it with a different delimiter, but this isn't what I want!

I can get StringTokenizer to do what I want by first using
CODE
<string>.replace("~~","~ ~")
or something similar. But why should this be necessary? Is this a design flaw in StringTokenizer?

Thanks for your help!
User is offlineProfile CardPM
+Quote Post

nick2price

RE: Java Stringtokenizer

29 Jun, 2009 - 02:49 AM
Post #6

D.I.C Lover
*****

Joined: 23 Nov, 2007
Posts: 1,345



Thanked: 121 times
My Contributions
You should be able to do it straight of with somthing similar in my last post. The second constructor is a String of all types of deliminaters. So if your input has a mix of deliminaters like a space, ! and |, you can tell your tokenizer
CODE
StringTokenizer st1 = new StringTokenizer(workStr, "  ! |");

And that should look for all three deliminaters.
User is offlineProfile CardPM
+Quote Post

cfoley

RE: Java Stringtokenizer

29 Jun, 2009 - 03:52 AM
Post #7

D.I.C Addict
Group Icon

Joined: 11 Dec, 2007
Posts: 660



Thanked: 63 times
Dream Kudos: 25
My Contributions
You don't get the error you are describing but the StringTokenizer does skip blank tokens. More precisely it treats contiguous delimiters as one. Try the String.split() method instead.

java
	public static void main(String[] args) {
		String workStr = "100643421~GB~ACTIVE~0~1~~true~5.0~1246132975801~~N";
		String[] tokens = workStr.split("~");
		for(String t : tokens) {
			System.out.println(t);
		}
	}
User is offlineProfile CardPM
+Quote Post

mikeash

RE: Java Stringtokenizer

1 Jul, 2009 - 04:44 AM
Post #8

New D.I.C Head
*

Joined: 27 Jun, 2009
Posts: 4


My Contributions
QUOTE(cfoley @ 29 Jun, 2009 - 03:52 AM) *

You don't get the error you are describing but the StringTokenizer does skip blank tokens. More precisely it treats contiguous delimiters as one. Try the String.split() method instead.

java
	public static void main(String[] args) {
		String workStr = "100643421~GB~ACTIVE~0~1~~true~5.0~1246132975801~~N";
		String[] tokens = workStr.split("~");
		for(String t : tokens) {
			System.out.println(t);
		}
	}


You're quite right - I didn't get the error I described, as I learned when I'd stopped pulling my hair out and examined the situation more calmly!

Many thanks for your solution.
User is offlineProfile CardPM
+Quote Post

cfoley

RE: Java Stringtokenizer

1 Jul, 2009 - 05:21 AM
Post #9

D.I.C Addict
Group Icon

Joined: 11 Dec, 2007
Posts: 660



Thanked: 63 times
Dream Kudos: 25
My Contributions
Glad it helps. Thanks for hitting the thanks button. smile.gif
User is offlineProfile CardPM
+Quote Post

Fast ReplyReply to this topicStart new topic

Time is now: 11/21/09 08:10AM
Invision Power Board v2.1.7 © 2009  IPS, Inc.
Licensed to: MediaGroup1

Live Java Help!

Be Social

Dream.In.Code RSS Feed Dream.In.Code LinkedIn Group Follow Us On Twitter Fan Us On Facebook

Java Tutorials

Reference Sheets

Java Snippets

DIC Chatroom

Bye Bye Ads

Monthly Drawing

Thumb Drive

Top Contributors

Top 10 Kudos This Month