6 Replies - 2738 Views - Last Post: 20 September 2009 - 12:47 PM Rate Topic: -----

#1 bpiche  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 18
  • Joined: 05-July 09

adding tokens to 2d array with scanner

Posted 20 September 2009 - 11:27 AM

hi all,

im trying to write a program for a natural language processing course im taking that will read in text from a file, line by line with scanner, and add each token it encounters (with the whitespace delimiter) to a two dimensional array.
ive never used 2d arrays before and im not sure if im doing this right. could you please help me to see my mistakes?

-thanks

import java.util.Scanner;
import java.io.File;
import java.util.StringTokenizer;
import java.io.FileNotFoundException;

public class CFGrammar{


private String location;
private String [][] theMatrix = new String [4][4];

	public static void main(String[] args,String loc) {

		System.out.println("Welcome to CFGrammar");
		System.out.println("Enter the location of your context free grammar: ");
		Scanner scanner = new Scanner(System.in);
		String location = scanner.next();
		CFGrammar g = new CFGrammar();
		g.CFGParser(loc);
	}

	private void CFGParser(String loc){
		loc = location;
		try{
		   File file = new File(location);
		   Scanner reader = new Scanner(file);
		   while (reader.hasNextLine()){
			   String line = reader.nextLine();
			   StringTokenizer st = new StringTokenizer(line);
			   while (st.hasMoreTokens()){
				   //add each token to the matrix
				   for(int i=0;i<theMatrix.length;i++){
					   for(int j=0;j<theMatrix[i].length;i++){
						   st.nextToken() = theMatrix[i][j];
					   }
				   }
			   }
			System.out.println(reader.nextLine());
		}
		} catch (FileNotFoundException e) {
			e.printStackTrace();
		}
	}



Is This A Good Question/Topic? 0
  • +

Replies To: adding tokens to 2d array with scanner

#2 mostyfriedman  Icon User is offline

  • The Algorithmi
  • member icon

Reputation: 727
  • View blog
  • Posts: 4,473
  • Joined: 24-October 08

Re: adding tokens to 2d array with scanner

Posted 20 September 2009 - 11:34 AM

what exactly is the problem?, any compilation errors or undesired output?

This post has been edited by mostyfriedman: 20 September 2009 - 11:35 AM

Was This Post Helpful? 0
  • +
  • -

#3 mostyfriedman  Icon User is offline

  • The Algorithmi
  • member icon

Reputation: 727
  • View blog
  • Posts: 4,473
  • Joined: 24-October 08

Re: adding tokens to 2d array with scanner

Posted 20 September 2009 - 11:46 AM

mmm well since you're writing the file into a 2d array you may wanna try out something like this
boolean stop = false;
for(int i=0;i<theMatrix.length;i++){
	if(stop)
		break;
	for(int j=0;j<theMatrix[i].length;j++){
			if(reader.hasNextLine())
					theMatrix[i][j] = reader.nextLine();
			else
			{
				stop = true;
				 break;
			}
		}
}



try this

This post has been edited by mostyfriedman: 20 September 2009 - 11:49 AM

Was This Post Helpful? 0
  • +
  • -

#4 bpiche  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 18
  • Joined: 05-July 09

Re: adding tokens to 2d array with scanner

Posted 20 September 2009 - 11:47 AM

View Postmostyfriedman, on 20 Sep, 2009 - 10:34 AM, said:

what exactly is the problem?, any compilation errors or undesired output?


the program compiles alright, at least in netbeans, but it's not doing what i want. for our purposes, just assume that you have a text file composed of several lines of words, and that youd like to read in each line with the scanner and add each word token in each line to an array called theMatrix[][]. that's what im trying to do here, but i can't understand what i'm doing wrong.
again, thank you very much for your help.
Was This Post Helpful? 0
  • +
  • -

#5 mostyfriedman  Icon User is offline

  • The Algorithmi
  • member icon

Reputation: 727
  • View blog
  • Posts: 4,473
  • Joined: 24-October 08

Re: adding tokens to 2d array with scanner

Posted 20 September 2009 - 11:50 AM

check out my last post and take out the while loops and the for loops you had..lemme know if it works
Was This Post Helpful? 0
  • +
  • -

#6 bpiche  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 18
  • Joined: 05-July 09

Re: adding tokens to 2d array with scanner

Posted 20 September 2009 - 11:50 AM

View Postmostyfriedman, on 20 Sep, 2009 - 10:46 AM, said:

mmm well since you're writing the file into a 2d array you may wanna try out something like this
boolean stop = false;
for(int i=0;i<theMatrix.length;i++){
	if(stop)
		break;
	for(int j=0;j<theMatrix[i].length;j++){
			if(reader.hasNextLine())
					theMatrix[i][j] = reader.nextLine();
			else
			{
				stop = true;
				 break;
			}
		}
}



try this


thanks a lot, i'll try that right now and let you know how it goes
Was This Post Helpful? 0
  • +
  • -

#7 bpiche  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 18
  • Joined: 05-July 09

Re: adding tokens to 2d array with scanner

Posted 20 September 2009 - 12:47 PM

also - this is a natural language processing assignment, so the file we're reading in is actually a context free grammar with chomskyan normal rules of the form X => Y Z with nonterminal and terminal indices
thus a line from the file we're reading in will look like
S !
S NP VP
NP D N
D the
N dog
N food
VP V NP
V eats

where S, NP, and VP in this case are nonterminals (because they have rewrite rules) and D, N, and V are terminals (because they dont have any rewrite rules, they just reference a word). this would print "the dog eats food!".

so, the plan is to read in this file line by line, tokenizing each unit delimited by whitespace into a two dimensional array where each line is an array object. then, we loop through each array in the 2d array. whenever we hit an element, we check to see if it occurs on the 'left hand side' (as a nonterminal). if it does, then we skip to the array where it occurs as a nonterminal, and we repeat the process. if at any point in this process we hit a terminal (i.e., if we hit an element which doesnt also occur on the left hand side), then we just print it out.

here's what ive got
import java.util.Scanner;
import java.io.File;
import java.util.StringTokenizer;
import java.io.FileNotFoundException;

public class CFGrammar{

private String location;
private String [][] theMatrix = new String [4][4];

	public static void main(String[] args,String loc) {

		System.out.println("Welcome to CFGrammar");
		System.out.println("Enter the location of your context free grammar: ");
		Scanner scanner = new Scanner(System.in);
		String location = scanner.next();
		CFGrammar g = new CFGrammar();
		g.CFGParser(loc);
	}

	private void CFGParser(String loc){
		loc = location;
		try{
		   File file = new File(location);
		   //read in the file with scanner
		   Scanner reader = new Scanner(file);
		   //scan the file line by line
		   while (reader.hasNextLine()){
			   String line = reader.nextLine();
			   //tokenize each unit in the line delimited by a (\s)
			   StringTokenizer st = new StringTokenizer(line);
			   while (st.hasMoreTokens()){
				   //add each token to the 2d array
				   boolean stop = false;
				   for(int i=0;i<theMatrix.length;i++){
					   if(stop)
						   break;
					   for(int j=0;j<theMatrix[i].length;j++){
						   if(reader.hasNextLine())
							   theMatrix[i][j] = reader.nextLine();
						   else
						   {
							   stop = true;
						   } break;
					   }
				   }
			   }
		   }
		   //recursively print the contents of the matrix to the console
		   //loop down the array
		   for(int i=0;i<theMatrix.length;i++){
			   //loop across the array[j][i]
			   for(int j=0;j<theMatrix[i].length;j++){
			   //if theMatrix[i] also occurs at [j][0] then skip to that array and loop through
				   if(theMatrix[i][j]==theMatrix[j][i]){
				   //i dont really know what to put here u_u
				   //if theMatrix[i] doesnt occur at [j][0] then print
				   } else System.out.print(i);
			   }
		   }

		} catch (FileNotFoundException e) {
			e.printStackTrace();
		}
	}


Was This Post Helpful? 0
  • +
  • -

Page 1 of 1