Reading comma separated values to and from an array

  • (2 Pages)
  • +
  • 1
  • 2

15 Replies - 1479 Views - Last Post: 19 February 2013 - 01:22 PM Rate Topic: -----

#1 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Reading comma separated values to and from an array

Posted 11 February 2013 - 08:33 AM

Hello! I am attempting to use stdio to archive and play back comma separated values to a text file, but I have a few questions. What I'm attempting to do is save data from two different motor controllers (1500 values each ranging from -1.000 to +1.000 taken every 1/100th of a second) into a text file using CSV. Currently I'm not sure if I can either write to two lines or write to two files simultaneously (and efficiently). I'm fairly certain that both are possible and fairly straightforward, but research hasn't yielded much yet. What I would really like to learn how to do is take that data out of the text file(s) and put it into two arrays (all 1500 values in each). I've been attempting to do this using the much more familiar fstream, but as it turns out that doesn't work on imbedded systems (we're programming a National Instruments cRIO-II, which has the same capabilities for interpreting and storing code as a typical computer, except for a few library issues). Could someone explain/demonstrate how this would be done? Thanks!

Is This A Good Question/Topic? 0
  • +

Replies To: Reading comma separated values to and from an array

#2 jimblumberg  Icon User is online

  • member icon


Reputation: 3993
  • View blog
  • Posts: 12,322
  • Joined: 25-December 09

Re: Reading comma separated values to and from an array

Posted 11 February 2013 - 08:46 AM

You need to show what you have tried, then possibly we can point you in the correct direction. There are many tutorials on C file input and output if you search the internet.

Jim
Was This Post Helpful? 0
  • +
  • -

#3 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Re: Reading comma separated values to and from an array

Posted 11 February 2013 - 09:22 AM

Well, what we've tried is using fstring, but it's not compatible with our environment (Which is a known fact. Not just our codes fault). It looked sort of like this for writing:
                                        ofstream left;
					ofstream right;
					left.open ("LEFT.txt", fstream::out);
					right.open ("RIGHT.txt", fstream::out);
					while (counter <= 1500) //while (samples <= 1500)
					{
					    right << RightDrive->Get() << "," << endl; 
						left << LeftDrive->Get() << "," << endl;
						counter++;
						Wait(0.01);
					}
					left.close();
					right.close();

I'm petty sure we found some faults in this, but I don't have access to the revised code at the moment. One of the reasons I didn't put the whole code is because it's based around the WPI library, which makes the code look pretty foreign because so much of the actual code is hidden. The logic for reading is something like this (we didn't actually get around to using the text files, just some short arrays...)
float leftside [] = {1,0.9,0.7,0.9,1,0.3,0.9}; //etc...etc..
float rightside [] = {1,1,0.8,1,1,0.4,1};      //etc...etc...
for (int i = arrayLength-1; i >= 0; i--)
{
	Leftmotor->Set(-leftside[i])
	Rightmotor->Set(-rightside[i])
	Wait(0.1); 
	/*Wait the same interval of time as the data collected.  
	0.1 for tenths, 0.01 for hundredths, etc...*/
}

Was This Post Helpful? 0
  • +
  • -

#4 Adak  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 331
  • View blog
  • Posts: 1,168
  • Joined: 01-April 11

Re: Reading comma separated values to and from an array

Posted 11 February 2013 - 09:28 AM

No, you can't write out two files of data, at the same time. The HD buffer will allow you to SIMULATE IT, however.

On PC's, you don't write bytes to the hard drive, byte by byte. You write it to the HD buffer, and it will quickly be stored there, until the buffer gets the command from the HD, to write the data NOW. THEN it will be written to the disk.

1/100th of a second recording is no problem for the HD buffer, but if it keeps up that pace, the HD head itself, might be unable to keep up. Data would then begin to back up in the HD buffer, and eventually (which depends on the specifics of your system), data could be lost.

Fixes could include:
1) write the data to memory if the system can handle it.

2) write the data to a RAM disk file in memory, if the system can handle it.

3) had the data sent to an SSD drive.

4) get a HD with a large buffer to prevent overflow. Using a non-system disk is also helpful since the other system services will not be using it. Be sure any "indexing" is turned off on this disk, during this time.

5) combine #4 with a faster HD - measuring throughput to it, and writing speed.

Post up your attempt, and a sample of your data (just small is fine), and we can advise further.
Was This Post Helpful? 1
  • +
  • -

#5 Skydiver  Icon User is online

  • Code herder
  • member icon

Reputation: 3479
  • View blog
  • Posts: 10,724
  • Joined: 05-May 12

Re: Reading comma separated values to and from an array

Posted 11 February 2013 - 11:02 AM

If you have 1500 floating point values, you only need 6K bytes, 12K bytes if you use doubles. Most modern systems will give you at least 32K or 64K of memory so you can afford to record your values into memory at realtime, and then later at your leisure write into a CSV file, or write the entire buffer into a binary file, and decode on the other machine later.
Was This Post Helpful? 0
  • +
  • -

#6 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Re: Reading comma separated values to and from an array

Posted 12 February 2013 - 09:11 AM

Okay so I can now store data to two different text files. It's not exactly what I wanted to do (store data in one file on two lines), but it works.
	void OperatorControl(void)
	{
		DriverStationLCD *driverStationLCD = DriverStationLCD::GetInstance();
		FILE *leftoutput = fopen("/left.txt", "w");  //The file written can be viewed with a web browser by visiting "ftp://10.te.am.2/text.txt"
		FILE *rightoutput = fopen("/right.txt", "w");
		//int counter = 0;
		while (IsOperatorControl())
		{	
			float driverLeftY = driverXbox->GetRawAxis(2);
			float driverRightY = driverXbox->GetRawAxis(5);
			float left = LeftDrive->Get();
			float right = RightDrive->Get();
			fprintf(leftoutput, "%f,", left);
			fprintf(rightoutput, "f,", right);
			myRobot->TankDrive(-driverLeftY, driverRightY);
			driverStationLCD->PrintfLine((DriverStationLCD::Line) 0, "Left: %f", left);
			driverStationLCD->PrintfLine((DriverStationLCD::Line) 1, "Right: %f", right);
			driverStationLCD->UpdateLCD();
			Wait(0.01);
		}
		fclose(leftoutput);
		fclose(rightoutput);
	}

The output of each file looks like this
0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.008197,0.016393,0.016393,0.024590,0.024590,0.024590,0.024590,0.024590,0.024590,0.081967,0.081967,0.122951,0.122951,0.122951,0.122951,0.122951,0.122951,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.319672,0.278689,0.278689,0.196721,0.196721,0.196721,0.196721,0.196721,0.196721,0.065574,0.065574,0.065574,0.065574

All I have left to do is figure out how to put the entire length of data into an array (actually two arrays, at the same time, which I have absolutely no idea how to do). I haven't been programming for long, and prior to this I have never needed to use text files for I/O. I'll post it here if I ever figure it out. Thanks!
Was This Post Helpful? 0
  • +
  • -

#7 jimblumberg  Icon User is online

  • member icon


Reputation: 3993
  • View blog
  • Posts: 12,322
  • Joined: 25-December 09

Re: Reading comma separated values to and from an array

Posted 12 February 2013 - 09:29 AM

Other than the missing '%' in your rightoutput printf() what seems to be the issue? Is there a reason you are trying to create a CSV file instead of just putting each number on it's own line?

Quote

0.008197
0.008197
0.008197
0.008197
0.008197
0.008197
0.008197
0.008197
0.008197
...
0.065574


Quote

All I have left to do is figure out how to put the entire length of data into an array (actually two arrays, at the same time, which I have absolutely no idea how to do).


Show the code where you are trying to read the data from your files, then maybe we can point you in the correct direction.


Jim
Was This Post Helpful? 0
  • +
  • -

#8 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Re: Reading comma separated values to and from an array

Posted 18 February 2013 - 09:21 AM

Hello! It's been a while since I have been here. I haven't made many changes (this is basically just a fun programming project; lower priority). THe only real change I have made Is have separating the values by line (thanks jimblumberg) to simplify the reading process when I eventually figure that out (no searching for commas or guessing the number of characters to take). My original plan was to put the entire file of data into one array and use a for{} loop to cycle through the data, but instead I'm just attempting to read line by line in real time (is this a good idea?). In a recent update, compatibility for fstream has been added to the system, so we're attempting to switch back to that. This is our writing code (which works flawlessly):
if (counter <= cycles) //cycles = 1500
				{
					if (Timer::GetPPCTimestamp() >= waitTimeout) //state machine
					{
						frontright << FrontRightDrive->Get() << "\n"; 
						frontleft << FrontLeftDrive->Get() << "\n";
						backright << BackRightDrive->Get() << "\n"; 
						backleft << BackLeftDrive->Get() << "\n";
						counter++;	
						dsLCD->PrintfLine((DriverStationLCD::Line) 0, "Recording...");						
						dsLCD->PrintfLine((DriverStationLCD::Line) 1, "Loops: %d/%d", counter, cycles);
						dsLCD->UpdateLCD();
						waitTimeout += 0.01;
					}

Our reading code should look something this:
				if (toggle->Get() == 0)
				{
					ifstream frontleft;
					ifstream frontright;
					ifstream backleft;
					ifstream backright;
					frontleft.open ("FLEFT.txt");
					frontright.open ("FRIGHT.txt");
					backleft.open ("BLEFT.txt");
					backright.open ("BRIGHT.txt");
					for (int a = 0; a < cycles; a++)
					{
						if (Timer::GetPPCTimestamp() >= waitTimeout) 
						{
							FrontLeftDrive->Set(std::getline(frontleft, a));
							FrontRightDrive->Set(std::getline(frontright, a));
							BackLeftDrive->Set(std::getline(backleft, a));
							BackRightDrive->Set(std::getline(backright, a));
							waitTimeout += 0.01;
						}
					}

but we get the error

Quote

error: no matching function for call to `getline(std::ifstream&, int&)'
I'm not sure how else to get the line from the file (and I'm not sure what exactly the line is considered... a string, etc). Any help would be appreciated! Thanks!
Was This Post Helpful? 0
  • +
  • -

#9 jimblumberg  Icon User is online

  • member icon


Reputation: 3993
  • View blog
  • Posts: 12,322
  • Joined: 25-December 09

Re: Reading comma separated values to and from an array

Posted 18 February 2013 - 09:43 AM

If you separate your numbers so they are each on a line of their own you don't need to use getline(), just use the extraction operator>> to extract the number.

fileStream >> doubleVariable;


You may want to study this link: Simple C++ file I/O


Jim

This post has been edited by jimblumberg: 18 February 2013 - 09:44 AM
Reason for edit:: Added link.

Was This Post Helpful? 0
  • +
  • -

#10 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Re: Reading comma separated values to and from an array

Posted 18 February 2013 - 11:54 AM

I still don't see how I would implement that to get to the next line. Thank you very much, though! That was the first actual somewhat successful test. I've been trying for a while...
Was This Post Helpful? 0
  • +
  • -

#11 jimblumberg  Icon User is online

  • member icon


Reputation: 3993
  • View blog
  • Posts: 12,322
  • Joined: 25-December 09

Re: Reading comma separated values to and from an array

Posted 18 February 2013 - 12:33 PM

Quote

I still don't see how I would implement that to get to the next line.

Did you try it? If so where is the code that doesn't work? I suggest posting the smallest complete program that illustrates your problem.

Start simple try to read and print the contents of only one of your file. Remove everything that is not concerned with the reading or writing of that one file. Once you can properly read a single file reading the other files should be simple. Also don't forget about using functions, a function here or there with the correct parameters can make this much simpler.

Jim
Was This Post Helpful? 0
  • +
  • -

#12 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Re: Reading comma separated values to and from an array

Posted 19 February 2013 - 08:34 AM

Based on what I saw in the example, I would get to the next line by using file<<variable<<endl which would store the line in the variable and then go to the next line. This is what I just tried, and there is about 900 lines of errors (WindRiver :P ).
while (counter < cycles) //cycles == 120
					{
						if (Timer::GetPPCTimestamp() >= waitTimeout) 
						{
							frontleft << w << endl;
							dsLCD->PrintfLine((DriverStationLCD::Line) 0, "FLEFT: %f", w); //text file for front left front motor controller
							dsLCD->UpdateLCD();
							counter++;
							waitTimeout += 0.1;
}}
The most prevalent being

Quote

C:/WindRiver/workspace/BuiltinDefaultCode/BuiltinDefaultCode.cpp:59: error: no match for 'operator>>' in '((std::basic_istream<char, std::char_traits<char> >*)(&frontleft))->std::basic_istream<_Elem, _Traits>::operator>> [with _Elem = char, _Traits = std::char_traits<char>](((float&)(&w))) >> std::endl'
c:/windriver/gnu/3.4.4-vxworks-6.3/x86-win32/lib/gcc/../../../x86-win32/include/c++/3.4.4/istream:133: note: candidates are: std::basic_istream<_Elem, _Traits>& std::basic_istream<_Elem, _Traits>::operator>>(std::basic_istream<_Elem, _Traits>&(*)(std::basic_istream<_Elem, _Traits>&)) [with _Elem = char, _Traits = std::char_traits<char>]
c:/windriver/gnu/3.4.4-vxworks-6.3/x86-win32/lib/gcc/../../../x86-win32/include/c++/3.4.4/istream:139: note: std::basic_istream<_Elem, _Traits>& std::basic_istream<_Elem, _Traits>::operator>>(std::basic_ios<_Elem, _Traits>&(*)(std::basic_ios<_Elem, _Traits>&)) [with _Elem = char, _Traits = std::char_traits<char>]
c:/windriver/gnu/3.4.4-vxworks-6.3/x86-win32/lib/gcc/../../../x86-win32/include/c++/3.4.4/istream:146: note: std::basic_istream<_Elem, _Traits>& std::basic_istream<_Elem, _Traits>::operator>>(std::ios_base&(*)(std::ios_base&)) [with _Elem = char, _Traits = std::char_traits<char>]
c:/windriver/gnu/3.4.4-vxworks-6.3/x86-win32/lib/gcc/../../../x86-win32/include/c++/3.4.4/istream:153: note: std::basic_istream<_Elem, _Traits>& std::basic_istream<_Elem, _Traits>::operator>>(std::_Bool&) [with _Elem = char, _Traits = std::char_traits<char>]

Was This Post Helpful? 0
  • +
  • -

#13 Skydiver  Icon User is online

  • Code herder
  • member icon

Reputation: 3479
  • View blog
  • Posts: 10,724
  • Joined: 05-May 12

Re: Reading comma separated values to and from an array

Posted 19 February 2013 - 08:55 AM

The code you posted above uses operator <<, but the errors you are showing us is with regards to operator >>. Are you sure you are posting the lines around line 59 from your code?

This post has been edited by Skydiver: 19 February 2013 - 08:55 AM

Was This Post Helpful? 0
  • +
  • -

#14 jimblumberg  Icon User is online

  • member icon


Reputation: 3993
  • View blog
  • Posts: 12,322
  • Joined: 25-December 09

Re: Reading comma separated values to and from an array

Posted 19 February 2013 - 08:58 AM

Quote

Based on what I saw in the example, I would get to the next line by using file<<variable<<endl

No endl is only defined for output you never use this on an input file. You don't need to manually read the end of line character when reading numbers from a file since the extraction operator automatically skips white space characters when inputting numbers. So get rid of the "<< endl".

Jim
Was This Post Helpful? 0
  • +
  • -

#15 freestyler  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 28
  • Joined: 31-March 12

Re: Reading comma separated values to and from an array

Posted 19 February 2013 - 11:40 AM

View PostSkydiver, on 19 February 2013 - 03:55 PM, said:

The code you posted above uses operator <<, but the errors you are showing us is with regards to operator >>. Are you sure you are posting the lines around line 59 from your code?

Thanks for catching that. I tried about twenty different changes while I was in the process of writing this, hoping I would get lucky and not have to send it. This is the actual error I was given:

Quote

C:/WindRiver/workspace/BuiltinDefaultCode/BuiltinDefaultCode.cpp: In member function `virtual void DefaultRobot::Autonomous()':
C:/WindRiver/workspace/BuiltinDefaultCode/BuiltinDefaultCode.cpp:56: error: no match for 'operator<<' in 'frontleft << w'
C:/WindRiver/workspace/BuiltinDefaultCode/BuiltinDefaultCode.cpp:57: error: no match for 'operator<<' in 'frontright << x'
C:/WindRiver/workspace/BuiltinDefaultCode/BuiltinDefaultCode.cpp:58: error: no match for 'operator<<' in 'backleft << y'
C:/WindRiver/workspace/BuiltinDefaultCode/BuiltinDefaultCode.cpp:59: error: no match for 'operator<<' in 'backright << z'

It's the same error I get when I get rid of "endl". It compiles when I switch "<<" to ">>", but I won't be able to test for a few hours. Even in the event that it does work, I still don't understand how it knows to stop at the end of one line during one loop cycle and then pick up from there and get the next line during the next cycle. I never realized how valuable file I/O could be until now. Thank you all for the help I've received!
Was This Post Helpful? 0
  • +
  • -

  • (2 Pages)
  • +
  • 1
  • 2