file read format in C++

  • (2 Pages)
  • +
  • 1
  • 2

15 Replies - 5953 Views - Last Post: 24 November 2007 - 12:37 AM Rate Topic: -----

#1 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

file read format in C++

Post icon  Posted 19 November 2007 - 08:33 PM

Hello,
How do i write the following matlab function is C++? The problem is with the last argument parameter "int8", This kind of fread format can be specified in matlab but what about C++?
[rawsignal,samplesread]=fread(fid,blksize, 'int8');
Here it reads blksize amt of data, integer 8 bits at a time from the fid.
I have written it as the following in C++: assuming that i have opened a file...using data.open("x.bin",ios::binary);
data.read(rawsignal,blksize);
samplesread=gcount();
So my concern is abt how i shud account for that int8 in C++??
Thanks,
Prads

Is This A Good Question/Topic? 0
  • +

Replies To: file read format in C++

#2 jjhaag  Icon User is offline

  • me editor am smartastic
  • member icon

Reputation: 44
  • View blog
  • Posts: 1,789
  • Joined: 18-September 07

Re: file read format in C++

Posted 19 November 2007 - 08:45 PM

int8 is essentially a single-byte integer datatype in MATLAB, with possible values ranging from -128 to 127. The format specifier in fread in MATLAB tells the program how to interpret the values that are being read into your array.

For C++, the single-byte integer datatype is the char. To get the same range of possible values, you'd be using the unsigned version.

The real reason for the format specifier in the MATLAB version is that the program has to interpret the values in the file based on the format specifier, and store them into a double array. In C/C++, you're basically performing a copy of the raw data, bit by bit, into your array, rather than an interpretation. So the important part is in having the array you're storing into be the proper data type in the first place, which in this case is the unsigned char that I mentioned above. If you're looking to actually interpret values from the file and store them into an array, you'll need a different approach. But if the bitwise copy of the data in the file is what you're looking for, then just using read to get the data into an unsigned char[] should work fine.

Hope that helps,

-jjh

*edit p.s. shud isn't a word

This post has been edited by jjhaag: 19 November 2007 - 08:46 PM

Was This Post Helpful? 0
  • +
  • -

#3 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 20 November 2007 - 01:37 AM

Sorry for using short forms......now as you said that int8 in matlab is used to interpret the values, for which in C++ has a different approach than using unsigned char, what should be done if that's the case. I am not sure on what the matlab code is intending to do other than just trying to read the data to a buffer rawsignal. Here's what he does:
% Read in the appropriate number of samples to process this interation
[rawSignal, samplesRead] = fread(fid,blksize, settings.dataType);
where settings,datatype is int8.
So can u please tell me what should be done to account for interpreting the values in C++.
Thanks,
Prads
Was This Post Helpful? 0
  • +
  • -

#4 jjhaag  Icon User is offline

  • me editor am smartastic
  • member icon

Reputation: 44
  • View blog
  • Posts: 1,789
  • Joined: 18-September 07

Re: file read format in C++

Posted 20 November 2007 - 02:00 AM

Well, interpreting isn't done with the read() method - read() just pulls the specified number of bytes from the file and sticks them into your array. The array will be a bitwise copy of the data contained in the file, if all goes well with the input step. However, if the data in the file is just numerical, you'll wind up getting each digit entered into it's own element in the array, which is probably not what you're looking for.

Without knowing anything about the file, it's tough to say what the best approach would be. If the values are separated by whitespace (spaces, tabs, newlines, etc.), you could use formatted stream extraction to pull each number out of the file:
#include <iostream>
#include <cstdlib>
#include <fstream>

using namespace std;

int main() {
	
	fstream infile("infile.txt", ios::in|ios::binary);
	
	const int n=100;
	
	int buffer[n];
	
	int i=0;
	while (!infile.eof() && i<100) {
		infile >> buffer[i];
		++i;
	}
	
	for (int j=0; j<i; ++j) {
		cout << buffer[j] << endl;
	}
	
	return 0;
}

This program reads in integer values from the file until either a) the end of the file is reached, or B) 100 elements have been read and written to the array (i.e. the array is full). However, if the values are separated by commas or other non-whitespace, then this version won't work.

But like I said, without knowing more about the file (e.g. without seeing an example line from the file), it's hard to make a very robust recommendation.

Hope that helps,

-jjh
Was This Post Helpful? 0
  • +
  • -

#5 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 20 November 2007 - 03:13 AM

hmm....i myself dont know what data it contains exactly but its a .bin file and should have alphanumeric data i suppose...so what can it be for a .bin file?
Thanks,
Prads
Was This Post Helpful? 0
  • +
  • -

#6 jjhaag  Icon User is offline

  • me editor am smartastic
  • member icon

Reputation: 44
  • View blog
  • Posts: 1,789
  • Joined: 18-September 07

Re: file read format in C++

Posted 20 November 2007 - 03:32 AM

Anything. The .bin extension is frequently used for storing disc images, but that doesn't really tell you much. It's also used as a kind of catch-all for data storage, similar to .dat. What is this file from?

First thing I'd do is change the extension to .txt and open it up in notepad or some other text editor. That way you can actually figure out what's in the file.

What is the file supposed to contain, and what are you trying to do with it? If you have no idea what's in it or how it's organized internally, there's probably not much point in reading it into a program - what were you planning on doing once you got it in there?

-jjh
Was This Post Helpful? 0
  • +
  • -

#7 jjhaag  Icon User is offline

  • me editor am smartastic
  • member icon

Reputation: 44
  • View blog
  • Posts: 1,789
  • Joined: 18-September 07

Re: file read format in C++

Posted 20 November 2007 - 04:36 AM

Just a little follow-up question here...

You've obviously got the MATLAB code for whatever it is that you're trying to do. Why exactly are you trying to do it in C++ instead? Is there a serious performance bottleneck in the MATLAB code that you're trying to get around?

Not that I'm trying to discourage you from figuring out C++...it's just that for a lot of applications, programming is supposed to make life easier, and porting code from one language to another without a good reason is definitely NOT making life easier.

-jjh
Was This Post Helpful? 0
  • +
  • -

#8 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 20 November 2007 - 08:54 PM

View Postjjhaag, on 20 Nov, 2007 - 04:36 AM, said:

Just a little follow-up question here...

You've obviously got the MATLAB code for whatever it is that you're trying to do. Why exactly are you trying to do it in C++ instead? Is there a serious performance bottleneck in the MATLAB code that you're trying to get around?

Not that I'm trying to discourage you from figuring out C++...it's just that for a lot of applications, programming is supposed to make life easier, and porting code from one language to another without a good reason is definitely NOT making life easier.

-jjh


Well yes your right.......I have a Matlab code which does the tracking of GPS satellite signals. I have been asked to write the same in C++ for speed issues. As you know matlab is slower than C++. I have done the same but unfortunately I am not getting the same output :(
I am really tensed and dont know what to do. Can you help me out with this. I am asking you because I need someone who knows both matlab and C++, and as far as I believe you seem to have mastered both (and perhaps much more). I would be extremely greatful to you. Eagerly waiting for a reply!
Thanks,
Prads

This post has been edited by prads: 20 November 2007 - 08:58 PM

Was This Post Helpful? 0
  • +
  • -

#9 jjhaag  Icon User is offline

  • me editor am smartastic
  • member icon

Reputation: 44
  • View blog
  • Posts: 1,789
  • Joined: 18-September 07

Re: file read format in C++

Posted 20 November 2007 - 09:36 PM

lol. Far from mastery on both fronts. However, I like to think that I've got a pretty good grasp of the basics of both of them, and for the moment at least, you're dealing with fairly simple I/O stuff.

But yeah, MATLAB can be pretty slow sometime, and if you're looking for speed, C or C++ is probably the way to do it.

Okay, back to the problem at hand. You're trying to read in values from a file into a MATLAB int8 array, so it's very likely that you're just dealing with numeric data with whitespace as delimiters. You should definitely check this out, though; either change the extension of the datafile to .txt and open it up in a text editor, or (this is windows only), right click the file, select Open With..., and choose notepad. That way you can actually see what you're trying to get out of the file.

To read in formatted data from a file, one generally uses the stream extraction operator >> on the file stream, repeatedly reading in single values until the end of the file. You can also read in an entire line all at once and use string streams to extract the individual numbers, but let's keep it simple for the moment.

If you know exactly how many numbers you're going to be extracting beforehand (i.e. at compile time), then the easiest way is to declare a static array and then store the numbers into that:
#include <iostream>
#include <fstream>

using namespace std;

int main() {
	const int maxElements=20;
	int rawsignal[maxElements];
	ifstream infile("infile.txt");
	
	int index=0;
	int temp;
	while (infile >> temp && index<maxElements) {
		rawsignal[index]=temp;
		++index;
	}
	
	for (int i=0; i<index; ++i) {
		cout << rawsignal[i] << " ";
	}
	
	return 0;
	
}


This program will open up a file named "infile.txt" containing whitespace-delimited values, and read them into an array of integers until the end of the file is reached. The critical components to recognize are that 1) there can be a maximum of 20 elements read in and stored, and 2) the input and interpretation of the values in the file is done using the infile >> temp statement in the continuation condition of the while loop.

If you don't know how many elements you're going to need to read in advance, you could either use a dynamically-allocated array, or an STL container like a vector or a deque. I'll hold off on that part until you've figured out a little more about the file that you're trying to read.

As for the datatype that you're reading into (an int array in this case), the biggest concern is that the datatype is capable of storing the largest and smallest elements that you're going to be reading in. In this case, since the int8 format can store -128 to +127, the int has more than enough storage space (generally, though not always, an int is 4 bytes on a 32-bit machine). If memory is going to be an issue, you could use a smaller datatype, like a short int or a signed char - but if you use the char version, you'll need to make some modifications to the above code (the above program, if the array type was switched to a char[], would try to read in character by character).

That's all for now. Without some more information about the file you're reading, we're just getting into a guessing game that could wind up wasting both of our time.

Hope it helps. Once you've figured out the information stored in the file, post again and we can move on from there.

-jjh
Was This Post Helpful? 0
  • +
  • -

#10 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 20 November 2007 - 10:49 PM

The .bin file is abt 2GB and doesnot open on a text or other editor. It says file too big to open on this editor. Even in matlab when i tried to open as text it says "Not enough memory to read file contents:Java heap space".
Actually as i told u, i have already written a C++ code for that matlab program and want someone to verify if the two match each other. Can i send u both the versions so that u can verify it for me.?
Thanks,
Prads
Was This Post Helpful? 0
  • +
  • -

#11 jjhaag  Icon User is offline

  • me editor am smartastic
  • member icon

Reputation: 44
  • View blog
  • Posts: 1,789
  • Joined: 18-September 07

Re: file read format in C++

Posted 20 November 2007 - 11:36 PM

Well, that's a pretty big file, for sure. I can see why a little speed boost might be in order. And also why notepad probably wouldn't work to open that :)

In the interest of transparency and making this a learning experience for everyone, just post both versions of the code here, in separate code blocks. Unless the code is really big for the forum (maybe >100 lines apiece), in which case post them as attachments.

And, just so we can see what we're dealing with, post a subset of the data file. Either write your own program to extract a portion, or use the following one, substituting the name of your file for "infile.txt" in the file opening method:
#include <iostream>
#include <fstream>

using namespace std;

int main() {
	
	fstream infile, outfile;
	infile.open("infile.txt", ios::in | ios::binary);
	outfile.open("sampleOutput.txt",ios::out | ios::binary);
	
	char temp;
	
	for (long i=0; i<10240; ++i) {
		infile.get(temp);
		outfile.put(temp);
	}
	
	infile.close();
	outfile.close();
	
	return 0;
}


This'll pull out a 10kB chunk of the file and save it to sampleOutput.txt for you to post.

Take care of that, get the source code and data file posted, and we can actually see what we're dealing with here :)

-jjh
Was This Post Helpful? 0
  • +
  • -

#12 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 21 November 2007 - 02:31 AM

Now thats the whole problem.......since my code is related to my ongoing research work, I have to keep it a little confidential and cannot post it in public. Thats the reason I asked you your email id so that I can send it you privately. Pls consider my request. You can send me your id to :
blahblahblahgarbageaddress@gmail.com
Thanks,
Prads

[edit]watch out posting those email addresses on a public forum - removed to prevent spam (and possibly worse)[/edit]

This post has been edited by jjhaag: 21 November 2007 - 03:15 AM

Was This Post Helpful? 0
  • +
  • -

#13 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 21 November 2007 - 03:04 PM

Here's the matlab and the C++ codes attached.....In my C++ code I have omitted certain GUI specific functions of matlab such as waitbar() etc because I mainly want my code to start working correctly. Pls check if the logic in the C++ code is same as that of the matlab one. Also initially in my C++ code, I have written a couple of structures(struct set; struct trac; struct chn;) in which I have initialized a few values. I have also written a couple of functions(calcloopcoef(); and generatecacode();) that are used in my program.
Eagerly waiting for a reply :)
Thanks, :^:
Prads
Was This Post Helpful? 0
  • +
  • -

#14 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 22 November 2007 - 07:19 AM

Hello,
I have attached both the matlab(the reference code) and my c++ codes in this post. Please look into it and let me know whats wrong in my C++ code. The problem is that i mainly need the value i_p which has a combination of +ve and -ve values. I am not getting the exact values as that obtained in MATLAB. Also after some iterations, the values remain constant as +ves or -ves.!! Please help me for which i would be really grateful.
Thanks and Regards,
Prads
Was This Post Helpful? 0
  • +
  • -

#15 prads  Icon User is offline

  • D.I.C Head

Reputation: 1
  • View blog
  • Posts: 113
  • Joined: 22-October 07

Re: file read format in C++

Posted 23 November 2007 - 08:35 PM

somebody pls reply.........hey jjhaag where are you.........please help me..!! :) :rolleyes:
Was This Post Helpful? 0
  • +
  • -

  • (2 Pages)
  • +
  • 1
  • 2