Search for a blank line in binary mode data

  • (2 Pages)
  • +
  • 1
  • 2

15 Replies - 2515 Views - Last Post: 14 January 2010 - 07:06 AM Rate Topic: -----

#1 fir3fly   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 34
  • Joined: 12-January 10

Search for a blank line in binary mode data

Posted 12 January 2010 - 10:16 PM

Hey Guys, I need some help.

I have an input file which is a RAW data file. I am supposed to store the data in a buffer then search for a blank line in the buffer and set the pointer to the blank line so that it can read the next lines. I have managed to store the data in the buffer but i have trouble with the searching for a blank line. Below is my code. I have attached my input file for you guys.
#include <fstream>
#include <iostream>
using namespace std;

int main () {

  char *buffer;
  long size;

  ifstream inFile;
  ofstream outFile;
  inFile.open("(some Input File)",ios::binary);
  outFile.open("test.txt",ios::binary);


  // get size of file
  inFile.seekg(0,ios::end);
  size = inFile.tellg();
  inFile.seekg(0,ios::beg);	

  // allocate memory for file content
  buffer = new char [size];

  // read content of inFile
  inFile.read (buffer,size);

  // write to outFile
  outFile.write (buffer,size);
  
  // release dynamically-allocated memory
  delete[] buffer;

  inFile.close();
  outFile.close();
  
  system ("pause");
  return 0;
}



My Question is : Should i manipulate the seekg or the buffer side to do this?

Is This A Good Question/Topic? 0
  • +

Replies To: Search for a blank line in binary mode data

#2 r.stiltskin   User is offline

  • D.I.C Lover
  • member icon

Reputation: 2032
  • View blog
  • Posts: 5,435
  • Joined: 27-December 05

Re: Search for a blank line in binary mode data

Posted 12 January 2010 - 10:34 PM

Do you have a specification for the format of the input file contents? What does a "blank line" contain?
Was This Post Helpful? 0
  • +
  • -

#3 fir3fly   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 34
  • Joined: 12-January 10

Re: Search for a blank line in binary mode data

Posted 12 January 2010 - 11:48 PM

View Postr.stiltskin, on 12 Jan, 2010 - 09:34 PM, said:

Do you have a specification for the format of the input file contents? What does a "blank line" contain?


The input file is a RAW format of a HTTP packet. The blank line is a CRLF (Carriage Return Line Feed) which is "\n".
The data portion is RAW binary data. So I have to set the input and output streams in binary mode.
E.g.

header
header
header
<Blank Line> --->
data...Blah Blah Blah
data...Blah Blah Blah
Was This Post Helpful? 0
  • +
  • -

#4 Guest_c.user*


Reputation:

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 03:04 AM

you can use stringstream it can be attached to the buffer
and be used as a stream
then getline it with a '\n' delimeter

add
by default the '\n' character is a delimeter
 
#include <iostream>
#include <string>
#include <sstream>

using namespace std;

// makes an istringstream
int main(void)
{
	istringstream is;
	string b = "e f g h\r\n \r\n\r\ni j k l\r\n";
	string s;
	
	is.str(b);
	
	while (getline(is, s)) {
		
		if (s.length() == 1 && s[0] == '\r')
			cout << "an empty line found" << endl;
		cout << s << endl;
		
	}
	
	return 0;
}


This post has been edited by c.user: 13 January 2010 - 03:33 AM

Was This Post Helpful? 0

#5 r.stiltskin   User is offline

  • D.I.C Lover
  • member icon

Reputation: 2032
  • View blog
  • Posts: 5,435
  • Joined: 27-December 05

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 06:15 AM

View Postfir3fly, on 13 Jan, 2010 - 12:16 AM, said:

My Question is : Should i manipulate the seekg or the buffer side to do this?

Obviously there are many ways you can do this, but my LAST choice would be to search in the actual file by manipulating the seek pointer.

Probably the fastest way is to read the entire file into a char buffer (as you have done) and then scan the buffer for two consecutive '\n' characters. Alternatively, you can use a c++ string as an input buffer and then use the global getline() function to read the file line by line, each time checking for a line of length 0.
Was This Post Helpful? 0
  • +
  • -

#6 fir3fly   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 34
  • Joined: 12-January 10

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 06:41 AM

View Postr.stiltskin, on 13 Jan, 2010 - 05:15 AM, said:

View Postfir3fly, on 13 Jan, 2010 - 12:16 AM, said:

My Question is : Should i manipulate the seekg or the buffer side to do this?

Obviously there are many ways you can do this, but my LAST choice would be to search in the actual file by manipulating the seek pointer.

Probably the fastest way is to read the entire file into a char buffer (as you have done) and then scan the buffer for two consecutive '\n' characters. Alternatively, you can use a c++ string as an input buffer and then use the global getline() function to read the file line by line, each time checking for a line of length 0.


Yes I was planning to do the first method. However, I'm not sure of what functions or methods to use to scan the buffer.
Was This Post Helpful? 0
  • +
  • -

#7 JackOfAllTrades   User is offline

  • Saucy!
  • member icon

Reputation: 6246
  • View blog
  • Posts: 24,014
  • Joined: 23-August 08

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 06:56 AM

A carriage return-line feed is not really a \n character, it is a \r character (carriage return) and a \n character (newline character), or a string of "\r\n".

Relevant
Was This Post Helpful? 0
  • +
  • -

#8 r.stiltskin   User is offline

  • D.I.C Lover
  • member icon

Reputation: 2032
  • View blog
  • Posts: 5,435
  • Joined: 27-December 05

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 07:25 AM

View Postfir3fly, on 13 Jan, 2010 - 08:41 AM, said:

Yes I was planning to do the first method. However, I'm not sure of what functions or methods to use to scan the buffer.

You're just looking for a pattern (either '\n' '\n' or '\r' '\n' '\r' '\n') of characters in the file, so the most straightforward way is to set up a loop from buffer[0] to buffer[size-1] and check each individual character. Give it some thought. You'll learn more if you try to come up with your own ideas.
Was This Post Helpful? 0
  • +
  • -

#9 NickDMax   User is offline

  • Can grep dead trees!
  • member icon

Reputation: 2255
  • View blog
  • Posts: 9,245
  • Joined: 18-February 07

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 04:39 PM

IF you are dealing with text data -- why are you using binary file IO? Why not just use getline() to get the various lines?
Was This Post Helpful? 0
  • +
  • -

#10 fir3fly   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 34
  • Joined: 12-January 10

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 05:31 PM

View PostNickDMax, on 13 Jan, 2010 - 03:39 PM, said:

IF you are dealing with text data -- why are you using binary file IO? Why not just use getline() to get the various lines?


Because for some reason, I cannot use string getline() functions as the data i'm interested is in RAW binary data. Hence, it cannot be fully extracted if i used the getline() methods.

Therefore, I have no choice but to use binary I/O as in my code suggests, i can use that to extract the RAW data out.
Was This Post Helpful? 0
  • +
  • -

#11 NickDMax   User is offline

  • Can grep dead trees!
  • member icon

Reputation: 2255
  • View blog
  • Posts: 9,245
  • Joined: 18-February 07

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 07:07 PM

The the data is "RAW binary data" then why are you looking for newlines? What kind of crazy file format you are reading?
Was This Post Helpful? 0
  • +
  • -

#12 r.stiltskin   User is offline

  • D.I.C Lover
  • member icon

Reputation: 2032
  • View blog
  • Posts: 5,435
  • Joined: 27-December 05

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 08:25 PM

View Postfir3fly, on 13 Jan, 2010 - 07:31 PM, said:

Because for some reason, I cannot use string getline() functions as the data i'm interested is in RAW binary data. Hence, it cannot be fully extracted if i used the getline() methods.
Did you actually try this? I ask this because regardless of what the data represents, it is still just a sequence of bytes in a file. You said that the file is organized in lines terminated by newline characters. Well, getline(is, str) will extract bytes from is
and store them in str until it reaches a newline character, so I don't see why you can't use that. Alternatively, you can use istream::getline to store the data in a char[] buffer instead of a string.

But anyway, if you can't or don't want to use getline, what's the problem with simply scanning the char buffer which you already have containing the entire file to find two consecutive newlines?

This post has been edited by r.stiltskin: 13 January 2010 - 08:33 PM

Was This Post Helpful? 0
  • +
  • -

#13 r.stiltskin   User is offline

  • D.I.C Lover
  • member icon

Reputation: 2032
  • View blog
  • Posts: 5,435
  • Joined: 27-December 05

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 10:15 PM

By the way, as far as I have been able to determine, each HTTP header line is terminated by CRLF, not just LF, so you are looking for sequences ending with '\r' '\n' or in other words, pairs of octets (bytes, 8-bit chars) containing the decimal values 13, 10.

Are you making any progress?

This post has been edited by r.stiltskin: 13 January 2010 - 10:15 PM

Was This Post Helpful? 0
  • +
  • -

#14 fir3fly   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 34
  • Joined: 12-January 10

Re: Search for a blank line in binary mode data

Posted 13 January 2010 - 11:56 PM

View Postr.stiltskin, on 13 Jan, 2010 - 09:15 PM, said:

By the way, as far as I have been able to determine, each HTTP header line is terminated by CRLF, not just LF, so you are looking for sequences ending with '\r' '\n' or in other words, pairs of octets (bytes, 8-bit chars) containing the decimal values 13, 10.

Are you making any progress?

Yes i have... Thanks Guys... One more thing... Is there a way to remove elements from an array using C++?
Was This Post Helpful? 0
  • +
  • -

#15 Guest_c.user*


Reputation:

Re: Search for a blank line in binary mode data

Posted 14 January 2010 - 01:03 AM

fir3fly said:

I cannot use string getline() functions


 
#include <iostream>
#include <string>
#include <sstream>

using namespace std;

// makes an istringstream
int main(void)
{
	istringstream is;
	const char *b = "e f g h\r\n \r\n\r\ni j k l\r\n";
	char s[300];
   
	is.str(b);
   
	while (is.getline(s, sizeof s)) {
	   
		if (s[1] == '\0' && s[0] == '\r')
			cout << "an empty line found" << endl;
		cout << s << endl;
	   
	}
   
	return 0;
}


Was This Post Helpful? 0

  • (2 Pages)
  • +
  • 1
  • 2