[mando414]

i want to be able to read in a file called doc.txt of arbitrary length and not get a seg fault, i know i have to allocate on the heap but am having trouble doing so.

#include <iostream>	
#include <string> 	
#include <fstream>	

using namespace std ;

int main() {


	char *file_name = "doc.txt" ; 

	ifstream fin ;
	fin.open( file_name ) ;

	if( ! fin ) {
		cout << "Problems opening " << file_name << endl ;
		return -1 ;
	}



	const unsigned MAX = 100 ; 
	string doc[MAX] ;

	unsigned word_count = 0 ;
	//while( fin >> doc[ word_count++ ] ) ;

	while( fin >> doc[ word_count ] ) {
		cout << doc[ word_count ] << endl ;
		word_count ++ ;
	}


	fin.close() ;

	return 0 ;
}

[Karel-Lodewijk]

A this is C++, it might be easier just to read it onto a string. This trick will do so and is fairly efficient:

#include <sstream>
...
std::stringstream sstr;
sstr << file.rdbuf();
std::string contents = sstr.str();

This post has been edited by Karel-Lodewijk: 22 February 2012 - 08:11 PM

[mando414]

im trying to learn the basics of allocating to the heap, i can allocate ints and chars fine normally, but the problem im having is reading in the contents of a file, and allocating that to the heap.

[Karel-Lodewijk]

Well you would start by getting the file size.

fin.seekg (0, std::ios::end); //puts file pointer at the end
size_t length = fin.tellg();  //gives position
fin.seekg (0, std::std::beg); //important, put file pointer back

Then you can allocate a buffer of the appropriate size.

char* buffer = new char[length];

And then you would fill the buffer.

fin.read(buffer, length);

There are no doubt other ways to go about it, but this will work and won't waste space.

This post has been edited by Karel-Lodewijk: 22 February 2012 - 08:32 PM

[mando414]

i want to pretend like i have no way of knowing the file size or obtaining the file size, just read how ever much there is and allocate to heap, i want the only limitation to be size of physical memory ur machine has.

[Karel-Lodewijk]

Being difficult are we

Well you would start by initializing a buffer of a certain size. Then you would read that amount of bytes or until eof(end of file). If you don't reach the end of file, you would allocate a new buffer that is larger, typically you would multiply the size by some multiplier > 1 because it gives a lower asymptotic complexity. Copy everything you've read and continue reading from the file.

If you don't mind using some C functions. Allocating with malloc will allow you to do the allocation of the new buffer with realloc. Realloc has the advantage that it will try to grow the buffer if there is trailing space and it will do the copying for you if there is not.

something like:

size_t size = 10;
size_t old_size = 0;
char* buffer = new char[size];
size_t read = 0;

while (true) {
    read += fin.read(buffer + old_size, size-read);

    if (fin.eof()) break;

    char* old_buffer = buffer;
    old_size = size;
    size *= 2;

    buffer = new char[size];
    std::copy(old_buffer, old_buffer+old_size, buffer);
    delete[] old_buffer;
}

or more c-style

size_t size = 10;
size_t old_size = 0;
char* buffer = (char*)malloc(size);
size_t read = 0;

while (true) {
    read += fin.read(buffer + old_size, size-read);

    if (fin.eof()) break;

    old_size = size;
    size *= 2;
    buffer = realloc(buffer, size);
}

Not tested, proceed with caution, I would much prefer the first 2 methods

Edit: Already found a mistake, you should offcourse start writing further into the buffer (buffer+old_size).

This post has been edited by Karel-Lodewijk: 22 February 2012 - 09:15 PM

[mando414]

instead of malloc could i do something like

char *buffer = new char[MAX] ;

This post has been edited by mando414: 22 February 2012 - 09:19 PM

[Karel-Lodewijk]

mando414, on 23 February 2012 - 04:17 AM, said:

instead of malloc could i do something like

double *dvalues = new double[MAX] ;

If you want to use realloc, no, the C and C++ heap functions do not mix. Either you use malloc/realloc/calloc and free or you use new/new[] and delete/delete[].

Also mind that the parameter of malloc is the size in bytes, so if you use it for doubles, use:

double *dvalues = (double*) malloc (MAX*sizeof(double));

This post has been edited by Karel-Lodewijk: 22 February 2012 - 09:21 PM

[mando414]

only because im slightly new i think im gonna use the new, new[] way of working, im slightly more familiar with them than using C functions.

yea sorry i pasted from old code, the file im reading in shouldnt contain any doubles

This post has been edited by mando414: 22 February 2012 - 09:26 PM

[mando414]

what would the c++ equivalent to realloc be?

[Karel-Lodewijk]

Allocating a new buffer and copying it's contents as demonstrated.

It's more complicated and less efficient, that's why I suggested the realloc in the first place.

I've always wondered why C++ doesn't have a standard function to resize 'new' allocated memory. They could have called it renew. I guess constructors got in the way.

This post has been edited by Karel-Lodewijk: 22 February 2012 - 09:37 PM

[mando414]

still having trouble trying to read in file "doc.txt". i think i have allocating to the heap figured out though. any suggestions based on what i have with allocating to the heap.keeping in mind that this is for ints and im trying to read in a file of strings

#include <iostream>

using namespace std ;

struct raw_node
{
	int x ;
	struct raw_node* next ;
} ;

typedef struct raw_node node ;


int main()
{ 

	node *head, *current ;

	head = new node() ;
	head->next = 0 ;

	current = head ;

	cout << "Enter your list of positive ints.  End with a negative or zero." << endl << endl ;

	int temp ;
	do
	{

		cout << "Next: " ;
		cin >> temp ;
		current->x = temp ;
		current->next = new node() ;
		current = current-> next ;
		current->next = 0 ;
	}
	while( temp > 0 ) ;

	current = head ;

	do
	{
		temp = current->x ;
		cout << temp << endl ;
		current = current->next ;
	}
	while( current->next != 0 ) ;

	return 0 ;
}

[Bench]

Karel-Lodewijk, on 23 February 2012 - 05:30 AM, said:

I've always wondered why C++ doesn't have a standard function to resize 'new' allocated memory. They could have called it renew. I guess constructors got in the way.

I expect they could have provided one for copyable objects, although I guess that's already catered for in std::vector. The ISO committee are always trying to make C++ a more memory-safe language, so a separate realloc function would have been pretty much obsolete already (not much point adding a feature and then going around telling everybody how bad it is and not to use it straight afterwards :-) )

[ishkabible]

Quote

I've always wondered why C++ doesn't have a standard function to resize 'new' allocated memory. They could have called it renew. I guess constructors got in the way.

I've actually never understood why C++ made operators for allocation; to me it seems to be something that should always be handled by a class or set of functions. std::allocator could have had a realloc but it didn't work out that way

think about it, how do you make an OS in C++ if C++ requires dynamic memory allocation as a fundamental? that part always has to be done in C or assembly.

also, I don't think constructors would be an issue. if the the old data is getting deleted only to copy the new data you can just do a binary copy. smart pointers wouldn'and file handles wouldn't be affected even. You would have to define it in the standard so that it didn't call any constructors or destructors though.

This post has been edited by ishkabible: 24 February 2012 - 03:05 PM

[Karel-Lodewijk]

ishkabible, on 24 February 2012 - 10:02 PM, said:

Quote

I've always wondered why C++ doesn't have a standard function to resize 'new' allocated memory. They could have called it renew. I guess constructors got in the way.

I've actually never understood why C++ made operators for allocation; to me it seems to be something that should always be handled by a class or set of functions. std::allocator could have had a realloc but it didn't work out that way

think about it, how do you make an OS in C++ if C++ requires dynamic memory allocation as a fundamental? that part always has to be done in C or assembly.

also, I don't think constructors would be an issue. if the the old data is getting deleted only to copy the new data you can just do a binary copy. smart pointers wouldn'and file handles wouldn't be affected even. You would have to define it in the standard so that it didn't call any constructors or destructors though.

Well for all practical purposes new and delete are functions. I can perfectly write you a new<Type>(constructor_params) function that acts as a new and passed the given parameters directly to the constructor, and accompanying delete2 function. In c++11 I can anyhow.

The major difference between C's memory allocation function and C++'s memory allocation function is basically that the C functions only allocate memory, they do not initialize they dot interpret what type it is, you ask for x bytes and get x bytes. C++'s new does 2 things, it allocates memory, just as C does, but after that it types the memory and starts initializing the data. i.e.: running constructors or in case of built-in types initializing with given value if provided. Therefor a renew/realloc would call for a bunch of new constructors to run or if you make it smaller a bunch of destructors. And if you have to copy the memory because of a realloc, do you run a bunch of copy constructors and destructors afterwards, in some use case you might have to for something to work properly, in some use cases it will break ?

You say, just doing a binary copy is often not that straightforward objects might have pointers or references pointing to things within the object itself or there might be external pointers/references pointing to something within an object, these pointers/references will all be invalid afterwards. Similar to how relocations within an stl container invalidates iterators. Keeping track of these things can be done but at quite a bit of hidden cost. C's realloc also has a similar problem but C has no references and it simply states that realloc invalidates internal pointers.

std::allocater actually works different than new/delete. It is created in part because for high-performance dynamic containers new and delete is not very efficient. For example When you create a vector, you want to allocate a large enough block of memory so your container can grow, but you really can't use new for this because you don't want to run a bunch of constructors for objects that do not exist. It might not even be possible, the type might not have a default constructor at all. So std::allocator has 3 function allocate/construct/destroy, which brings us back to the C world really where allocating memory and interpreting/initializing/constructing are 2 separate things. You could do something similar with a placement new when initializing and placing those objects in larger blocks of memory allocated by C's memory function. std::allocater might very well be implemented this way.

About the OS part, you can write an OS in C++, initially you cannot use virtual memory, C++ does not require it as a fundamental, only the stl does. But the way an OS or rather a kernel handles memory is different than how a user process does. One of the first thing a kernel does is run some bios(the kernel's os ) functions to map all available memory, so it knows which ranges are safe to use. After that it can implement dynamic memory for itself and can start handing out pieces of memory to user processes that request it. A kernel has no limit

Anyway I've strayed way off topic here.

This post has been edited by Karel-Lodewijk: 24 February 2012 - 05:28 PM