VB.NET memory limit

  • (2 Pages)
  • +
  • 1
  • 2

24 Replies - 11561 Views - Last Post: 05 December 2011 - 08:42 AM Rate Topic: -----

#1 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2392
  • View blog
  • Posts: 5,025
  • Joined: 11-December 07

VB.NET memory limit

Posted 03 December 2011 - 04:37 PM

I'm working with VB.NET express 2010 and the program I am writing gives an OutOfMemoryException at about 1.2GB. However, my machine easily has 1 GB of free space. To see if I could reproduce the problem, I wrote this test program which keeps extending a linked structure. The same happens. At about 1.2GB of memory, I get an OutOfMemoryException.

I was wondering if this is a limit in the Express version. I have spent some time today googling for possible causes and solutions. I found some info on older versions running on 32 bit systems (mine is 64 bit) but nothing that seems to match my experiences here.

If anyone could help me increase the limit for my program or at lease explain why I can't I would be very grateful.

A great solution would be to reduce the amount of memory that my program needs. Unfortunately, I've dome as much as I can on that front. My input data files are over 50GB binary files and I'm doing some pretty heavy graph computations. I should be able to get by if VB will let me use 2 GB of memory.

Module Module1

    Sub Main()
        Dim first = New Node
        Dim current = first
        Do
            current.nextNode = New Node
            current = current.nextNode
        Loop
    End Sub

End Module

Class Node
    Public nextNode As Node
    Private bigArr(1000) As Integer
End Class


Is This A Good Question/Topic? 1
  • +

Replies To: VB.NET memory limit

#2 DimitriV   User is offline

  • vexing conundrum
  • member icon

Reputation: 587
  • View blog
  • Posts: 2,746
  • Joined: 24-July 11

Re: VB.NET memory limit

Posted 03 December 2011 - 04:53 PM

Usually an OutOfMemory exception is a .Net Framework problem.
I had a look and the OutOfMemory exception occurs when a program uses too much memory in the RAM, not the HDD.
So how much RAM do you have in your computer? Even if you have 2GB it won't let you use all two GB as it has to leave room for windows to run as well as vital programs and services.

Just because you use VB Express there shouldn't be a limit imposed apart from those imposed by the .Net framework.
Was This Post Helpful? 1
  • +
  • -

#3 AdamSpeight2008   User is offline

  • MrCupOfT
  • member icon

Reputation: 2298
  • View blog
  • Posts: 9,535
  • Joined: 29-May 08

Re: VB.NET memory limit

Posted 03 December 2011 - 04:56 PM

Do you really need to have the full 50Gb in memory?
Have you consider using a stream of the data?
Was This Post Helpful? 0
  • +
  • -

#4 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2392
  • View blog
  • Posts: 5,025
  • Joined: 11-December 07

Re: VB.NET memory limit

Posted 03 December 2011 - 05:08 PM

Adam, It's about 2GB of memory I need.

Jimmy, I have 4 GB of RAM. At the point when the program crashes, I still have 1 GB free (Task manager is showing RAM useage at 75% so I don't think it's because it's allocated to the graphics chip or anything like that.)

It's good to know Express should not make a difference here.

One question: in Java, you have to specify the maximum heap size when you start the program. Are there any switches like that in .NET?
Was This Post Helpful? 0
  • +
  • -

#5 AdamSpeight2008   User is offline

  • MrCupOfT
  • member icon

Reputation: 2298
  • View blog
  • Posts: 9,535
  • Joined: 29-May 08

Re: VB.NET memory limit

Posted 03 December 2011 - 05:31 PM

My questions still hold.
Was This Post Helpful? 0
  • +
  • -

#6 DimitriV   User is offline

  • vexing conundrum
  • member icon

Reputation: 587
  • View blog
  • Posts: 2,746
  • Joined: 24-July 11

Re: VB.NET memory limit

Posted 03 December 2011 - 05:31 PM

View Postcfoley, on 04 December 2011 - 10:08 AM, said:

Adam, It's about 2GB of memory I need.

Jimmy, I have 4 GB of RAM. At the point when the program crashes, I still have 1 GB free (Task manager is showing RAM useage at 75% so I don't think it's because it's allocated to the graphics chip or anything like that.)

It's good to know Express should not make a difference here.

One question: in Java, you have to specify the maximum heap size when you start the program. Are there any switches like that in .NET?

Now, I'm not going to pretend that I know what a heap size is in java, so I will assume that it's like an internal buffer? I am pretty sure that with a lot of classes and objects you can pass parameters such as internal buffer max size (an Integer value, BTW).
Was This Post Helpful? 0
  • +
  • -

#7 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2392
  • View blog
  • Posts: 5,025
  • Joined: 11-December 07

Re: VB.NET memory limit

Posted 03 December 2011 - 05:47 PM

The heap size is really the amount of RAM it can use.

Adam, it's possible that I could do stiff to reduce the memory footprint. I'd rather not because it would take hours to refactor my work and I bet it would hit performance badly. I'd rather not do this, especially because I have the hardware right now. We are evaluating VB and if we go ahead, we will be running it on a server with over 10 GB of free RAM.

The comments in my opening post were really just to pre-empt people suggesting I change my code to use less memory. From a purely academic point of view, I'd like to make my silly linked list grow till it fills up all my RAM. Can anyone help?
Was This Post Helpful? 0
  • +
  • -

#8 DimitriV   User is offline

  • vexing conundrum
  • member icon

Reputation: 587
  • View blog
  • Posts: 2,746
  • Joined: 24-July 11

Re: VB.NET memory limit

Posted 03 December 2011 - 05:48 PM

That is technically impossible - the system cannot allocate all of its memory to a single process.
Was This Post Helpful? 0
  • +
  • -

#9 trevster344   User is offline

  • The Peasant
  • member icon

Reputation: 225
  • View blog
  • Posts: 1,526
  • Joined: 16-March 11

Re: VB.NET memory limit

Posted 03 December 2011 - 06:02 PM

An app developed for commercial use, I'd say that's your go ahead to invest a little more money. ;) The difference is great, you're not gonna have access to a lot of great controls but that's neither here nor there. Sounds to me like your possible options are money, or time. :D
Was This Post Helpful? 0
  • +
  • -

#10 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2392
  • View blog
  • Posts: 5,025
  • Joined: 11-December 07

Re: VB.NET memory limit

Posted 03 December 2011 - 06:12 PM

Sure. If I go ahead with VB, I'll shell out for the full product. But nothing I have read has even hinted that it would solve my current problem.

Quote

That is technically impossible - the system cannot allocate all of its memory to a single process.


Aaah now we are getting somewhere. Why not, and what's the limit? I have 4 GB of RAM on a 64 bit system. The OS and whatever else I have running takes up 2 GB. Why can't I have one program take up the other 2 GB?
Was This Post Helpful? 0
  • +
  • -

#11 DimitriV   User is offline

  • vexing conundrum
  • member icon

Reputation: 587
  • View blog
  • Posts: 2,746
  • Joined: 24-July 11

Re: VB.NET memory limit

Posted 03 December 2011 - 06:14 PM

I'll look into that.
Was This Post Helpful? 0
  • +
  • -

#12 trevster344   User is offline

  • The Peasant
  • member icon

Reputation: 225
  • View blog
  • Posts: 1,526
  • Joined: 16-March 11

Re: VB.NET memory limit

Posted 03 December 2011 - 06:16 PM

The amount of memory you have available isn't precise. In fact my operating system is taking more than 2 G's. So you'll need to look into the actual amount of free space you have.

This post has been edited by trevster344: 03 December 2011 - 06:18 PM

Was This Post Helpful? 0
  • +
  • -

#13 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2392
  • View blog
  • Posts: 5,025
  • Joined: 11-December 07

Re: VB.NET memory limit

Posted 03 December 2011 - 06:51 PM

For reference, I did the same thing in Java. The TLDR is that it's perfectly possible to fill up 94% of my RAM. One I go over that it gets interesting. So, this leaves 3 options.

1. VB on the CLR Just can't do it. (Don't believe that!)
2. VB Express has a built in limit that the full version doesn't have (I've seen no evidence of this)
3. I just don't know how to make it use all the memory. (most likely)

For reference, here is mt Java code:

public class EatMem {
	int[] arr = new int[1000];
	EatMem next;
	
	public static void main(String[] args) {
		EatMem first = new EatMem();
		EatMem now = first;
		while (true) {
			now.next = new EatMem();
			now = now.next;
		}
	}
}


and I run it with:
java -Xint -Xmx3000m EatMem


-Xint disables some of the optimisations. In this case, it works out I never do anything with the linked list and goes into an infinite loop never running out of memory (clever, but not what I want today!) -Xmx3000m tells Java I'm letting it use 3000 MB.

First I ran it with 2048 MB and the task manager reported it using ~2,121,000 K before I got an OutOfMemoryError. By that time, the memory had flatlined and it was using virtual memory. I then tried it with Xmx3000 -- way over my free RAM and it got to 3,087,000 K using RAM and virtual memory.

The really interesting thing is that it made Windows do a little housekeeping and the rest of the stuff that was taking 2 GB squashed down to 800 MB. That's the operating system, Chrome and whatever else I was using.

So, this is all to prove that my OS will let me run processes taking 2 GB or 3 GB of RAM. Something about visual studio or VB is capping it at 1.2 GB.

This post has been edited by cfoley: 03 December 2011 - 06:53 PM

Was This Post Helpful? 1
  • +
  • -

#14 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2392
  • View blog
  • Posts: 5,025
  • Joined: 11-December 07

Re: VB.NET memory limit

Posted 04 December 2011 - 09:34 AM

For information, I asked a friend to run my snippet on his full version of visual studio. He got the exception at about 1.2 GB too. Looks like this issue is not limited to the express version.

I'm also more convinced than ever that there must be a setting somewhere that I am missing.
Was This Post Helpful? 0
  • +
  • -

#15 trevster344   User is offline

  • The Peasant
  • member icon

Reputation: 225
  • View blog
  • Posts: 1,526
  • Joined: 16-March 11

Re: VB.NET memory limit

Posted 04 December 2011 - 09:51 AM

Then your only option is to go through your programs code and change those variables, limit that memory usage.
Was This Post Helpful? 0
  • +
  • -

  • (2 Pages)
  • +
  • 1
  • 2