[Donpa]

Here is my situation. I have a single jpg (size: 4000,4770) of 100 images (400 x 477 each). The reason I combined them into a single image was to reduce the size of the program (have over a hundred of these combined images). When I load up my program to redivide these images it takes around 6 seconds to divide a single combined image into single bitmaps for use (100 images again). Since I have to do this over 100 times it'll take over 10 minutes just to boot up the program. So, I've come here to see if anybody might have any ideas on how to speed up my division.

Note.. The divided images will only be used in the program and will not be saved out.

Here is a moc version of my spilt function

 private Bitmap[] spiltImage(Bitmap wImage)
        {
            Stopwatch timer = new Stopwatch();
             List<Bitmap> images = new List<Bitmap>();

            Rectangle cropArea = new Rectangle(400, 477, 400, 477);
            Bitmap bmpImage = new Bitmap(cropArea.Width, cropArea.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
            
            Graphics g = Graphics.FromImage(bmpImage);
            timer.Start();    
            for (int y = 0; y < 10; y++)
            {
                for (int x = 0; x < 10; x++)
                {
                    g.Clear(Color.Black);
                    g.DrawImage(wImage, -cropArea.X * x, -cropArea.Y * y);
                    images.Add(new Bitmap(bmpImage));

                    label2.Text = "X: " + x + "  Y: " + y;
                    Application.DoEvents();
                }
            }
            timer.Stop();
            MessageBox.Show(timer.Elapsed.ToString());
            return images.ToArray();
        }

Any ideas?

[tlhIn`toq]

Donpa, on 09 November 2011 - 11:09 AM, said:

Here is my situation. I have a single jpg (size: 4000,4770) of 100 images (400 x 477 each). The reason I combined them into a single image was to reduce the size of the program (have over a hundred of these combined images).

Other than minor improvement for object overhead, and image storage in 32bit strides even for 24bit images and so on - how do you expect this to save you any substantial amount of memory? Its the same amount of data. If I have 100 one dollar bills or 1 one-hundred dollar bill I still have a hundred bucks.

4000 x 4770 = 19,080,000 pixels. At 32bits per pixel thats 4 bytes per pixel, so 80 megs per image in rough terms. If you have 100 such images you will need 8000 megs or 8 gigs of ram just for the images if you are pre-loading them all.

Quote

When I load up my program to redivide these images it takes around 6 seconds to divide a single combined image into single bitmaps for use (100 images again)

But here's the rub, as soon as you start dividing them you now have duplicated the memory footprint needs. Because you have them in their consolidated form AND their split-up counter-parts. Now you need 16 gig of RAM just for your images. So how is this scheme saving you anything?

Back to the speed problem... Stop combining and dividing. Its probably not saving you enough hard disk space to worry about. Have you empirically compared what it is really doing for you? I mean have you made a version with 100 combined images and one with all the images as separated individual images and compared the two?

Also, line 19 where you have the GUI update the status is a HUGE slow down. Don't do anything GUI or console.writeline-ish in this loop.

Just start the timer then end the timer and check the end time of actual processing without the GUI update slowdown.

Also, if you insist on continuing down this path you probably want to put this work on its own thread so it doesn't lock your user interface. Then free up the program for use when it reports completed.

[Donpa]

Quote

Other than minor improvement for object overhead, and image storage in 32bit strides even for 24bit images and so on - how do you expect this to save you any substantial amount of memory? Its the same amount of data. If I have 100 one dollar bills or 1 one-hundred dollar bill I still have a hundred bucks.

4000 x 4770 = 19,080,000 pixels. At 32bits per pixel thats 4 bytes per pixel, so 80 megs per image in rough terms. If you have 100 such images you will need 8000 megs or 8 gigs of ram just for the images if you are pre-loading them all.

The folder containing a set of images (100) = 2.30 - 2.69mb~ (400,477)
A combined image = 802k~ (4000,4770)

(Sizes vary based on the detail of the image)

Quote

Have you empirically compared what it is really doing for you? I mean have you made a version with 100 combined images and one with all the images as separated individual images and compared the two?

It was one of the first things I did. The only reason why I'm really set on this path is because I was able to combine 11k images (270mb) into 110 images (98mb) which reduces the size of the program greatly. I've already completed the program that uses these images I looking for areas I can improve it.

The idea I was going for was to preload all the images at the start of the program to allow for quick execution of the images during the program. I could load a single image in .0025 of a millisecond (to screen), but if all of the images were in single form then the size of the program would be 270mb+.

Most likely I'll have to just deal with the large database of images because it doesn't appear I'll be able to preload them quickly enough to justify combining them. I've tried cloning & drawing sections with poor results. Combining them has clear benefits, but unless I can preload/divide the images quick enough it isn't worth doing.

[tlhIn`toq]

When you said you were trying to reduce the size of the program, I assume you meant the memory consumption of the program, not the HDD footprint. My mistake.

I hate to sound jaded by today's large computers but - it saves you 270-98 = 172 megs of hard drive footprint. So what? That's petty in today's world. Its not going to change the delivery from a CD to floppy. Either way you deliver on DVD or broadband. Once your download approaches the 100 meg mark anyway nobody really cares about the extra 172 megs. At 300 megs total it is still less than most people download for a single TV show and far less than the monthly Microsoft updates. What does that save someone downloading it? 2 minutes?

People aren't going to care about the download or HDD footprint like they are the responsiveness of the application - Or the minimum PC requirements to run it.

Tell someone 4 minute download or 2 minute download and they'll shrug. But tell someone they have to have a minimum of 20 gigs of ram in their PC because your application will require 16 just for images and they'll scream. Tell them that in addition to the 20 gigs they have to allow 6 minutes to boot up and you're toast.

You did follow my calculation of the memory consumption earlier, right? Just because they take 270megs of COMPRESSED jpg space doesn't mean that's all the space they'll each in RAM once you turn them into Image objects, which are really just uncompressed bitmaps with some extra features.

This post has been edited by tlhIn`toq: 09 November 2011 - 12:46 PM

[Donpa]

Yeah, the ram would have been another issue on our slower computers. Eh, I guess I'll just stay with the image data base. Like I said, I was merely looking for ways to improve the program. Compressing the size of the program would have been a nice change, but wasn't nessecary. Thanks anyway

[tlhIn`toq]

07  Bitmap bmpImage = new Bitmap(cropArea.Width, cropArea.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);

Something to look at would be whether you really need full 32bit images for everything. You can drop a LOT of consumption if you can use a smaller custom color pallet.

Also, do you really need all the images at the same time?
If you only need 20 images for "Mode 1" or "Level 1" then just load those on the fly. When you need the 20 images for "Level 6" then load them. Most people are used to seeing screens for "Loading next level..." in games and such.

[Donpa]

tlhIn`toq, on 09 November 2011 - 01:02 PM, said:

07  Bitmap bmpImage = new Bitmap(cropArea.Width, cropArea.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);

Something to look at would be whether you really need full 32bit images for everything. You can drop a LOT of consumption if you can use a smaller custom color pallet.

Also, do you really need all the images at the same time?
If you only need 20 images for "Mode 1" or "Level 1" then just load those on the fly. When you need the 20 images for "Level 6" then load them. Most people are used to seeing screens for "Loading next level..." in games and such.

The program that uses these images only loads one image at a time when needed. The reason I was trying to preload them was because of the delay from having to spilt them.

This post has been edited by Donpa: 09 November 2011 - 01:15 PM

[AdamSpeight2008]

But it still requires the storage of the large image, in memory. So doesn't save you anything.

[tlhIn`toq]

I think he's looking at installer size, not memory footprint.
All of this seems to be about reducing the installer - probably for smaller download. That's just a guess.

[Serendipitus]

Assuming your talking about the program loading up, why don't you just have a background task loading the images and open up the interface to use. Just make sure it handles all the use cases!