7 Replies - 952 Views - Last Post: 09 January 2011 - 01:28 AM

#1 karimkhan_it   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 6
  • Joined: 07-January 11

Storing images coming from telescope

Posted 07 January 2011 - 04:47 AM

I want to store images coming from telescope at 24 locations . Images coming at rate of 200 /sec or more. Then after reducing these 24 images into two images have to deal with next 24 images .Next 24 images should replace previous 24 images as 25th on 1st position like vice . No need for storing all the images here , just have to work on current 24 images . can anyone help me how can it be done using c++ ?

thanks....
Is This A Good Question/Topic? 0
  • +

Replies To: Storing images coming from telescope

#2 Salem_c   User is offline

  • void main'ers are DOOMED
  • member icon

Reputation: 2194
  • View blog
  • Posts: 4,271
  • Joined: 30-May 10

Re: Storing images coming from telescope

Posted 07 January 2011 - 06:07 AM

> Images coming at rate of 200 /sec or more
Do this calculation.

How large is each image (in megabytes)? Let's say it's 10MB, since these are coming from some nice hi-res CCD cameras on the telescopes.

So 200 * 10MB = 2GByte/second.

Bandwidth 101
USB3 (theoretical is 5Gbits/sec) - http://www.everythin...rspeed-usb.html
Gigabit ethernet is (you guessed it) 1Gbit/sec - http://en.wikipedia....igabit_Ethernet

The first big question is, how are you proposing to get the data into the machine in the first place?
A single interface of either kind just isn't going to cut it.

Maybe you have 24 separate USB leads (from the 24 cameras on the 24 telescopes).

Some useful things would be
- a diagram of how you're going to wire everything together.
- the specs of your computer that's going to be running this application.

This is nowhere near being a C++ question yet. This is all "system design" to make sure that the basic parameters are plausible.
Shoving 2GB/Sec into your standard off the shelf PC is not feasible, even if you were just ignoring the data.

Doing some rather interesting image manipulation in real-time makes it even less so.
Was This Post Helpful? 1
  • +
  • -

#3 JackOfAllTrades   User is offline

  • Saucy!
  • member icon

Reputation: 6246
  • View blog
  • Posts: 24,014
  • Joined: 23-August 08

Re: Storing images coming from telescope

Posted 07 January 2011 - 06:28 AM

I agree, this is more of a design discussion at this point, so I'll move this to Software Development.
Was This Post Helpful? 0
  • +
  • -

#4 karimkhan_it   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 6
  • Joined: 07-January 11

Re: Storing images coming from telescope

Posted 08 January 2011 - 12:07 AM

View PostSalem_c, on 07 January 2011 - 05:07 AM, said:

> Images coming at rate of 200 /sec or more
Do this calculation.

How large is each image (in megabytes)? Let's say it's 10MB, since these are coming from some nice hi-res CCD cameras on the telescopes.

So 200 * 10MB = 2GByte/second.

Bandwidth 101
USB3 (theoretical is 5Gbits/sec) - http://www.everythin...rspeed-usb.html
Gigabit ethernet is (you guessed it) 1Gbit/sec - http://en.wikipedia....igabit_Ethernet

The first big question is, how are you proposing to get the data into the machine in the first place?
A single interface of either kind just isn't going to cut it.

Maybe you have 24 separate USB leads (from the 24 cameras on the 24 telescopes).

Some useful things would be
- a diagram of how you're going to wire everything together.
- the specs of your computer that's going to be running this application.

This is nowhere near being a C++ question yet. This is all "system design" to make sure that the basic parameters are plausible.
Shoving 2GB/Sec into your standard off the shelf PC is not feasible, even if you were just ignoring the data.

Doing some rather interesting image manipulation in real-time makes it even less so.



Sorry for last empty post ,

well , I would like to say that I in this project other guys also working , everyone has different phase to work , my side is just to store images coming from telescope , size is expected of 2MB , i have 8 - core cpu parallel processing system to deal with the task , is there any code or methods which allow me to store images at 24 location and and then I can use it soon after storing , 24 location should be overwritten for every next 24 images..
Was This Post Helpful? 0
  • +
  • -

#5 Salem_c   User is offline

  • void main'ers are DOOMED
  • member icon

Reputation: 2194
  • View blog
  • Posts: 4,271
  • Joined: 30-May 10

Re: Storing images coming from telescope

Posted 08 January 2011 - 12:54 AM

So show us your calculations of the data transfer capabilities of the various components of the system.

400MB/Sec (based on your latest figure) is still an awful lot to deal with.
Was This Post Helpful? 0
  • +
  • -

#6 horace   User is offline

  • D.I.C Lover
  • member icon

Reputation: 768
  • View blog
  • Posts: 3,832
  • Joined: 25-October 06

Re: Storing images coming from telescope

Posted 08 January 2011 - 02:46 PM

The maximum I have acquired data at was 64Mbytes/sec from a image scanner and I used double buffers, i.e. concurrently
buffer 1: stores the last scanned image, now being processed
buffer 2: stores the next image as it is scanned (in practice using DMA to read from scanner directly into memory)
when buffer 2 was full they are switched (one assumes that processing takes less time than reading an image).

I assume you could get some concurrency in processing with your 8 core system.
How does the data arrive? I doubt if gigabit ethernet could cope with 400Mbyte/second when one adds in protocol overheads

This post has been edited by horace: 08 January 2011 - 02:47 PM

Was This Post Helpful? 0
  • +
  • -

#7 Salem_c   User is offline

  • void main'ers are DOOMED
  • member icon

Reputation: 2194
  • View blog
  • Posts: 4,271
  • Joined: 30-May 10

Re: Storing images coming from telescope

Posted 09 January 2011 - 12:42 AM

> i have 8 - core cpu parallel processing system to deal with the task
The number of cores is irrelevant whilst the problem remains I/O bound. The problem with modern PC architectures is that the memory and I/O subsystems are shared by all 8 cores. It is only when you can get each core working primarily out of each separate L1/L2 cache that you will really see a benefit in the computational performance.

Lets say (for discussion) that you choose to use 24 USB lines. No problem you say, 20 to 40MB/Sec down each USB3 line - no problem at all!.
Ah, but then the next I/O bottleneck will be the internal PCI bus where all these USB interfaces congregate. Can that cope with the combined data rate of all the I/O interfaces that you're proposing to use?

I don't know, but it is something you need to figure out from looking at the system specs of your chosen platform. As yet, you haven't given us anywhere near enough information to even guess one way or the other.

What you're saying at the moment is that you've got 8 buckets to empty the bath. What you haven't realised yet is that you've only got one tap (and only ONE tap) to fill the bath to begin with.
Was This Post Helpful? 0
  • +
  • -

#8 horace   User is offline

  • D.I.C Lover
  • member icon

Reputation: 768
  • View blog
  • Posts: 3,832
  • Joined: 25-October 06

Re: Storing images coming from telescope

Posted 09 January 2011 - 01:28 AM

I would assume that the "i have 8 - core cpu parallel processing system" was not an 8 core PC but a specialist parallel processor which appropriate inconnectivity and IO capability ?? Programming them is none trivial requiring the use of specialist APIs, etc. I have never used such a system myself but had collegues who did.
I guess we need more details of the actual configuration.

This post has been edited by horace: 09 January 2011 - 01:29 AM

Was This Post Helpful? 0
  • +
  • -

Page 1 of 1