[hpeeze]

I work for a small gas meter refurbishing company and reduce the amount of human errors i want to create an application that will allow me to take a picture of a meter index which has 4 dials, the application would use a webcam and when the index is placed in front of the camera it would determine the angles/position of the dials and determine a number

in the attached image, the determined number should read: 9578

the index would be placed in an exact location every time and at a specific distance from the camera.

would this be possible and how should i go about creating something this?

i am pretty fluent in VB/VB.net and know a couple people who know many different languages and would be able to help me along with this project.



Note:
i know this is kind of a double post since i also posted something very similar in the mobile development section. however i would still like a version of this application for a computer that we use to enter the reading of each index into a database along with the meter it belongs to and other related information.

[Asscotte]

Id say this is a little to much for vb.net but if you want to then you would need a stock image that is perfect in every single way and has no background, this image would be the one to compare it against next you would need to find a way of detecting where the dials where, i dont know how to do it but it should be quite simple (or not). Erm one other thing, this could be quite hard as if you are slightly out the image wont detect quite right and also you would have to do some checks for different coloured pointy bits. if you really think it through it could be quite gd.

Things to include image recognition ie checking if ceartin pixels arein ceartin places

once image is recognised checking what the actual figure is again cheking where the pixels are but then calculating where they are pointing (MATH ) then adding the total up.

enjoy,

[krum110487]

I would do this.

make an algorithm that will find the center/beginning of the line, then you will check all the surrounding colors and the darkest black you go to, then you do that until you get to a point where the next colors aren't black (color of the pointer) except for where you came from.... for example...

SXXXXXX
XSXXXXX
XXSXXXX
XXXSXXX
XXXXSXX
XXXXXSX
XXXXXXT

now this may be kinda hard to read, but starting at "T" (which is the center) and the "S" are the rest of the pointer.

From T it will look at the top left, top, top right, middle right, bottom right, bottom, bottom left and left.

it will find the next closest color at "Top left" so then it will go there and check all of the same ones again (Not including the one you came from)

once you get to the last "S" there is nothing left over to move to, so now you have the beginning and end.

calculate the slope.

this will be MUCH harder than I believe it will be worth...

This post has been edited by krum110487: 03 April 2009 - 11:00 AM

[Predictor]

hpeeze, on 3 Apr, 2009 - 06:35 AM, said:

I work for a small gas meter refurbishing company and reduce the amount of human errors i want to create an application that will allow me to take a picture of a meter index which has 4 dials, the application would use a webcam and when the index is placed in front of the camera it would determine the angles/position of the dials and determine a number

in the attached image, the determined number should read: 9578

the index would be placed in an exact location every time and at a specific distance from the camera.

would this be possible and how should i go about creating something this?

i am pretty fluent in VB/VB.net and know a couple people who know many different languages and would be able to help me along with this project.



Note:
i know this is kind of a double post since i also posted something very similar in the mobile development section. however i would still like a version of this application for a computer that we use to enter the reading of each index into a database along with the meter it belongs to and other related information.

I would think that once the new image is registered (image processing speak for "aligned with a standard reference"), you could establish a ring of pixels to check for each dial. The hand for each dial must cross that rin at some point. Each pixel on a ring would have an associated output. Given the comparatively wide arc used for each integer value displayed on a dial, this should be fairly accurate.


-Will Dwinnell
Data Mining in MATLAB

[hpeeze]

i did some research and learned i could use "GetPixel" in vb.net to get the pixel color from an image but i cant find any information on how to get that to work with a live webcam stream.

if i can get the getpixel to work then i can set a static range of pixels it can check for and if the hand on the dial is black at a certain pixel it would check a few more pixels around it to make sure its really looking at the dial and then use that to determine which number the dial is pointing at

does anyone have any idea on how i can get it to work with a live webcam stream?

to me this seems like the easiest method and could also be pretty easy to port it over to a windows mobile device with a built in camera for use out in the field.


i could also use the getpixel method to get pixel colors in the center of the dial and the tip of the hand and determine the angle of the hand to get the numerical reading which is another possibility im looking at

[hpeeze]

for now i have decided to start with just 1 dial instead of all 4

in the following picture you see blue lines going from 1 number to the next and if you look closely you can see there is a small gap between the lines and therefor the blue lines arent connected.

i think it may be easier to use the getpixel api and have it search along a blue line to see if a black pixel is present
if a black pixel is present it would then determine the numerical reading for that dial (would be the lower number of the 2 that the hand is between)

the complex part would be when the dial is exactly on a number and even that wouldn't be too complex.



that is a low quality version i am working with for now until i get a better webcam that can focus somewhat clearly on all 4 dials at the same time


i just need to figure out how to make the getpixel api to work and also how to make it only search along that blue line (blue line will not be in the picture when its actually searching for the black pixel) instead of giving it exact coordinates of where to look for a black pixel

once i figure those 2 things out, the rest is all down hill.
thanks for everyone that has helped so far and i appreciate any and all help/advice one may have

[krum110487]

a couple ideas

1) since you have control of the meter why don't you color it (Physically) like so:

the dial is black, the marks for the numbers are red and the rest is white, if the camera has good color then you should be able to find each color very easily and compare. You could even put a peice of blue tape on the very end of the dial to show the tip.


2) you will most likely want to take a "Snapshot" of the camera and analyze that.

3) if you want it on a mobile device the best thing to do would be make an online database and have your (always running) program update the database, and have the palm program check that for values (that would save on calculations for the mobile device)

You could even have it interact with your program to "refresh" on the fly.


If you want a SUPER accurate dectection system, I would set the cameras up and manually set where the tick marks are, then all you would have to worry about is where the dial is...and as I stated before, if you make the middle and tip of the dial a different color, it would be easy to find both.

This post has been edited by krum110487: 06 April 2009 - 02:53 PM

[hpeeze]

the problem with coloring the dials is that the dials are covered by a clear plastic cover once the indexes go on the meter so it would be impossible to color them

also, the gas company does not want us to alter the indexes in any way

[LemonMan]

There are many books on image processing you can read that would help you.

[hpeeze]

i already have the finding a black pixel part figured out, i just need to figure out how to make the program automatically scan each pixel on a line after i give it the starting and ending points.

here is what im doing to draw t he line(s)

		e.Graphics.DrawLine(Pens.PaleGreen, 194, 43, 180, 47)
		e.Graphics.DrawLine(Pens.PaleGreen, 176, 51, 167, 62)
		e.Graphics.DrawLine(Pens.PaleGreen, 166, 67, 166, 82)
		e.Graphics.DrawLine(Pens.PaleGreen, 167, 87, 175, 99)
		e.Graphics.DrawLine(Pens.PaleGreen, 180, 102, 194, 107)
		e.Graphics.DrawLine(Pens.PaleGreen, 199, 107, 212, 102)
		e.Graphics.DrawLine(Pens.PaleGreen, 216, 100, 226, 87)
		e.Graphics.DrawLine(Pens.PaleGreen, 227, 82, 227, 67)
		e.Graphics.DrawLine(Pens.PaleGreen, 226, 63, 217, 51)
		e.Graphics.DrawLine(Pens.PaleGreen, 214, 48, 199, 43)

as you can see, it draws 10 lines that are slightly separated (will explain the separation part later)
i want to make vb scan each of those lines to see if a black pixel exists underneath it

here is the image im analyzing:

NOTE: the image is enlarged to show details, actual size of the image is 320x240
NOTE2: this picture is just temporary to get the basics in place, the end result will be a non distorted and very sharp picture

the red lines represent the lines that VB draws over the existing image
the black pixel im searching for is actually the hand on that dial (represented by the yellow pixel at the intersection of the hand and the red line)

im using getpixel() to get the pixel's color at any given coordinate

eventually the program would scan each dial to find out where the hand is pointing and determine a numerical reading of all of the dials combined.

how would i code an algorithm that will automatically scan each of the lines for a black pixel and stop scanning once it finds a black pixel?

This post has been edited by hpeeze: 09 April 2009 - 10:15 AM

[microstoc]

It's an interesting problem.

Can I offer another approach that maybe simpler to compute.
This proposed method is an alogorithm to determine the 'angle' of the pointer by inspecting the black/dark pixels

The middle part of each dial has a circular black blob which is the center of the pointer.
Coming off that circular black blob is the pointer arm. The arm sweeps 360 degrees, so there can be 360 possible angles for the pointer.

Imagine drawing a square in the center of the dial face.
The square would contain the circular black blob and part of the black arm.
The square can be considered to be a grid of pixels or cells.
Say the grid was 500 wide by 500 pixels high. That would equal 2500 cells.
Each cell in the grid of 2500 cells is assigned a value, from 1 to 2500. The top left cell has a value of 1, the last cell on line 1 has a value of 500, the bottom right cell has a value of 2500.

Start scanning the grid, left to right, line by line, top to bottom.
Before you start scanning, set an accumulator counter to zero.
The program scans each line, pixel by pixel. If a pixel is black then get the value of the cell at that position and add that value to the accumulated value. Each time a black pixel is found, a value is added to the accumulator.
When you have finished scanning you will have an accumulated value in the counter.
The counter value will be different for each orientation of the pointer. There could be 360 different values.

You will need a master array that holds 360 possible values, one value for each of the possible angles. The values in this master array have been determined beforehand, using the scanning method above. The master array values are hard coded into the program.

Now, simply compare your accumulated value against the values in the master array and determine the angle of the arm. You can then determine the reading for the dial from thereon.

HTH

Best of luck with it.

[AdamSpeight2008]

microstoc, on 9 Apr, 2009 - 05:16 PM, said:

The counter value will be different for each orientation of the pointer. There could be 360 different values.

They could all be the same.
Dial 1

11110
11101
11011
11111
11111

Dial 2

11111
11111
11011
10111
01111

The different dial values, both with the same "counter value".
Then you have to take account the effect of the images not being perfect copies of the ideal dial.

Image Analysis is often done using neural nets (Don't ask me, I don't understand them either.)

This post has been edited by AdamSpeight2008: 09 April 2009 - 02:21 PM

[microstoc]

AdamSpeight2008, on 9 Apr, 2009 - 01:14 PM, said:

microstoc, on 9 Apr, 2009 - 05:16 PM, said:

The counter value will be different for each orientation of the pointer. There could be 360 different values.

They could all be the same.
Dial 1

11110
11101
11011
11111
11111

Dial 2

11111
11111
11011
10111
01111

The different dial values, both with the same "counter value".
Then you have to take account the effect of the images not being perfect copies of the ideal dial.

Image Analysis is often done using neural nets (Don't ask me, I don't understand them either.)

Thanks Adam, they're valid comments. I'm glad someone is reading and understanding. lol.

I figured that too, but left it out to keep the post simple and not too long winded. I looked at neural nets too, way over my head and too much too learn in a short time.

I'm pretty sure the 'grid' can be numbered so as to give unique values for each of the 360 positions. Maybe put all 1's in the grid squares that run along the 1 degree line, 2's along the 2 degree line etc.

Also it is odds on that the 'derived values' will not correspond exactly with the 360 stored values due to differences in the actual images. I was thinking that a 'close match' might work here.

I'm also wondering if the photgraphed images of the dials will be consistent and if they will closely match say a master photograph, everytime. Some trials would show this. The camera angle and distance from the dials would have to be kept constant. The dials must always be in the same place and be the same size on every photograph.

Hah, this interests me. I have been playing around with bitmap code for a couple of hours. I have managed to convert the single dial image (posted in this topic) to a pure black and white image of pretty decent quality without too much fuzz. (Had to learn a little bit about this bitmap coding, so it's been an education).

I have also managed to read all the pixels in the 320x240 black and white image, and stored them into a 'grid' array. It was easy identying blacks pixels and white pixels.

I'm just about to figure out the actual coordinates for the 'square' in the 320x240 pixel image. The 'square' will enclose the hub of the pointer and some of the pointer arm.


Steve

[AdamSpeight2008]

The grids I wrote are to represent pixels Black (0) and white(1).

[microstoc]

AdamSpeight2008, on 9 Apr, 2009 - 03:51 PM, said:

The grids I wrote are to represent pixels Black (0) and white(1).

Yes I see that.