5 Replies - 1092 Views - Last Post: 26 March 2020 - 02:46 AM

#1 salazar   User is offline

  • D.I.C Addict

Reputation: 111
  • View blog
  • Posts: 660
  • Joined: 26-June 13

How to unit test code whose output is an image?

Posted 23 March 2020 - 12:21 AM

I'm writing this android project that deals with images. I have business logic that deals with images, specifically creating Bitmaps programmatically. The issue i'm having is how I should unit test that logic.
It uses a Canvas and Pain object to create a Bitmap.
The only way I know how to verify that the code works is by displaying the Bitmap in an ImageView. In other words, I don't know how else to create the expected Bitmap other than writing the code that is supposed to be tested and then running to make sure it works.

I'm trying to use TDD and write the tests first. But in this case, how can I?

This is my testing setup
CreateBitmap {
    public static Bitmap createBitmap(Input in) {}
}

UnitTest {
    public void test() {
        // Load input data
        Input in = ...
        // Load expected Bitmap
        Bitmap expectedBm = ...
        Bitmap actualBm = CreateBitmap.creatBitmap(input);
        assertEquals(expectedBm, actualBm);
    }
}


Any replies are welcomed.

This post has been edited by salazar: 23 March 2020 - 12:21 AM


Is This A Good Question/Topic? 3
  • +

Replies To: How to unit test code whose output is an image?

#2 Salem_c   User is offline

  • void main'ers are DOOMED
  • member icon

Reputation: 2452
  • View blog
  • Posts: 4,601
  • Joined: 30-May 10

Re: How to unit test code whose output is an image?

Posted 23 March 2020 - 01:12 AM

https://www.raymond....looking-images/

ImageMagick compare and perceptual diff are tools I've used in the past.
IIRC, you can get some kind of command line numerical pass/fail stats out of them.
Was This Post Helpful? 1
  • +
  • -

#3 Ornstein   User is offline

  • D.I.C Head

Reputation: 103
  • View blog
  • Posts: 209
  • Joined: 13-May 15

Re: How to unit test code whose output is an image?

Posted 23 March 2020 - 01:20 AM

I don't think there's any industry-standard solution to this particular problem - so you'll just have to use your best judgement.

The first and most obvious suggestion is per-pixel comparison - if you're able to compare against an expected output image. You may instead find it faster to calculate and compare a hash (MD5/other) of the two images.

Knowing your particular algorithm/requirements, perhaps you could think up a more creative solution.
Was This Post Helpful? 1
  • +
  • -

#4 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2410
  • View blog
  • Posts: 5,050
  • Joined: 11-December 07

Re: How to unit test code whose output is an image?

Posted 23 March 2020 - 01:59 AM

Here is one idea. I am going to assume that your output is made by drawing rectangles and ovals. Whatever you are doing, you can probably adjust this idea to fit.

For the above scenario, write two helper methods:

void assertRectangle(x, y, width, height, Color);
void assertOval(x, y, width, height, Color);

Inside these two methods, you can sample some key pixels to distinguish between the two shapes. For example, the corner of the bounding box should be coloured for rectangles but not for ovals.

Then follow the normal TDD cycle.

1. Add an assertion for a shape that is missing.
2. Add this shape to the production code.
3. Refactor production and test code.

You will probably have an easier time if you go for lots of simple image conversion tests rather than a small number of complex ones.

Obviously, you're unlikely to be doing rectangles and ovals. But it should be possible to come up with some assertions that describe the effects that you are creating.

If you follow this then the tests you end up with will also describe what the image should look like and therefore be more readable than a pixel for pixel comparison, or building hashes.

Taking the readability more seriously, you could add a String description to the assertion methods. That way, a more helpful message could be displayed when the test fails:

assertRectangle("door", 350, 350, 100, 250, Color.RED);
This could fail with the message:
Could not find door in the image. Expected a rectangle at 350, 350, 100, 250 coloured #ff0000.
Was This Post Helpful? 2
  • +
  • -

#5 salazar   User is offline

  • D.I.C Addict

Reputation: 111
  • View blog
  • Posts: 660
  • Joined: 26-June 13

Re: How to unit test code whose output is an image?

Posted 25 March 2020 - 09:04 PM

View PostSalem_c, on 23 March 2020 - 01:12 AM, said:

https://www.raymond....looking-images/

ImageMagick compare and perceptual diff are tools I've used in the past.
IIRC, you can get some kind of command line numerical pass/fail stats out of them.

Thanks, I tried searching for some. But they aren't exactly what i'm looking for. I will look into your suggestions.

View PostOrnstein, on 23 March 2020 - 01:20 AM, said:

I don't think there's any industry-standard solution to this particular problem - so you'll just have to use your best judgement.

The first and most obvious suggestion is per-pixel comparison - if you're able to compare against an expected output image. You may instead find it faster to calculate and compare a hash (MD5/other) of the two images.

Knowing your particular algorithm/requirements, perhaps you could think up a more creative solution.

Doing a hash comparison is a good idea. I didn't think of that. Thanks.

View Postcfoley, on 23 March 2020 - 01:59 AM, said:

Here is one idea. I am going to assume that your output is made by drawing rectangles and ovals. Whatever you are doing, you can probably adjust this idea to fit.

For the above scenario, write two helper methods:

void assertRectangle(x, y, width, height, Color);
void assertOval(x, y, width, height, Color);

Inside these two methods, you can sample some key pixels to distinguish between the two shapes. For example, the corner of the bounding box should be coloured for rectangles but not for ovals.

Then follow the normal TDD cycle.

1. Add an assertion for a shape that is missing.
2. Add this shape to the production code.
3. Refactor production and test code.

You will probably have an easier time if you go for lots of simple image conversion tests rather than a small number of complex ones.

Obviously, you're unlikely to be doing rectangles and ovals. But it should be possible to come up with some assertions that describe the effects that you are creating.

If you follow this then the tests you end up with will also describe what the image should look like and therefore be more readable than a pixel for pixel comparison, or building hashes.

Taking the readability more seriously, you could add a String description to the assertion methods. That way, a more helpful message could be displayed when the test fails:

assertRectangle("door", 350, 350, 100, 250, Color.RED);
This could fail with the message:
Could not find door in the image. Expected a rectangle at 350, 350, 100, 250 coloured #ff0000.


Your post, cfoley, was extra helpful. I think I've figured out a way to test the images efficiently where I just sample features I need. That is much easier. Using your method, I think I even know how to create the expected images without having to write the program, which would violate TDD. I appreciate the replies very much.
Was This Post Helpful? 1
  • +
  • -

#6 cfoley   User is offline

  • Cabbage
  • member icon

Reputation: 2410
  • View blog
  • Posts: 5,050
  • Joined: 11-December 07

Re: How to unit test code whose output is an image?

Posted 26 March 2020 - 02:46 AM

Thanks for the feedback! Good luck with your image testing.
Was This Post Helpful? 1
  • +
  • -

Page 1 of 1