In software development industry it is often discussed if it is necessary to have quality assurance on the projects. And what is the most effective presence of testing part in projects. In other words what is the most effective ratio of programmers and testers?

The ratio of programmers and testers can be different depending on a project. This doesn’t mean that large number of testers will provide their work effectively. The most effective average proportion is where a number of testers and developers is considered as 1 to 2.

In order to show this, let’s imagine a project with 4 developers working on it. For the best ratio of quality we should employ 2 testers.

When we don’t have testers on the project, only 10% of bugs can be found by programmers. When we were young and enthusiastic (but not as clever as today), starting our new project, we thought that well motivated and qualified programmers can provide their work in a best way.

But it’s obvious that everybody can make mistakes, and few programmers can find these mistakes.

If we have 2 testers they will approximately find 85% of bugs. Here is the rule that it is impossible to find 100% of bugs.

If we employ 3 testers they will find 90% of bugs, but their work won’t be effective and will cost more money. The price of each found error will be higher.

Thus, we will achieve the best ratio of efforts and the best quality if we have 2 testers and 4 programmers. Every additional tester will find some bugs but general price of his work is higher than effectiveness. Only on vitally important projects such as healthcare and financial projects this ineffective (we say here ineffective meaning expensive) work should be applied, because potential defect can be very expensive.

The same situation comes up when we talk of the time spent on a bug search: if we spend too much time on testing, it will become ineffective.
So, the task of testing is very important in the context of software development. We understand that the ideal ratio of testers and programmers can be applied only to the average projects. But the main idea is that the cost of the quality assurance may take up to 50 percent of software development cost.

And what is the situation on your projects? We know that it depends on the project, but may be you can formulate average (ideal) ratio.

There is no defined ration that exists. Software Engineering is a very dynamic process and continually changes.
One of the principles they enforce at my work environment is a technique called TDD (Test Driven Development).
At the end of this month we are having a professional on TDD come to speak and teach us techniques to apply it.
http://en.wikipedia.org/wiki/Test-driven_development
Anyone who programs knows that some bugs can be very difficult to detect. Have you ever experienced a segmentation fault
or null pointer exception?

This post has been edited by trixt.er: 9 Sep, 2009 - 10:04 AM

So for your wiki link it's basically saying before I implement anything into my main project I should test exactly what I want to add first?

like if I wanted to add a loop that printed 100 peoples names I should do it in a separate test program before putting it into my main program. I mean things will be harder to test than others but this is the main idea right?

What about things that don't need to be tested and how do you decide how big something should be before you have to test it?
like that loop would be pointless to test since the only thing that could really go wrong is a syntax error how complex must something be before you decide you have to put it through tests is what I'm trying to say I guess

because I test a lot of things before I put it in but then I don't test a lot of things as well
sometimes I loose track of my main goal because I'm testing too much or what I was trying to accomplish was a lot harder than I thought and I get discouraged and move on to something else.

You probably don't really need TDD for anything small. Most often it is for large projects where things could easily break and it would be harder to debug in between multiple people.

Most small methods would not need to be tested at all. Just the larger methods.

On my current project there are 10 developers and 3 testers total. Unit test have been already written for about 90% of all methods, but we still receive 150+ new bugs (not usability) per week. So, developers can test their code by themselves, but it's absolutely necessary to have manual testers, who can check relations between big and difficult modules i guess.

Now we have a project for Hospitality Industry. it's a large Web Portal created specially for Hospitality exhibition.
Customer refused all testing services. And now programmers test their code themselves. I don't think that it's good but Customer's desire is a law....

According to TDD methodology i know that in my company nobody uses this one, because we have mostly small projects with limited budget, and customers are happy to know that it won't take much money and time to implement particular application.

Beware of the trap of knowing the cost of everything and the value of nothing. In other words, what is the value of being able to trap 5% more bugs (using your numbers from your example)? Would that extra 5% of bugs fixed prevent a hacker from being able to download the credit card database or do a SQL injection?

The law of diminishing marginal returns doesn't account for bad press, irate users and lost revenue due to insufficient testing.



It seems to me, from another post you made in this thread, that your real constraint isn't the number of testers but the amount of money the customer is willing to pay for the project. If your customer really does not understand the cost of inadequately tested code, then you either need to educate him or find a better customer. IMHO.

cdk

My solution to the problem is to adapt a workflow system with built-in quality assurance. Enforcing coding standards is one such example. Another is code inspection and walk-through.

Over the years, I have adapted the ideas of Tom Gilb in his book Principles of Software Engineering Management and combine it with CMMI concepts.

Thanks a lot, cdk! You have excellent expressed situation. It is realy hard to assure clients that this price of software product i is not exaggerated, but minimum. When we speak about cost of quality, as for example in our article http://andersensoft.com/en/article_quality.php all says that quality is very important, that they prefer pay for it. But in reality it is no so.


kanshu you have really good advice! Sometimes we realy use this method.