How many of you work somewhere where there are actual end users helping test the software. What is your process? I would think it could become a royal pain in the rear because they would constantly be changing the scope of the features. What is your process for end user testing. As soon as you get a new feature out, at the end of a sprint...how does this work? How frustrating is it dealing with the end users?

This post has been edited by skyhawk133: 30 Nov, 2007 - 08:42 PM

End user input can only be taken so much into account. Why? Because every end user wants something different. I'm not hating on a particular OS here, but just take a look at Vista. It is the result of TOO much user input I think. Like "handy" little buttons and widgets everywhere. Which all consume ram and overload the screen with stuff, etc. Parameters for design must be decided ahead of time so that whatever you're working on can be streamlined to that end. If you want to allot for more things then what I think is the best solution is allowing modules to be incorporated into the software (like with a scripting language of some sort). So that if end users want something, well they can code it themselves, and the main workability of the program remains intact. The only thing I think end users should contribute to a project is perhaps layout and bug testing. I designed a website recently and it was horrible because the client couldn't decide on colors or schematics or blah blah blah. So it was just trying to structure her into picking something concrete and going from there, and just being admant about not changing it so much and that too much change would cost time and money, because sometimes the indecisiveness of the user or the "I want the whole world" attitude just isn't feasible.

Heres the process (for testing) we use at work:

1. Developer performs unit testing as he is developing the new functionality/product, using NUnit
2. Once developer is complete (in his judgment) it is passed to a 2nd developer who reviews the code, and when time allows, tests the new functionality.
3. From there it is passed to the QA Department, along with all documentation and functional specification documents, where they rigorously test the new functionality/product for approximately 7-10 days. All QA testers are non-developers, no coding background experience whatsoever so they test as an end user.
4. Once QA approves it is then pushed out to the users of that application for use. If, and when, they want something added to the new functionality/product they open a Technology Request which is prioritized by the BA's
5. Repeat 1 - 4 as needed

Remember, we don't develop software and web applications for the outside world, our "consumers" are the members of the department that has requested the new functionality/product.

HAH! When I worked at a fortune 500 company, they would push projects out based on date, not on completion. That's why the DNS server would be down, the network communications would be down, Dallas would be on constant backup ISDN, when a sales user hooked their laptop into the 10/100 switch & fired up outlook the network would crash. Test servers ran different versions of Tom-Cat & JavaC than the production servers. & those were even different than the development servers!


By the time the product got to them, it had more holes then swiss cheese. They were not happy =-)

Once a new feature is in the build, then QA tests it. After that it goes to UAT. Fortunately we are not directly communicating with the end users (operators) themselves, but with their organizer / lead tester / manager whatever. So any shiny things the end user dreams of, we don't hear it directly, it has to go throught their managers - and cost considerations must be taken into consideration. At this point we are getting to a simple point of success: if that manager is good, then the whole end user testing is a piece of cake. If he is not competent, then most of the worries mentioned in the op apply. Right now we have both.

Your suggestions are good, GrimPirate I can imagine where you are coming from I meant end users to help find bugs, I realize if we were to assign them this task then bug reports would not be the only thing reported. New features and change in scope would most definitely happen. This would be controlled by the client, their employee may want something to be one way, but what the owner says goes.

No.2 What you say sounds very familiar, but w/o the server problems, just bugs that got left in the system, because we don't really have a true QA department. The people who do QA have other major responsibilities, they don't necessarily have time to go through every little case. This is something I have no control over, but the idea is up in the air to try and get a person that just does solely QA

Psycho unit testing is something that we are dying to implement but just can't find the time. How long do you think it would take a team who is unexperienced in unit testing to adapt it, and how would you suggest them start it. In every project they do...or start fresh w/ a new project?

Lacca what is UAT?

The biggest issue where I work is not the end users changing their minds, but the BAs getting the end user's requirements wrong in the first place. Our users are often employees of our parent company, though, so coordinating testing with them is a little easier. Once a release has moved out of unit testing and past QA testers, we open it up to the users to test it and report bugs via Test Director.

user acceptance testing

For most of the UI related work, I'm practically the only developer. I deal with some backend processes too, but my other colleagues handle those most of the time.

So the process in my department works something like
1. Developers code and test by themselves
2. Release to QA department to test
3. QA approve and we release to production

In my case, the QA department only tests anything involving billing information. So if the request has nothing to do with billing stuff, then step 2's gone. And yeah, the QA people are no coders, so they act as end users.

If it's for internal users, and the request involves a major change or addition to existing applications, then I try to get the users involved in testing.

I've coded web applications for public customers (of my internal users) too. Those projects basically went straight to production. I was the only one coding, there was a really short timeline (they want the customer to sign contracts quickly), and there weren't any transactional stuff going on, so it's ok by the users. If my users' customers found the application easy to use, then that's all the testing needed. (talk about pressure...)

Most of my change requests are minor, so I'm to just migrate code to production (barring any billing stuff involved). Then my users just use the feature or new screen or whatever in production mode. Hah! I only get one chance at this...

Anyway, my users know me personally, so they call or email me directly. No middle people, just go straight to the source (code ( r )). I've been training them to go through my team leader...

This post has been edited by orcasquall: 4 Dec, 2007 - 06:46 AM

Brief answer:
unit test->integration test->smoke test->system test->acceptance test->alpha, beta, gamma-> user acceptance test.

Note that smoke test (may be skipped) and acceptance tests are run by the QA team, while alpha, beta, and gamma ones are usually (not always) done by "pedestrians" or some people who were chosen from the target group.