14 Replies - 1126 Views - Last Post: 04 May 2013 - 08:59 PM

#1 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3572
  • View blog
  • Posts: 11,106
  • Joined: 05-May 12

Waterfall method and the DRY principle

Posted 27 April 2013 - 09:41 PM

I'm currently frustrated by a current project that is using the Waterfall method, so take most of what I write below with a grain of salt.

As I'm taking a step away to get a break this weekend from a project that is following the Waterfall method with lots and lots of ceremony and their attendant artifacts, I'm left musing if the DRY (Don't Repeat Yourself) principle were articulated by Andy Hunt and Dave Thomas back in the 60's, would the Waterfall method have never existed?

I guess my major frustration is that the artifact that comes out of the Requirements Gathering is a list of requirements and constraints. But to assemble the Design Document, the requirements and constraints need to be restated to verify that all requirements and constraints are met. Once a Design Document in assembled, the Test Plan needs to restate the requirements, constraints, as well as the designed components, activity diagrams, etc. and list tests that verify the requirements, constraints, components, activity states, etc. There seems to be a lot of repetitive work that is happening here.

I tried to suggest the use of a wiki instead of plain old documents so that the repetitiveness and cross-referencing would have a better chance of being kept up to date, but management wants documents and so more time is actually spent updating documents whenever something is discovered downstream and all the upstream documents need to be updated, and be run by all the stakeholders all over again. Argh! Talk about the Waterfall Method being all WET (Write Everything Twice).

Do you think that the Waterfall method would have still been invented if the DRY principle had been articulated and gotten traction earlier during the development of computer science/engineering?

Is This A Good Question/Topic? 0
  • +

Replies To: Waterfall method and the DRY principle

#2 modi123_1  Icon User is online

  • Suitor #2
  • member icon



Reputation: 9196
  • View blog
  • Posts: 34,544
  • Joined: 12-June 08

Re: Waterfall method and the DRY principle

Posted 27 April 2013 - 09:56 PM

That's not quite the way I've done the whole 'waterfall' method.. basically once a doc's written, and signed off, it is just part of the repository for the project. No need to restate it, and definitely no need to recopy it (lord knows someone may try and sneak something in or remove something).

On the plus side I've seen a chunk of work stop dead in the requirements phase or the design phase.. thankfully never to be actually worked out. This means, with limited warm bodies to throw at a problem, multiple projects can queue up while I am doing work. Though that does require a decent BA and PM to make sure I have all the needed parts when it gets to me.
Was This Post Helpful? 2
  • +
  • -

#3 BobRodes  Icon User is offline

  • Your Friendly Local Curmudgeon
  • member icon

Reputation: 574
  • View blog
  • Posts: 2,989
  • Joined: 19-May 09

Re: Waterfall method and the DRY principle

Posted 28 April 2013 - 06:06 PM

You may find it enlightening to read this. Royce, the "father of the waterfall method", was actually pointing out the flaws in what is now accepted as the waterfall method. This is another favorite of mine, talking about the interaction of people and methodologies. Finally, we mustn't overlook http://www.waterfall2006.com/ for an interesting take on the waterfall model in general.
Was This Post Helpful? 1
  • +
  • -

#4 baavgai  Icon User is online

  • Dreaming Coder
  • member icon

Reputation: 5823
  • View blog
  • Posts: 12,675
  • Joined: 16-October 07

Re: Waterfall method and the DRY principle

Posted 29 April 2013 - 05:01 AM

How requirements are stated and documented vary greatly. Sounds like your methodology is, um, different.

Once your have requirements set, they're done. It's like a financial closing, you don't get to go back and screw with it. If someone comes by later and points out you're missing something, you ideally tell them to suck it up as it's not part of the current project. If the "change" is simply a recognition of the scope of the requirements, then it's not really a change at all but only a design detail.

Requirements should be what MUST happen. The HOW of that can, and usually does, change a lot between design and implementation. I think of design as a working hypothesis, where implementation is an attempt to prove it, if that makes sense. You might convince yourself that design can be written in stone prior to implementation work, but in practice that doesn't seem to happen. Implementation points out the holes in design and they tend to feed off each other. However, the requirements, the description of the results of this work, should, by definition, not be fluid.

I tend to believe that the quantifying of things like development processes and design patterns mostly give people a warm fuzzy. They are a "proven" pattern that, if simply followed correctly, will yield the desired result. While this idea appeals to project managers and their superiors, the reality is that no magic bullet exists and you ultimately have to deal with it.

Don't get me wrong, having some kind of well defined foundational framework is useful, if not required. However, you still have to be willing to adapt when elements of the chosen methodology aren't working.
Was This Post Helpful? 1
  • +
  • -

#5 jon.kiparsky  Icon User is online

  • Pancakes!
  • member icon


Reputation: 7743
  • View blog
  • Posts: 13,080
  • Joined: 19-March 11

Re: Waterfall method and the DRY principle

Posted 30 April 2013 - 08:37 AM

I agree with baavgai - it sounds to me like the problem is not with waterfall, but with your management.

The worst of all worlds, in my view, is the semi-agile process. This is where you have design documents and requirements and specs and all of that - in principle at least - but nobody feels constrained to lock those down. This is what you're dealing with here. I feel your pain, because it's what I'm working with now myself. I don't know what'll happen with your project, but I'm looking at a project that's well over a year overdue, and no realistic projected end date. If you can't get management to get a serious process, it might be time to give the resume a spit shine. This looks like a rough ride.


Quote

I tried to suggest the use of a wiki instead of plain old documents so that the repetitiveness and cross-referencing would have a better chance of being kept up to date, but management wants documents and so more time is actually spent updating documents whenever something is discovered downstream and all the upstream documents need to be updated, and be run by all the stakeholders all over again.


There's no reason why design documents can't be developed with modern collaborative tools, stored in wikis or repositories, kept under change control, whatever. The form of the document isn't interesting. It's the fact that the design document is law: now everyone knows what they're building to and testing against. This can be useful, for all concerned. An agile™ process I think has definite advantages, but there are lots ways to manage projects, and waterfall can be fine.

You still need to do all of the depth-first projections for any changes to the documents, but they don't need to be written with goose quills on parchment scrolls.
Was This Post Helpful? 1
  • +
  • -

#6 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3572
  • View blog
  • Posts: 11,106
  • Joined: 05-May 12

Re: Waterfall method and the DRY principle

Posted 01 May 2013 - 06:10 AM

There are two pain points:
1) The churning of the Requirements document as pointed out by baavgai above. Unfortunately, it's not "like a financial closing, you don't get to go back and screw with it."; and
2) Each of the downstream documents have a Traceability matrix that map back to the Requirements document and other upstream documents. Churn anything upstream, and things downstream get churned. Or if not churned, time and energy be spent to do a crosscheck as to what the effects would be downstream.

As much as Kent Beck's WordUnit was described in jest in the link provided by BobRodes, it seems like it maybe a useful tool especially if the mantra is "you break it, you fix it". So if an upstream requirement is changed, then the changer gets to fix all the downstream artifacts he broke.
Was This Post Helpful? 0
  • +
  • -

#7 jon.kiparsky  Icon User is online

  • Pancakes!
  • member icon


Reputation: 7743
  • View blog
  • Posts: 13,080
  • Joined: 19-March 11

Re: Waterfall method and the DRY principle

Posted 01 May 2013 - 06:43 AM

View PostSkydiver, on 01 May 2013 - 08:10 AM, said:

So if an upstream requirement is changed, then the changer gets to fix all the downstream artifacts he broke.


I so want to introduce this in my office.
Was This Post Helpful? 0
  • +
  • -

#8 baavgai  Icon User is online

  • Dreaming Coder
  • member icon

Reputation: 5823
  • View blog
  • Posts: 12,675
  • Joined: 16-October 07

Re: Waterfall method and the DRY principle

Posted 01 May 2013 - 08:05 AM

You broke you fix it sounds awesome in theory. In practice, the ones who break seem incapable of fixing...

View PostSkydiver, on 01 May 2013 - 09:10 AM, said:

Unfortunately, it's not "like a financial closing, you don't get to go back and screw with it."


I completely understand. How it SHOULD be and how it IS too often fails to mesh.

But that's the point of following a methodology. To take well defined, proven, practices and then follow them. If you aren't following the practices that define the methodology, then you aren't using that methodology; by definition.

People like to say they follow a named practice, because then failure is the fault of the practice and not the manager. We hire contractors for the same reason; when they fail, it's never the fault of the person who hired them or the employees that dealt with them. Always the contractor failed.

Never underestimate the power of a scapegoat.
Was This Post Helpful? 0
  • +
  • -

#9 BobRodes  Icon User is offline

  • Your Friendly Local Curmudgeon
  • member icon

Reputation: 574
  • View blog
  • Posts: 2,989
  • Joined: 19-May 09

Re: Waterfall method and the DRY principle

Posted 02 May 2013 - 09:28 PM

In my opinion, any software design methodology has to have some sort of feedback and change mechanism. Churn is what happens when the mechanism isn't formalized, which is what happens when the methodology denies that the mechanism is necessary. Despite the small army of project managers that cheerfully believe that they are imposing their will on the creative process, hammering it into a four-phase linear phenomenon (create it, design it, build it, fix it--with oh-so-many variations in nomenclature), the creative process just doesn't work that way. It's a process of evolution, which does indeed have those four phases, but each of the phases implies a process of potential change in each of the others. A software creation succeeds when all four of these phases reach a simultaneous state of balance. This state of balance is one way of describing a release point; at the point of release there are still requirements in the pipeline for future versions.

I recall an interesting situation in a shop that used a "modified" waterfall method. The PM called it modified because it was possible to enter a high-ceremony "change request" into the requirements at any point in the process, with a Change Review Board approval, grilling of the why didn't you find this before type, and so on. Now, the business analysts reported to said PM in the management structure. The PM lavished praise on the BA's for getting their BRD's (Business REquirement Documents) in on time, holding up his people as a model for the rest of the department to follow. Of course, he was much too busy PMming to actually read said BRD's. Now the result was that the BRD's were pretty much unusable, full of contradictions, incomplete specs, inaccurate specs, the list goes on. The advice from our boss (very intimidated by PM) was to get the general idea from the docs--agile group that we were-- and apply them in the tech doc. The actual way that things were done was that we would get the BRD, call the developers in China once a day, and invent about 50% of the features ourselves and the rest from the BRD.

I still recall my first assignment: take a BRD and create a Technical Design Document from it. I remember going into a meeting and saying that I wasn't able to create it from the BRD as it stood. Well, why was the chinese lady able to do it with the other ones? Well, she sits around for hours each night and talks Chinese with the Chinese people, and documents whatever they come up with. Which only marginally has anything to do with the BRD, which anyone reading both of them can easily determine. Well, Bob, this is a critical path document. Are you saying that you want to hold up the entire project because you aren't able to get this done on time? Well, says I, since the Chinese people have been developing for a month before I got here, how is it that the TDD is a critical path document when nobody seems to need it to get the job done? Dead silence, VP's looking at VP's, and finally one saying well everyone said we had to hit this deadline and we had to do something, so we're just working without the TDD.

Needless to say, I was the first head to roll in the next round of layoffs, and when they didn't get the site out in time, costing them a year's worth of business, my ex-boss and a bunch of other people got laid off, but the PM got promoted to a VP position. Hey, like DUDE, if he hadn't been there, it would have been even worse...

This post has been edited by BobRodes: 03 May 2013 - 06:05 AM

Was This Post Helpful? 0
  • +
  • -

#10 BobRodes  Icon User is offline

  • Your Friendly Local Curmudgeon
  • member icon

Reputation: 574
  • View blog
  • Posts: 2,989
  • Joined: 19-May 09

Re: Waterfall method and the DRY principle

Posted 03 May 2013 - 05:36 AM

Oops double post.

This post has been edited by BobRodes: 03 May 2013 - 05:37 AM

Was This Post Helpful? 0
  • +
  • -

#11 BobRodes  Icon User is offline

  • Your Friendly Local Curmudgeon
  • member icon

Reputation: 574
  • View blog
  • Posts: 2,989
  • Joined: 19-May 09

Re: Waterfall method and the DRY principle

Posted 03 May 2013 - 05:48 AM

View Postjon.kiparsky, on 01 May 2013 - 08:43 AM, said:

View PostSkydiver, on 01 May 2013 - 08:10 AM, said:

So if an upstream requirement is changed, then the changer gets to fix all the downstream artifacts he broke.


I so want to introduce this in my office.


Of course you're both joking and know that's unrealistic, but you also sound as if you feel a bit hard done by. You're the ones left holding the bag, after all. But the issue isn't the changes per se, it's the lack of understanding of the ramifications of the changes by the people making them. You can just fit that in and still make the deadline, can't you, there's a good fellow. Not as if we're asking for the moon, now is it? Nevertheless, an engineer who expects an immutable set of requirements to work against is simply out of touch with reality. Just as much so, a stakeholder who expects that changes can be implemented without additional time and effort is equally out of touch with reality. This is a reality not attributable to human failing, however, but rather to the fundamental nature of human creation. Creation is messy, time-consuming, and requires a lot of effort. You can't avoid that by "doing it right the first time." Ask any parent.

The business decision about making a change boils down to this: if the cost of not doing it outweighs the cost of doing it, then do it, otherwise don't do it. Quite simple really, once the egos are out of the equation. Part of the job of an engineering manager is to make sure that the change management function (whether such function is formally defined or not) understands the cost of change, so that the stakeholders can make an informed business decision about it. That can mean anything from standing up to a boss who put a piece of software together and now has hired a few people to expand his business and has few if any people management skills to talking to anyone who will listen and keeping a record for the meeting to find out why the team is running behind.

This post has been edited by BobRodes: 03 May 2013 - 06:01 AM

Was This Post Helpful? 1
  • +
  • -

#12 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3572
  • View blog
  • Posts: 11,106
  • Joined: 05-May 12

Re: Waterfall method and the DRY principle

Posted 03 May 2013 - 06:09 AM

Yes, but let's say a change is considered worth the cost, wouldn't it be cool if the DRY principle could be applied so that if the requirement 3.8 changes from "The system will popup an error box." to "The system will beep and and reject the input", that the text only be changed in the BRD and all the other traceability matrices in the design document, test document, etc. get updated automatically?

Of course there is the corresponding risk for the convenience. It's possible that no human eyes will see the downstream impact of such a change. On one hand, it's no worse than somebody just mechanically copying and pasting into the downstream documents, but on the other hand, if you force people to edit the downstream files you take away the plausible deniability of "I didn't know the requirement changed." Perhaps that is the point of all the duplication -- force somebody to look again.

This post has been edited by Skydiver: 03 May 2013 - 06:36 AM

Was This Post Helpful? 0
  • +
  • -

#13 BobRodes  Icon User is offline

  • Your Friendly Local Curmudgeon
  • member icon

Reputation: 574
  • View blog
  • Posts: 2,989
  • Joined: 19-May 09

Re: Waterfall method and the DRY principle

Posted 03 May 2013 - 05:13 PM

Automated linking of the different components of a requirement's lifecycle becomes more and more useful as your project's complexity increases. If you want to spend some cash, you can use Rational Requirements Composer to do that very thing. In fact, you can get a bona fide IBM RRC cert and run around charging big companies hundreds of dollars an hour making sure that that plausible deniability you mention isn't so plausible. :)

This post has been edited by BobRodes: 03 May 2013 - 05:17 PM

Was This Post Helpful? 0
  • +
  • -

#14 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3572
  • View blog
  • Posts: 11,106
  • Joined: 05-May 12

Re: Waterfall method and the DRY principle

Posted 03 May 2013 - 09:23 PM

A few years ago, I spent a couple of years working with somebody who was on the team writing the code behind Rational Rose. Apparently they didn't always use their own product internally. :)
Was This Post Helpful? 0
  • +
  • -

#15 BobRodes  Icon User is offline

  • Your Friendly Local Curmudgeon
  • member icon

Reputation: 574
  • View blog
  • Posts: 2,989
  • Joined: 19-May 09

Re: Waterfall method and the DRY principle

Posted 04 May 2013 - 08:59 PM

View PostSkydiver, on 03 May 2013 - 11:23 PM, said:

A few years ago, I spent a couple of years working with somebody who was on the team writing the code behind Rational Rose. Apparently they didn't always use their own product internally. :)

No, I shouldn't think that they would. :) The thing that I like so much about Cockburn's article is the whole concept of the discrepancy between what people say about their processes and their actual processes. It's human nature, after all.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1