Can Agile software development be used in projects defined by a contract?

Question

I recently had a conversation with a fellow developer about Agile Software Development. While I understand the principle, it seems that continuously changing requirements creates the potential for the project to go on forever. But, at least where I work, projects need to reach completion because it's a contract.

That is, the first iteration could take months, because for some projects the customer cannot use an incomplete application. For some projects, I think you need to define finished first, then you can break it into iterations and refine your definition after each iteration. But you must always have this definition.

If Agile Software Development embraces changing requirements how do you know where it ends? How can you budget for a project when the end result is always changing?

Is Agile Software Development more about an agile process, than an agile product?

it ends when you need to actually deliver something that works, rather than tinker around. At that point you have to start imposing structure, planning, fix requirements and deadlines, and start working rather than fooling around. — jwenting, Jul 14 '11 at 08:39
Each agile iteration results in a working, deliverable product which the client can use and learn more from. This goes on until they are satisfied, which often happens earlier than originally planned. This guarantees that the product is always working and takes into account the fact that software is never finished but rather evolves forever or dies. Just pick a point in time when you think the product is good enough and stop there (for now). — Martin Wickman, Jul 14 '11 at 12:35
@Martin Wickman: I understand this, but "a deliverable product which the client can use" is the problem here. If this is the case, the first iteration could take months, because for some projects the customer cannot use an incomplete application. For some projects, I think you need to define finished first, then you can break it into iterations and refine your definition after each iteration. But you MUST ALWAYS have this definition. — Verax, Jul 15 '11 at 02:31
@Verax: The agile manifesto was created to manage changes. If your project has has no changes, then agile is not for you. End of story. — Martin Wickman, Jul 15 '11 at 07:48
@Martin Wickman: I don't believe you read my comment very well. It says "...refine your definition after each iteration.", so obviously I am aware that all projects have changes and I have identified a way to manage it. You could have helped by better defining "a deliverable product which the client can use". But instead you chose sarcasm. How unfortunate. — Verax, Jul 15 '11 at 11:21
@Verax: You should clarify your question and add more context to it. Your comments shows there is more to the question. This is also obvious from the vote counts on the answer and that the accepted answer is unrelated to the actual question text (and even says "from the OPs comments..."). — Martin Wickman, Jul 15 '11 at 12:00
Hi Verax, I've incorporated the feedback you've provided here and the interpretation by the accepted answer back into your question. If there's anything more you'd like to add, please edit your question. — , Jul 15 '11 at 19:39

score 20 · Answer 1 · answered Jul 14 '11 at 02:17

20

If you're focusing on doing (what you believe is) the most important stuff first, then the project will finish when:

You've spent the money you budgeted.
You've elapsed the time you budgeted.
You no longer want to add or change anything.
The next batch of your highest priority changes aren't worth what they will cost.

answered Jul 14 '11 at 02:17

Dale Emery

330

5
No more money - Customer spent all their money on an incomplete useless product. 2. No more time - Customer still has an incomplete useless product. 3. Nothing to add - Yeah right! 4. Not worth it - Customer just gave up on the project. --- What am I missing? This doesn't make sense to me.

Verax

Jul 14 '11 at 05:08

7

For 1 and 2: If you do the most important stuff first, then when you run out of money, you'll have the most important stuff you can get for the money. Similar for time. I admit 3 is rare. For 4: Stopping doesn't necessarily mean the customer just gave up. It might mean that they have the most important stuff they wanted from this, and now would rather do other things with their money. How do you decide when to end other projects? Whatever criteria you use now are still available on agile projects. – Dale Emery Jul 14 '11 at 05:48

1

Dale, thank you for your thoughts. I think this only works if the customer is paying for each iteration individually and values each iteration as a product itself. I don't see how this could work well for fixed-cost products or products that require all or nothing. – Verax Jul 14 '11 at 07:04

5

@Verax: There's no such thing as a product that requires "all or nothing". There are always features that are merely "nice to have" and bugs that are cosmetic rather than functional. A fixed-cost project is a success if those are all that's left when money runs out. Agile methods try to maximize the likelihood of that. – Michael Borgwardt Jul 14 '11 at 07:42

@Michael: I challenge your statement that there is no such thing as all or nothing. I develop software for machine control. On my current project, if I delivered even one missing feature, the machine would not work completely and my customer could not sell it. That is what I mean by all or nothing. I could see how some projects could be partially completed and still be of value, but I think "all or nothing" does exist. Unless I missing something. – Verax Jul 14 '11 at 08:11

@Verax: Well, I know nothing about your specific requirements and my experience is in a completely different field (as is most programmers', which may explain why methods conceived to deal with mainstream application development seem unapplicable to you), I find it very hard to believe that there are no features without which the machine would still be usable and sellable. It implies that the proposed feature set is perfect and cannot possibly improved. – Michael Borgwardt Jul 14 '11 at 08:29

1

Of course there is a cost to changing the requirements. If you build something in one iteration, then change the requirements for that stuff in the next, there's a cost for that. Agile reduces the cost. It doesn't eliminate it. If you have a budget, you keep the budget in mind when deciding whether to add a new feature or change an existing one, knowing that you're always trading one off against the other. You learn to prioritize and re-prioritize, and you learn the consequences. – Dale Emery Jul 14 '11 at 08:52

This answer almost perfectly encapsulates the agile spirit. The fact that people used to waterfall have trouble accepting the underlying truth as such, is troubling in itself, but also points out why so many waterfall projects fail. – wolfgangsz Jul 14 '11 at 11:09

1

@wolfgangsz: So what evidence do you have that comparably more waterfall projects fail compared to agile projects? Just because you deliver something, even though it may suck, does not make an agile project a success. Also, the problem that many waterfall proponents have is that they don't realize that the type of products they work on have a history of being waterfallish out of necessity (ie. It has to have almost all the features). Whereas, the type of products utilizing agile have a history of little to no process at all (ie. business apps). The needs are very different. – Dunk Jul 14 '11 at 22:08

score 14 · Answer 2 · answered Jul 14 '11 at 05:03

You stop when the business decides they do not wish to do any more iterations. You would hope that this is after a significant amount of value has been delivered but before you get too far into the realm of diminishing returns.

It should always be driven by "the business" whatever that means in your circumstance. It could be the senior management of a software development shop or the actual business sponsors in an in-house development environment. They will decide when the cost of the next iteration outweighs the benefit of the features that will be deliverable in the next iteration.

Morons · Accepted Answer · 2011-07-15T01:00:06.867

7

From the OP's Comments is seems that he like me works for a Consulting shop, where we provide development services for our clients... I think because on this mind-frame that is causing his/her confusion... So I am going to state a well know but never stated fact.

Agile is incompatible with software development that is defined by a contract.

Contracts need to be Hard, You pay X we do Y. You want X+M you pay us Y+(M*N)
Contracts MUST be satisfiable , (IE not open ended ) otherwise they are not legal contacts. (When a contact is involved you must go though a strict change control process.)

Many Consulting shops claim to Agile, they are lying. They just say that because Agile has attained Buzz word status.

Agile works best for internal development where programmers are full time, and there is little talk of budgets. Just Time Frames and Features.

edited Jul 15 '11 at 01:00

answered Jul 14 '11 at 19:16

Morons

14,704

As I learn more about this, I am coming to the same conclusion. Your last sentence seems to be absolutely correct. I used to work for the government and my customer was the agency I worked for, and programs had to be maintained for years and years as long as there were employees using them. I can see agile working there. Now I develop embedded systems. The project ends when the machine works (all or nothing). If the machine partially works, it can't be sold - project failed. – Verax Jul 15 '11 at 00:10
2

Actually a consulting shop that I was working for a couple of years ago actually had a paper written by an agile adherent describing how agile could be fitted into a fixed price service model. – mcottle Jul 15 '11 at 03:26
2

I have to disagree with this answer. The reason is that if you have a contract that is not open-ended, it means that you don't want to manage scope-creep (which almost always happens). The contracts I'm used to see starts with: You pay X, we do Y. They then have a clause that states that any scope change comes with an extra price. As long as you are very early about notifyng scope creep (which requires extra resources and time) then the earlier customers can react to the changes (getting approvals and budget from upper management etc.). Then the management process itself becomes agile. – Spoike Jul 15 '11 at 05:18
This is not incompatible if the contract is for service (writing code) as opposed to product (finished software). Agile allows to estimate what will be done under what budget, an if requirement changes, budget must change too. You want another feature? You must contract another 500 man-hours. Feature creep is also price creep, completely satisfactory to developers, and if the customer is willing to pay, who are we to question that? – SF. Jul 15 '11 at 06:36
2

There are agile contracts, so obviously this answer is wrong by definition. – Martin Wickman Jul 15 '11 at 07:50
NO! That is BS Agile. – Morons Jul 15 '11 at 14:51

score 5 · Answer 4 · answered Jul 14 '11 at 06:41

Never, and that's the beauty of it.

The project is never finished. You reached another release, another milestone, but as long as the money is flowing, there's always one more feature to add, one more piece to make better, one more bug to fix. The project will die when it's no longer needed, but it will never get finished. As opposed to Waterfall model with requirement->project->product->end, this is a loop that can spin forever - as long as you're getting paid.

Not a frequently mentioned business feature of this technology, is it?

Waterfall projects aren't ever completely finished either, just more likely to be delivered take-it-or-leave-it with important features missing, making a new, expensive project necessary. — Michael Borgwardt, Jul 14 '11 at 07:44

score 4 · Answer 5 · answered Jul 14 '11 at 18:33

4

There is a misconception here: Agile does not encourage the project's requirements to change. It instead allows for change without wasting work, or sacrificing important areas of development.

There are four fundamental constraints to any engineering project; scope, cost, time and quality. Waterfall assumes that these will be static. That is an incorrect assumption; one or more of these ALWAYS change. Scope creep, slashed budgets, and other "unknown unknowns" ALWAYS interfere with a project, changing the constraints. Waterfall does not anticipate this, so when it happens, the project changes in undesirable ways; important features that haven't yet been added go away, or are quickly done, or the release has to be pushed back, or cost balloons as the PM throws money at new developers to come in and help get it all done right.

Agile, by contrast, allows constraints to change, and actually expects it. It does this by doing work in small, useable chunks, according to the priorities of the owner, and thus the chunks are ideally immediately useful to the project owner. Thus reduces exposure to the unknown by not making big plans out in a timeframe where the unknowns are large. If the timeline changes, teams can be added, or less important features "de-scoped", and the system the team has already built is unaffected.

It also provides for better estimates of the time and cost required to produce the given scope at the required quality. People are notoriously bad at estimating big jobs; it takes a LOT of experience, and a LOT more upfront calculation, to do it properly. By contrast, people are generally good judges of what they can get done in a day, or a week or two. That quickly produces a steady-state where you can extrapolate the time and cost of the work left to be done based on your historic pace, with a fair amount of accuracy.

As for defining endpoints, you're right; an Agile project CAN go on forever. However, so can the traditional SLDC; the client often comes back with more money and a wishlist of upgrades. The difference is that there isn't a clear line between "analysis", "design", "development" and "maintenance" when looking at the project as a whole; it all happens brick-by-brick, sprint by sprint. If at any point the owner wants to call the project "done", they can, and they will have the sum total of "bricks" they've paid for in a solid "wall"; it may not be as high or extend as far as they originally planned, but it's firmly in place, does the job, and can be added to at a later date with a minimum of tearing down.

answered Jul 14 '11 at 18:33

KeithS

22,104

Sorry but "It instead allows for change without wasting work," is a humungous fallacy that is used to convince management of how great it is. Doesn't refactoring and/or redesigning the system to accomodate unexpected changes count as wasting work? In the waterfall camp it does, apparently not in the agile camp. Also, if the customer only wanted a job that takes 2 weeks to complete then it doesn't matter what methodology is used, people can give good estimates. What the customer really wants is how long before I have the full product, where agile is no better than other methods of estimating. – Dunk Jul 14 '11 at 22:15
1

Also, you make it sound like a good thing that the owner can call done at any point and what you have finished is what he gets. IME, generally the customer wants to know that his X dollars are going to provide a certain set of features before he plunks down the cash. I don't see it as a benefit that the customer spent a pile of cash and only got half the features they expected. I consider that to be a failure on the developing companies part even though they may have delivered something they call working.... – Dunk Jul 14 '11 at 22:20
2

What if a customer contracted for a house but the money ran out before the roof was put on? The agile camp would still call that a success. Nobody else would; in particular the customer. – Dunk Jul 14 '11 at 22:23
@Dunk - wow, you really have an axe to grind for Agile. First, the whole point of Agile is that what you have already developed should not have to change. If you have to redesign the system from scratch something has gone terribly wrong in your design process. And, if the customer said they wanted it one way but now want it another, that's on them, and they'll pay to change it. That's only fair to both sides, and any contractor in the world would charge for the parts and labor if the customer came back and said they wanted a different baseboard after they were cut and installed. – KeithS Jul 14 '11 at 22:52
On your second point, it IS a good thing that the customer can call done and they get what they've paid for so far. That protects both sides; the customer can abort, or less severely can accept the functionality he has which if he prioritized correctly is what he needs most, and the developers get fair price for what they've done to date. Most other contracts would normally result in the customer keeping their money and the developers being out their time, and left with code nobody will use. No house contractor works that way, especially since the materials are on the client's lot. – KeithS Jul 14 '11 at 22:57
1

As for estimating, the team estimates what they can do in a sprint, and that is extrapolated to create a timeline for the whole project. Again, it helps protect both the developers and the client. You can put anything you want in a contract, including a "not to exceed" amount and date. It's negotiable. Agile still helps, by showing both sides very quickly whether the constraints are feasible. Two weeks in, if it doesn't look like it'll get done in time for the money, teams can be added, features descoped, or the schedule extended. – KeithS Jul 14 '11 at 23:02
@Dunk let us continue this discussion in chat – KeithS Jul 14 '11 at 23:02
@Dunk: "What if a customer contracted for a house but the money ran out before the roof was put on? The agile camp would still call that a success. Nobody else would; in particular the customer" This is exactly the problem I have with the Agile methodology. – Verax Jul 15 '11 at 00:22
The Agile camp would not call that a success, because they're out of work relatively early into a project. Besides, the analogy is ridiculous; if the customer didn't have enough money to even get the house to the point where it was closed in, they should never have signed the contract in the first place. There WILL be enough knowledge of the complexity of the project up front to say that the money the customer is offering is woefully inadequate. More likely, the money would run out two weeks early, and the client would have some unfinished rooms and no kitchen appliances. – KeithS Jul 15 '11 at 00:27
@KeithS: Thanks for response. I would like to better understand, but I find the analogy valid. There seems to be a point (80% ~ 100%) finished that would be acceptable. The unfinished rooms and kitchen appliances would fall in that unfinished 20%. But it seems anything less than 80% is a failure, and this is the trouble I'm having with agile. The project is a failure until 80% of the iterations are complete. But what if the project ran out of time, money, etc... at 70%? – Verax Jul 15 '11 at 00:42
@KeithS: Also, your first statement "Agile does not encourage the project's requirements to change" seems to be in conflict with the 2nd principle of the Agile Manifesto: "Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage." I'm beginning to think there are many interpretations of this methodology. – Verax Jul 15 '11 at 00:44
@KeithS let us continue this discussion in chat – Verax Jul 15 '11 at 00:45
1

What if it had done the same in a waterfall SLDC? Either development would stop and the client might get a codebase with serious functionality holes, or anticipating a money/time shortage, the remaining schedule would be crammed into the remaining time. That ALWAYS sacrifices quality, and at the end of such a project the development team is fried. Also, a lot of cost overruns happen because a traditional waterfall doesn't produce a product until development is complete; that limits the ability of the customer to say "that's not what I wanted". – KeithS Jul 15 '11 at 00:48
@KeithS: Sorry I can't use chat because my browser doesn't support StackExchange Global authorization or something like that. Anyways, I don't have an axe to grind against agile. It has its uses for specific types of projects. However, I have an axe to grind against proponents of agile who act as though everyone should be doing it because it is the silver bullet of software development. They fail to realize that different types of projects and industries require different methods. Agile is terrific for simple systems, ramp up the complexity and it isn't so good anymore. – Dunk Jul 19 '11 at 17:42
@Keith: Also, in regards to not doing rework in agile. "You should get it right the first time or you are doing something wrong." Once again it depends on the project. Maybe in simple systems you can get it right the first time, but I seldom get the luxury of developing simple systems. It has been my experience that if you only design for a couple of user stories you will almost certainly need to refactor your design and code as new stories are added to the mix. Sometimes, only small changes are necessary but sometimes a complete redesign is required. It is not generally an efficient approach. – Dunk Jul 19 '11 at 17:47
@Keith: Most of your other points boil down to "Is it acceptable to the customer to receive a partially working product". You strongly believe that it is and for your customers that may be true. However, that would seldom be the case to my customers. In my business, if we don't deliver on time and within cost then my company takes the hit. We still deliver a quality product because we know that a happy customer will want enhancements and bring in more money to make up for the initial hit. While you may have delivered junk at your companies in those situations, on my projects we never did. – Dunk Jul 19 '11 at 17:55

score 1 · Answer 6 · answered Jul 14 '11 at 02:07

1

It ends once all the features are implemented and all the bugs are fixed.

If the deadline is fixed and the requirements are also fixed. Then this won't be a problem. But if the deadline is fixed, but the requirements are changing, then there is something that you should do in order to proceed the project successfully.

Fixed price part 1, what's so bad?

Fixed price part 2, Fix it with agile!

answered Jul 14 '11 at 02:07

CharithJ

111

It's hard to know when all bugs are fixed. – Martin Wickman Jul 14 '11 at 12:39
Maybe "when all the known bugs that are worth fixing are fixed"? – Dan Ray Jul 14 '11 at 13:08
@CharithJ, your links are broken. Are they still available somewhere? Thanks. – TwainJ Feb 20 '19 at 21:24

score 1 · Answer 7 · answered Jul 14 '11 at 02:35

The big assumption behind agile development is that requirements are always changing, no matter which methodology you use. Now, of course you could build a requirements document, build a plan to execute on it, and deliver at the end, and it may seem that your requirements didn't change. They may not have changed in your plan, but with market changes and your and your customer's better understanding of the product, requirements in terms of what the customer wants is going to change. Agile comes in and suggests a process that doesn't hide these these changes through a fixed schedule, but instead builds responding to inevitable changes into the process.

When you are done now shifts from finishing a fixed schedule to when you product is at a place where you can deliver enough business value where your customer can ship and market the software in its current state. Being done is tied much more to how much value you are delivering instead of how you adhered to a schedule.

Agile proponents make the very wrong assumption that in the waterfall world the developers disappear after a contract is signed and are never heard from again until a product pops out the door. The way it works in real life is that there are a fair number of checkpoints and many reviews that the customer can be as involved with as much as they like. If they don't like the direction or decisions they can request changes. By the time a product is delivered it should be what the customer wanted to the extent the customer chose to be involved. Agile does not improve upon this for many projects. — Dunk, Jul 19 '11 at 18:04

Can Agile software development be used in projects defined by a contract?

7 Answers7

Linked