Agile Methodologies in a Nutshell

Agile methodologies are a relatively recent reaction to more traditional approaches to application development, particularly those approaches that emphasize large-scale planning, documentation, and control in the development process. The latter generally attempt to determine all of the requirements for a project up front, think through all of the various potential outcomes and contingencies, formulate and document detailed design plans, lock in end-user "buy-in," and then turn the plans over to development teams.

In contrast, agile methodologies abandon much of the detailed up-front planning and documentation in favor of a relatively quick assessment of what needs to be accomplished. They proceed from a number of key assumptions, which include the following:

• It is the nature of the game for requirements to change.
• Systems are best built in short, manageable steps or "iterations," each of which builds on the functionality of what has been done before.
• The most appropriate processes for a project can only be discovered--and may need to be changed--as the project evolves.
• Collaboration is what is most important, not a detailed master plan.

One of the key reasons for taking an "agile" approach is the recognition that, in today's fast-paced, constantly changing business climate, it is virtually impossible to nail down truly realistic long-term requirements for most projects. Even if you can come up with something close, the odds are high that the requirements will have changed by the time you're only a month or two into development--if not before.

Agile supporters argue that the only sensible response to such a situation is to employ a process specifically designed to adapt to changing requirements rather than prevent or limit changes once a plan is in place, as many traditional approaches attempt to do. They also assert that it's essentially a waste of time to try to develop a comprehensive, detailed, documented plan up front, since the longer-term components of such a plan (the requirements) are by definition unpredictable.

The most successful projects (perhaps the only successful projects) are those that focus on short-term, concrete, immediately doable pieces of whatever the "final" system might ultimately turn out to be, building on each short-term success in subsequent "iterations" that add functionality as the long-term system--and its requirements--evolve.

Not surprisingly, agile methodologies give some people the jitters, in that they can seem to be too unstructured and uncontrolled. In good hands, however, they can be surprisingly effective, especially on less-complicated projects (or smaller chunks of larger projects) and in environments, like a lot of businesses today, where requirements are unpredictable more than a few months out. They focus on what can reasonably be done now to meet immediately known requirements.

A Practical Approach

The intent of agile development methodologies is not to abandon process altogether; it is merely to redefine processes and break them down into more manageable chunks, in order to increase the odds of success with one part of a project before a new set of requirements kicks in. Their basic principles and assumptions make excellent sense for other kinds of projects, too. Here are some of the most relevant ones:

1. Requirements are unpredictable and will change over the course of a project. The most successful projects adopt processes specifically designed to accommodate changes in requirements rather than try to limit or prevent those changes.

Traditional approaches often try to plan out an entire project before ever getting started. That's fine until things change, which may begin to happen long before the project plan is completed. Traditional approaches also tend to resist change, often by stressing the need to get end-user buy-in up front--and put it in writing--so there will be no question later what everyone signed off on.

In a fast-paced business environment, however, change is the norm. No matter how hard you try to lock a plan in place with fixed requirements and hold end-users to their agreements, the requirements will still change, and those changes will impact the potential success of the project. You can waste energy on trying to limit the changes, as many project managers using traditional approaches do, but you'll eventually lose the battle, and the plan will ultimately fall apart. As Martin Fowler put it, "one of the big dangers [he might also have said time wasters] is to pretend that you can follow a predictable process when you can't." (See "The New Methodology.")

Agile methodologies assume that change is a given and look specifically for ways to accommodate it.

2. Because requirements inevitably change, enterprises need to break out of the old "plan, design, build" mentality of traditional approaches. They also need to find a way to insure constant and immediate feedback on the success or failure of what they're doing.

Traditional approaches to project management typically separate design from implementation. It is not unusual for companies to invest heavily in exhaustive planning, design, and scheduling before project "blueprints" are eventually handed over to various teams to implement. This reflects the origins of many traditional approaches in engineering and construction, where the working principle has always been to design first, then build. In theory, this also lets managers employ people with what they consider to be "lesser" skills on the implementation end of things, moving them around as necessary to accommodate different work demands.

In fast-paced business environments, however, where requirements are likely to change even before implementers have a chance to get to work, this doesn't cut it. Agile development addresses the problem by replacing extensive planning with a quick assessments of needs, articulating a general vision of what the ultimate system will look like, getting going on a small, manageable piece of the project, and then doing the rest of the project design on the fly. As the first piece of the project is completed and validated, additional small pieces of the project are initiated that build on the previous piece and take into account any changes in requirements that may have happened in the meantime.

An important part of this process is feedback. Only when you have constant and immediate feedback on the success or failure of what you're doing--as it relates to requirements (either those you started out with or the new ones that have popped up along the way)--can you have any chance of exercising some control over the unpredictabilities of business. Approaching projects iteratively in small, manageable chunks makes this more likely.

3. Rigorous testing and debugging are critical at every iteration.

In most kinds of agile development, each project iteration yields a fully functional, fully tested, working version of the final system, even if it only represents a restricted subset of features for what that "final" system is envisioned to be. Achieving that kind of functionality entails at least the same degree of rigorous testing and debugging that should (in theory) be applied to a fully blown system under a more traditional approach.

Such testing and debugging is all the more important in agile development, precisely because the larger system is being built up progressively by means of iterative enhancements to smaller chunks. If problems are not eliminated as they appear, they can be carried forward and propagated in the larger system.

Luckily, testing and debugging are easier under agile development because, in any given iteration, (a) you are working on a relatively small piece, and (b) if you've done the job properly, you have already eliminated all of the problems from all previous iterations. It is far easier to add new functionality to a bug-free base, iteration by iteration, than it is to try to debug a fully blown system all at once.

Although the focus here is overtly on application development, the idea of "rigorous testing and debugging" is appropriate for virtually all kinds of project management. Each time you validate that you've met a project requirement, worked out all the bugs, and confirmed that whatever you're implementing works smoothly, you have a much better chance that the next project iteration--and the next and the next--will also be successful.

4. Project teams need to resist the temptation to do more than what is required.

As we have seen, one of the main purposes of focusing iteratively on small, manageable pieces of a project is to insure that each piece is completed successfully before requirements change. Although it is always tempting to do more than the stated requirement, succumbing to that temptation only increases the time to completion, increases project complexity, and increases the difficulty of testing and debugging thoroughly--all risks that diminish rather than heighten the possible reward.

Because of the iterative nature of agile development, additional functionality can always be incorporated later on the base that has already been established--assuming that there is indeed an actual requirement for that functionality. Since the goal of agile development is to control the unpredictability of requirements, why turn around and introduce unpredictability yourself? Don't do more than necessary for a particular iteration of a project.

5. Requirements aren't the only things that change. Processes do, too. The most successful projects don't impose a process on a project but look for and adopt the processes that best fit the requirements at hand.

Agile development takes as a given that there is not necessarily one process that is suitable for all aspects of a project. Different iterations of a project may require different processes. Different sub-groups working on a project may also find that different processes work better for them than for other groups working on other iterations at other times.

As a result, under an agile development approach, determining which process to use is more a matter of finding the best process for the particular challenge than one of trying to apply a process that is part of some more formal methodology. Let the project requirements guide you to the best process.

6. Constant and immediate feedback is just as important for processes as it is for requirements.

As is the case with requirements, constant and immediate feedback is critical to insure that the processes being followed are in fact the most appropriate ones, and as is the case with requirements, focusing on smaller project chunks makes this easier to achieve. It is much simpler to match a process to a relatively discrete subset of project tasks than to some grand project as a whole.

Agile development generally makes it a regular exercise, after each project iteration, to think consciously about what went well, what the experience taught the development team, what they're still not sure of, and what they think they could do better on the next iteration--and how. That's a good tack to take on any project.

7. The people involved in a project--and how they are managed--are probably more critical to project success than any other factor.

There are a couple of twists to this principle.

First, traditional approaches to project management often work against basic human nature. Take, for example, the tendency to seek end-user or "stakeholder" buy-in, ideally getting that buy-in in writing and then sticking to it. How often have you heard someone say "Yes, but this is what we agreed to"?

When requirements change, as they inevitably do, earlier buy-ins don't really mean much, especially from an emotional point of view. Regardless of what stakeholders may have signed off on, they will still feel slighted (or disappointed or angry) in the end, when the project doesn't deliver on what they need. It's human nature.

Second, traditional approaches often treat the people who work on projects as replaceable components, particularly in projects that have been planned and designed by one group and then handed off to another group to implement. As noted earlier, the motivation is usually to make implementation more "efficient," moving resources around to handle different parts of the plan.

This gives rise to something of a self-fulfilling prophecy, however. If you view people just as implementers of some grand design, or, worse, as replaceable parts in the implementation process, then that's probably just how they'll act, and you'll lose a considerable degree of the creativity, capability, and commitment that they might otherwise bring to the task.

Agile development methodologies attempt to tailor processes to human nature. Rather than seek stakeholder buy-in on a locked-in master plan, they assume that the plan will change and involve the stakeholders on an ongoing basis in decisions about the evolution of the project. Rather than have them buy in to some distant end point whose requirements will likely have changed by the time the project gets there, they ask stakeholders only to buy in to the outcome of the current iteration. Then, subsequently, they insure that the stakeholders are a part of the evaluation process that helps determine what the requirements for the next iteration of the project will be.

Agile development methods also seek to involve all of the people in a project in all levels of its planning and design. They place a great deal of trust in everyone on a development team and recognize that what a traditional approach might view as "only" implementers may actually have better insights into processes--and what is doable in any given iteration--than independent project planners and designers.

In short, with both stakeholders and members of project teams, agile development methodologies recognize that the best way to insure project success, get the best work from everybody, and insure satisfaction with the result is true collaboration and sharing of responsibility, keeping everyone fully involved and informed every step of the way. That's an important consideration for any kind of project.

8. The most successful projects are those with enabling managers and leaders.

Since collaboration and communication are so important to agile development methodologies, they typically put a premium on leaders and managers with strong enabling and facilitating skills. These are people who excel at removing obstacles, insuring that sufficient resources are available, fine-tuning processes, and helping team members work well together--not whose forte is necessarily controlling people, processes, plans, and schedules.

Strong business leadership is also important, since the goal of every iteration in an agile development project is to produce a fully functional version of the final system that delivers on some immediate business requirements. The best project managers here are the ones who can help everyone stay focused on the key business outcomes of a particular iteration and make process decisions based on those outcomes--at the same time that they facilitate ongoing support of the collaboration and communication that is so essential to the process.
In Conclusion
Many of the principles underlying agile application development methodologies provide significant insight into how to manage all kinds of projects more successfully. They are especially useful for what they have to say about coping with changing requirements in a fast-paced business environment, the nemesis of many a well-intentioned IT project.

Agile development methodologies are not necessarily intended to supplant traditional methodologies. In fact, they are frequently used together with traditional methodologies in many enterprises. They are most effective with somewhat smaller or less complicated projects, but many enterprises find that they can quite easily break larger projects down into smaller, more manageable chunks and apply the agile development principles to those smaller chunks. Regardless of how they are used, their principles are worth keeping in mind for projects of all kinds.

John Knapp is currently Manager of Program Management for VUE Testing Services, a division of NCS Pearson, as well as an independent analyst and author. Formerly, he was a college professor, a Senior Industry Analyst with Andrews Consulting Group, an association executive, and Manager of Program Management for Sylvan Prometric, not necessarily in that order. He can be reached by email at This email address is being protected from spambots. You need JavaScript enabled to view it..