This has been a very active website in 2012. Thank you all for your interest and participation. We are looking forward to some great discussions again this year.
Mark & Scott
Here’s an excerpt:
4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 36,000 views in 2012. If each view were a film, this blog would power 8 Film Festivals
One of the key aspects of the Disciplined Agile Delivery (DAD) process decision framework is that it promotes a full, end-to-end, solution delivery lifecycle. The figure below overviews a high-level view of the DAD lifecycle. It is a three-phase lifecycle, more on this in a minute, where you incrementally build a consumable solution over time. We start with this high-level view so that we can explore several important concepts before diving into details.
First, the DAD lifecycle explicitly calls out three phases:
- Inception. During this phase project initiation activities occur. Although “phase” tends to be a swear word within the agile community, the reality is that the vast majority of teams do some up front work at the beginning of a project. While some people will mistakenly refer to this effort as Sprint/Iteration 0 it is easy to observe that on average this effort takes longer than a single iteration (the 2009 Agile Project Initiation survey found the average agile team spends 3.9 weeks in Inception whereas the most common iteration/sprint length is two weeks). So in DAD’s Inception phase we do some very lightweight visioning activities to properly frame the project. It takes discipline to keep Inception short.
- Construction. During this phase a DAD team will produce a potentially consumable solution on an incremental basis. They may do so via a set of iterations (Sprints in Scrum parlance) or do so via a lean, continuous flow approach (different versions of the lifecycle are discussed later). The team applies a hybrid of practices from Scrum, XP, Agile Modeling, Agile Data, and other methods to deliver the solution.
- Transition. DAD recognizes that for sophisticated enterprise agile projects deploying the solution to the stakeholders is often not a trivial exercise. DAD teams, as well as the enterprise overall, will streamline their deployment processes so that over time this phase become shorters and ideally disappears from adopting continuous deployment strategies. It takes discipline to evolve Transition from a phase to an activity.
Granted, there is more to the agile SDLC than just those phases. First, there are pre-project aspects to portfolio management where potential projects or products are identified, priortized, and sufficiently funded to start an Inception phase effort. After Transition a solution is operated and supported in production (or the marketplace in the case of commercial products). Eventually, potentially after decades of use, a solution is retired (taken out of operation). If we were to look at things from the point of view of IT, there are also cross-product/project concerns at the enterprise level such as enterprise architecture, portfolio management, strategic reuse, and more (for more details, see Enterprise Unified Process). Having said this, these aspects of the overall lifecycle are not covered by DAD although do have an affect on how DAD teams work.
Earlier I said that the DAD framework does not prescribe a single lifecycle, unlike other agile methods such as Scrum. In the DAD book we focused on two versions of the lifecycle, the basic/agile and the advanced/lean versions, although since the first version came out we’ve also described a continuous delivery version. The point is that every team is in a unique situation, so for the DAD framework to be effective it must be flexible enough to support several versions of a lifecycle. Let’s explore three such versions of the lifecycle.
The Agile/Basic DAD Lifecycle: Extending Scrum
The figure below presents a more detailed view of what we call the Agile/Basic DAD lifecycle which extends Scrum’s construction lifecycle. In addition to this being a more detailed view of the lifecycle, there are several interesting aspects to this lifecycle:
- It’s iteration based. Like many agile methods, including both Scrum and XP, the solution is built incrementally in a time-boxed manner. These timeboxes are called iterations (what Scrum calls sprints).
- It uses non-Scrum terminology. Although the lifecycle is Scrum-based we chose to use non-branded terminology in DAD, in the case of this diagram the term iteration instead of sprint. The terminology doesn’t really matter, so if you’re more comfortable with Scrum terminology use that instead.
- It shows inputs from outside the delivery lifecycle. Although the overview diagram above showed only the delivery lifecycle, the detailed diagram below shows that something occurs before the project before Inception and that agile teams often get new requirements (in the form of change requests and defect reports) coming in from production. These inputs provide important context for the overall delivery lifecycle.
- There is a work item list, not a product backlog. DAD has a greater scope than Scrum, and when you take this greater scope into account you begin to realize you need a more robust change management approach than Scrum’s product backlog. Work items include requirements, defects, and other non-functionality oriented work such as training, vacations, and assisting other teams. All of this work needs to be prioritized somehow, not just implementation of requirements. For more on this, read Agile Best Practice: Prioritized Requirements.
- In includes explicit milestones. Along the bottom of the lifecycle diagram there is an indication of suggested light-weight milestones that DAD teams should strive to meet. Such milestones are an important aspect of agile governance.
We call this the basic/agile lifecycle because it’s likely where you’re going to start with DAD. Common scenarios for adopting this version of the lifecycle include situations where you’re extending Scrum to be sufficient for your needs or you’re transitioning from RUP to a disciplined agile approach.
The Advanced/Lean DAD Lifecycle
The figure below depicts what we call the Advanced/Lean DAD lifecycle. There are several interesting features to this lifecycle:
- It supports a continuous flow of delivery. In this lifecycle the solution is deployed as often, and whenever, it makes sense to do so. Work is pulled into the team when there is capacity to do it, not on the regular heartbeat of an iteration.
- Practices are on their own cadences. With iterations/sprints many practices (detailed planning, retrospectives, demos, detailed modeling, and so on) are effectively put on the same cadence, that of the iteration. With a lean approach the observation is that you should do something when it makes sense to do it, not when the calendar indicates that you’re scheduled to do it.
- It has a work item pool. All work items are not created equal. Although you may choose to prioritize some work in the “standard” manner, either a value-driven approach as Scrum suggests or a risk-value driven approach as both DAD and RUP suggests, but other work may fit this strategy. Some work, particularly that resulting from legislation, is date driven. Some work must be expedited, such as fixing a severity one production problem. So, a work item pool and not a prioritized stack makes a bit more sense when you recognize these realities.
We call this the advanced/lean lifecycle because it’s something we see teams evolve to over time. They often start with the basic lifecycle described earlier but as they learn, as they improve the way that they work, the lifecycle they follow becomes leaner.
The Continuous Delivery DAD Lifecycle
The figure below depicts a Continuous Delivery version of the DAD lifecycle. It is basically a leaner version of the previous lifecycle where the product is shipped into production or the marketplace on a very regular basis. This could be often as daily, although weekly or monthly is quite common too.
Why Support Several Lifecycles?
This is a good question. First, one lifecycle clearly does not fit all. Teams find themselves in a unique situation: team members are unique individuals with their own skills and preferences for working, let alone the scaling/tailoring factors such as team size, geographic distribution, domain complexity, organizational culture, and so on which vary by team. Because teams find themselves in a wide variety of situations shouldn’t a framework such as DAD support several lifecycles? Furthermore, just from the raging debates on various agile discussion forums, in agile user groups, at agile conferences, and even within organizations themselves it’s very easy to empirically observe that agile teams are in fact following different types of lifecycles.
Second, we were uncomfortable with the idea of prescribing a single lifecycle. We wanted to avoid prescription in the DAD framework to begin with, for all the rhetoric about the evils of prescription within the Scrum community it’s clear that Scrum is in fact quite prescriptive in practice. We’ve seen many teams get into trouble trying to follow agile methods such as Scrum to the letter of the law in an environment where “Scrum out of the box” really isn’t a good fit.
Third, we’re firm believers in process improvement. If you are in fact improving your process as you learn, is it not realistic that your lifecycle will evolve over time? Of course it will. We’ve seen teams start with something close to the basic/agile lifecycle that evolve it to the advanced/lean or continuous delivery lifecycles over time.
The Downside of Supporting Several Lifecycles
There is clearly a downside to explicitly supporting several lifecycles. By doing so we explicitly admit that DAD teams will be follow a unique tailoring of the process that best fits their situation, a concept that can be ant-thetical in organizations still clinging to the notion of repeatable processes (DAD promotes repeatable results over repeatable processes). It also makes it clear that enterprise professionals, such as enterprise architects or data management groups, need to be sufficiently flexible to support several delivery lifecycles. Instead of suboptimizing around such enterprise processes (i.e. forcing all project teams to conform to a single data management strategy) you instead want to build enterprise teams that are sufficiently skilled and flexible to support delivery teams in a manner which best suits the delivery teams. Finally, it’s clear that governance needs to be results based, not artifact based. The good news is that DAD builds effective governance right into the framework.
The Disciplined Agile Delivery (DAD) framework suggests a robust set of roles for agile solution delivery. These roles are overviewed in the following figure:
Primary roles are commonly found regardless of the level of scale. There are five primary roles:
- Stakeholder. A stakeholder is someone who is materially impacted by the outcome of the solution. In this regard, the stakeholder is clearly more than an end-user: A stakeholder could be a direct user, indirect user, manager of users, senior manager, operations staff member, the “gold owner” who funds the project, support (help desk) staff member, auditors, your program/portfolio manager, developers working on other systems that integrate or interact with the one under development, maintenance professionals potentially affected by the development and/or deployment of a software project. DAD teams will ideally work together with their stakeholders daily throughout the project.
- Team Member. The role of team member focuses on producing the actual solution for stakeholders. Team members will perform testing, analysis, architecture, design, programming, planning, estimation, and many more activities as appropriate throughout the project. Note that not every team member will have every single one of these skills, at least not yet, but they will have a subset of them and they will strive to gain more skills over time. Team members are sometimes described by core agile methods as “developers” or simply as programmers. However, in DAD we recognize that not every team member necessarily writes code. Team members will identify tasks, estimate tasks, “sign-up” for tasks, perform the tasks, and track their status towards completion.
- Team Lead. On agile projects the role of the traditional project manager changes substantially, and in fact the term project manager is now unfortunately frowned upon. The agile community has focused on project or team leadership over team management, observing that the best “managers” prioritize leadership over technical management issues such as planning and estimation. An important aspect of self-organizing teams is that the team lead facilitates or guides the team in performing technical management activities instead of taking on these responsibilities him or herself. The team lead is a servant-leader to the team, creating and maintaining the conditions that allow the team to be successful. The team lead is also an agile coach, helping to keep the team focused on delivering work items and fulfilling their iteration goals and commitments that they have made to the product owner. He or she acts as a true leader, facilitating communication, empowering them to self-optimize their processes, ensuring that the team has the resources that it needs, and removes any impediments to the team (issue resolution) in a timely manner. When teams are self organizing, effective leadership is crucial to your success.
- Product Owner. In a system with hundreds or thousands of requirements it is often difficult to get answers to questions regarding the requirements. The product owner is the one individual on the team who speaks as the “one voice of the customer”. He or she represents the needs and desires of the stakeholder community to the agile delivery team. As such, he or she clarifies any details regarding the solution and is also responsible for maintaining a prioritized list of work items that the team will implement to deliver the solution. While the product owner may not be able to answer all questions, it is their responsibility to track down the answer in a timely manner so that the team can stay focused on their tasks. Having a product owner working closely with the team to answer any question about work items as they are being implemented substantially reduces the need for requirements, testing, and design documentation. You will of course still have need for deliverable documentation such as operations manuals, support manuals, and user guides to name a few. Each DAD team, or subteam in the case of large programmes organized into a team of teams, has a single product owner. A secondary goal for a product owner is to represent the work of the agile team to the stakeholder community. This includes arranging demonstrations of the solution as it evolves and communicating project status to key stakeholders.
- Architecture Owner. Architecture is a key source of project risk and someone needs to be responsible for ensuring the team mitigates this risk. As a result the DAD framework explicitly includes Agile Modeling’s role of architecture owner. The architecture owner is the person who owns the architecture decisions for the team and who facilitates the creation and evolution of the overall solution design. The person in the role of team lead will often also be in the role of architecture owner on small teams. This isn’t always the case, particularly at scale, but it is very common for smaller agile teams. Although the architecture owner is typically the senior developer on the team – and sometimes may be known as the technical architect, software architect, or solution architect – it should be noted that this is not a hierarchical position into which other team members report. He or she is just like any other team member and is expected to sign-up and deliver work related to tasks like any other team member. Architecture owners should have a technical background and a solid understanding of the business domain.
Secondary roles are typically introduced, often on a temporary basis, to address scaling issues. There are five secondary roles:
- Specialist. Although most agile team members are generalizing specialists, sometimes, particularly at scale, specialists are required. For example, on large teams or in complex domains one or more agile business analysts may join the team to help you to explore the requirements for what you’re building. On very large teams a program manager may be required to coordinate the team leads on various subteams. You will also see specialists on DAD teams when generalizing specialists aren’t available – when your organization is new to agile it may be staffed primarily with specialists who haven’t yet made the transition to generalizing specialists.
- Domain Expert. The product owner represents a wide range of stakeholders, not just end users, so it isn’t reasonable to expect them to be experts in every nuance in your domain, something that is particularly true with complex domains. The product owner will sometimes bring in domain experts to work with the team, for example, a tax expert to explain the details of a requirement or the sponsoring executive to explain the vision for the project.
- Technical Expert. Sometimes the team needs the help of technical experts, such as a build master to set up their build scripts, an agile database administrator to help design and test their database, a user experience (UX) expert to help design a usable interface, or a security expert to provide advice around writing a secure system. Technical experts are brought in on an as-needed, temporary basis to help the team overcome a difficult problem and to transfer their skills to one or more developers on the team. Technical experts are often working on other teams that are responsible for enterprise-level technical concerns or are simply specialists on loan to your team from other delivery teams.
- Independent Tester.Although the majority of the testing is done by the people on the DAD team themselves, some DAD teams are supported by an independent test team working in parallel that validates their work throughout the lifecycle. This independent test team is typically needed for agility at scale situations within complex domains, using complex technology, or addressing regulatory compliance issues.
- Integrator.For large DAD teams which have been organized into a team of subteams, the subteams are typically responsible for one or more subsystems or features. The larger the overall team, generally the larger and more complicated the system being built. In these situations, the overall team may require one or more people in the role of integrator responsible for building the entire system from its various subsystems. On smaller teams or in simpler situations the Architecture Owner is typically responsible for ensuing integration, a responsibility that is picked up by the integrator(s) for more complex environments. Integrators often work closely with the independent test team, if there is one, to perform system integration testing regularly throughout the project. This integrator role is typically only needed at scale for complex technical solutions.
Why So Many Roles?
This is a common question that we get from people familiar with Scrum. Scrum has three roles – ScrumMaster, Product Owner, and Team Member - so why does DAD need ten? The primary issue is one of scope. Scrum mostly focuses on leadership and change management aspects during Construction and therefore has roles which reflect this. DAD on the other hand explicitly focuses on the entire delivery lifecycle and all aspects of solution delivery, including the technical aspects that Scrum leaves out. So, with a larger scope comes more roles. For example, because DAD encompasses agile architecture issues it includes an Architecture Owner role. Scrum doesn’t address architecture so doesn’t include such a role.
Some Important Thoughts About Roles
On a DAD project any given person will be in one or more roles, an individual can change their role(s) over time, and any given role will have zero or more people performing it at any given time. For example, Peter may be in the role of team member and architecture owner right now but step into the role of team lead next month when Carol, the existing team lead, goes on vacation.
Roles are not positions, nor are they meant to be. For example, Jane may be in the role of stakeholder for your project but have the position of Chief Financial Officer (CFO) within your organization. In fact, although there may be hundreds of stakeholders of your project none of them is likely to have a position of “stakeholder.”
Agile deemphasizes specialized roles and considers all team members equal – everyone pitches in to deliver a working solution regardless of their job description. An implication of this is that with the exception of stakeholder everyone is effectively in the role of team member.
Traditional roles, such as business analyst and project manager, do not appear in DAD. The goals which people in traditional roles try to achieve, for example in the case of a business analyst to understand and communicate the stakeholder needs/intent for the solution, are still addressed in DAD but in different ways by different roles. There isn’t a perfect one-to-one match between any given traditional role and a DAD role, but as you will find in the Disciplined Agile Delivery book there are reasonable transition strategies. The critical thing for traditionalists to understand is that because the underlying paradigm and strategy has changed, they too must change to reflect the DAD approach.
Material for this blog posting was modified from Chapter 4 of Disciplined Agile Delivery: A Practitioner’s Guide to Agile Software Delivery in the Enterprise (IBM Press, 2012)
I will be teaching a two-day Advanced DAD workshop in Zurich on April 8&9th. The workshop is hands-on and is meant for intermediate-level people. People new to agile can still attend of course but may find it a bit overwhelming. I’m always happy to stay late and help anyone struggling, and happier yet to do so over a beer. Anyway, click here for more information.
Governance establishes chains of responsibility, authority and communication in support of the overall enterprise’s goals and strategy. It also establishes measurements, policies, standards and control mechanisms to enable people to carry out their roles and responsibilities effectively. You do this by balancing risk versus return on investment (ROI), setting in place effective processes and practices, defining the direction and goals for the department, and defining the roles that people play with and within the department.
Governance and management are two different things. Governance looks at a team from the outside, treating it as a system that needs to have the appropriate structure and processes in place to provide a stream of value. Management, on the other hand, occurs inside the team and ensures that the structure and processes are implemented effectively. The Disciplined Agile Delivery (DAD) process framework characterizes governance as an element of enterprise awareness from the team’s point of view because governance looks at the team from the outside.
It is easier to avoid your traditional governance and tell management that “agile is different” than it is to work with your governors to adapt your governance to properly guide the delivery of your agile teams. As we described in the book every organization has a necessary degree of governance and there are ways to make it especially effective on agile initiatives. It takes discipline to work with your governors to help them understand how disciplined agile teams operate and then discipline to accept and conform to the resulting governance process.
Our experience is that the most effective way to govern agile teams is to focus on collaborative strategies that strive to enable and motivate team members implicitly. For example, the traditional approach to motivating a team to provide good ROI would be to force them to develop and commit to an “accurate” project budget, and then periodically review their spending to ensure they’re on track. An agile approach would be to ask the team to provide a ranged estimate of what they believe the cost will be so as to set expectations about future funding requirements. Then the team works in priority order as defined by their stakeholders, visibly providing real business value through the incremental delivery of a potentially consumable solution. Costs are tracked via the team’s burn rate (the fully burdened cost of the people on the team plus any capital outlays for equipment or facilities) and value is tracked by the stakeholders’ continuing satisfaction (hopefully) with what the team is delivering for that cost. In short, a traditional approach often measures financial progress against a budget whereas an agile approach seeks to maximize stakeholder value for their investment by always working on the most valuable functionality at the time.
The DAD framework includes several important agile governance strategies:
- Adopting a risk-value driven lifecycle
- Explicit, light-weight milestone reviews
- Agile enterprise teams that work closely with agile teams
- Regular coordination meetings (daily standups in Scrum)
- Iteration/sprint demos
- All-hands demos
- Follow enterprise guidelines (coding standards, UI standards, data conventions, …)
- Retrospectives, and better yet measured improvement
- Increased stakeholder visibility
- Development intelligences (BI for IT)
- Aligning agile team governance with other governance (operations, security, data, …) strategies
- Agile measurement/metrics programs
- Active risk mitigation
- Robust role definitions
Many of the strategies described above are “standard” agile governance strategies, and a few are unique to DAD. It requires discipline to adopt and then execute on effective governance strategies, particularly in organizations where you already have a strong traditional governance program in place.
RDA is a Gold certified Microsoft Partner and four time winner of Microsoft partner of the year located in the Eastern US. They have standardized their agile delivery model on DAD and have recently created a DAD process template for Team Foundation Server (TFS). John O’Hara has written an excellent article about it here…
This post is a follow-up to Comparing DAD to the Rational Unified Process (RUP) – Part 1. In that post I described in some detail why Disciplined Agile Delivery (DAD) is not “Agile RUP”. DAD is quite different in both approach and content. There are however some very good principles that the Unified Process (UP) incorporates that are not part of mainstream agile methods. This post describes what parts of the UP made it into the DAD process decision framework.
DAD suggests a full-lifecyle approach similar to RUP. DAD recognizes that despite some agile rhetoric projects do indeed go through specific phases. RUP explicitly has four phases for Inception, Elaboration, Construction, and Transition. For reasons that I described in the last post, DAD does not include an explicit Elaboration phase. However the milestone for Elaboration is still in DAD which I will describe shortly. As the DAD basic lifecycle diagram shows, DAD has three of the four RUP phases.
- The Inception phase. An important aspect of DAD is its explicit inclusion of an Inception phase where project initiation activities occur. As Scott Ambler says in one of his posts “Although phase tends to be a swear word within the agile community, the reality is that the vast majority of teams do some up front work at the beginning of a project. While some people will mistakenly refer to this effort as Sprint/Iteration 0 it is easy to observe that on average this effort takes longer than the general perception (the 2009 Agile Project Initiation survey found the average agile team spends 3.9 weeks in Inception)”. So in DAD’s Inception phase (usually one iteration) we do some very lightweight visioning activities to properly frame the project. The milestone for this phase is to obtain “Stakeholder consensus” on how to proceed. In the book we describe various strategies to get through the Inception phase as quickly as possible, what needs to be done, and how to get stakeholders consensus.
- The Construction phase. This phase can be viewed as a set of iterations (Sprints in Scrum parlance) to build increments of the solution. Within each iteration the team applies a hybrid of practices from Scrum, XP, Agile modeling, Agile data, and other methods to deliver the solution. DAD recommends a risk-value approach of prioritizing work in the early iterations which draws from the RUP principle of mitigating risk as early as possible in the project by proving the architecture with a working solution. We therefore balance delivering high-value work with delivering work related to mitigating these architectural risks. Ideally we deliver stories/features in the early iteration that deliver functionality related to both high business value and risk mitigation (hence DAD’s “risk-value” lifecycle). It is worthwhile to have a checkpoint at the end of the early iterations to verify that indeed our technical risks have been addressed. DAD has an explicit milestone for this called “Proven architecture”. This is similar to the RUP Elaboration milestone without risking the confusion that the Elaboration phase often caused for RUP implementations. All agile methods seek to deliver value into the hands of the stakeholders as quickly as possible. In many if not most large enterprises it is difficult to actually deliver new increments of the solution at the end of each iteration. DAD therefore recognizes this reality and assumes that in most cases there will be a number of iterations of Construction before the solution is actually deployed to the customer. As we make clear in the book, although this is the classic DAD pattern, you should strive to be able to release your solution on a much more frequent basis in the spirit of achieving the goal of “continuous delivery”. The milestone for the end of Construction is that we have “Sufficient functionality” to deploy to the stakeholders. This is the same milestone as the RUP’s Construction milestone. During the Construction phase it may make sense to periodically review the progress of the project against the vision agreed to in Inception and potentially adjust course. These optional milestones in DAD are referred to as “Project viability”.
- The Transition phase. DAD recognizes that for sophisticated enterprise agile projects often deploying the solution to the stakeholders is not a trivial exercise. To account for this reality DAD incorporates the RUP Transition phase which is usually one short iteration. As DAD teams, as well as the enterprise overall streamline their deployment processes this phase should become shorter and ideally disappear over time as continuous deployment becomes possible. RUP’s Transition milestone is achieved when the customer is satisfied and self-sufficient. DAD changes this to “Delighted stakeholders”. This is similar to lean’s delighted customers but we recognize that in an enterprise there are more stakeholders to delight than just customers, such as production support for instance. One aspect of RUP’s Transition phase is that it is not clear on when during the phase deployments actually take place. Clearly stakeholders aren’t delighted and satisfied the day the solution goes “live”. There is usually a period of stabilization, tuning, training etc. before the stakeholders are completely happy. So DAD has a mid-Transition milestone called “Production ready”. Some people formalize this as a “go/no go” decision.
So in summary, DAD frames an agile project within the context of an end-to-end risk-value lifecycle with specific milestones to ensure that the project is progressing appropriately. These checkpoints give specific opportunities to change course, adapt, and progress into the next phases of the project. While the lifecycle is similar to that of RUP, as described in Part 1 of this post it is important to realize that the actual work performed within the iterations is quite different and far more agile than a typical RUP project.
At Scott Ambler + Associates we are getting a lot of inquiries from companies seeking help to move from RUP to the more agile yet disciplined approach that DAD provides.