Home > DAD discussions, Transition iteration > Injecting Transition iterations into your Release plan

Injecting Transition iterations into your Release plan


For complex agile projects, deploying (or transitioning) the release to a “live” environment for the users is seldom a trivial exercise.  Most large enterprises have defined milestones, gates, and or reviews that need to be coordinated with many diverse stakeholder groups such as users, governance bodies (such as Project Management Offices), DevOps, and marketing before the application can be released.  In DAD we therefore describe a distinct set of activities to prepare our stakeholders for the release and support of the new application.  This could include activities such as user training, data conversion, documentation, hardware deployment, preparing customer support, database tuning, and last minute defect fixing.  To recognize the clear difference from a typical Construction iteration, we describe iterations dedicated to deployment as “Transition” iterations.  The illustration shows how we can inject iterations into our release schedule to deploy increments to the user community.  In this example, after the fourth Construction iteration, we may have an additional set of features representing a minimally marketable release that is worthy of a Transition iteration to deliver the value to the users.

When we have an application that needs to be delivered over multiple releases, following the Transition phase we may start work on a new release by continuing the Construction phase.  Since we would typically have an existing work item list (backlog) of outstanding requirements, we can simply pull more requirements off this stack and continue to build more functionality.  In this way, it may not be necessary to have another Inception phase unless the vision has materially changed and needs to be revisited.  Your organization may, however choose to run small Inception iterations at the beginning of each new release.

In this example, after the fourth Construction iteration, we may have an additional set of features representing a minimally marketable release that is worthy of a Transition iteration to deliver the value to the users.

Some people mistakenly interpret DAD as one deployment to the customer at the end of the project.  If possible, we prefer to deploy frequently, in support of the agile alliance principle “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software”.  Our minor quibble with this principle is that what we deliver to the customer is a “solution”, which may not only include software, but also business process changes, training, or marketing activities, for example.  Our experience is that large projects typically involve a lot of work beyond the software itself which can also benefit from agile collaborative practices.

Injecting Transition iterations to deploy increments to customers

While the illustration refers to each iteration delivering a “potentially shippable” increment, which is what the agile community typically calls it, we actually prefer to use the term “consumable” to indicate that it not only works, but it is also actually usable by the customer.
________________

In our upcoming book we discuss patterns like these with considerations for what might make sense for your program or project in various circumstances.

About these ads
  1. January 8, 2012 at 3:18 am

    >>activities such as user training, data conversion, documentation, hardware deployment, preparing customer support, database tuning, and last minute defect fixing.

    – Would a setup of a practice like “Continuous Delivery” help to overcome these hurdles, especially: “Last minute defect fixing”, “Hardware deployment” and “database tuning”?

    – Why is “documentation” not part of a iteration goal before the “Transition iteration”?

    – Is the “Transition iteration” a whole separate iteration, where the goal is only deployment?

    Peter

  2. January 8, 2012 at 10:38 am

    Excellent questions Peter. I can tell that this is going to be a long reply to do it justice :) And I am sure that Scott will have his own opinions as well. First of all, let me give you a bit of context around DAD. It is a framework of ideas that help us deal with agile in the real, enterprise world. Mainstream agile theory has very good ideas such as keeping teams small, collocated, no external intervention, etc. However, DAD says, ok, but what if I have to deal with governance, scaling, enterprise architecture etc?
    ___
    So to your point about Continuous Delivery, sure, we should strive for push-button deployment to all environments, including production. Unfortunately, I don’t typically see many organizations that allow us to do it as simply and as frequently as that. There absolutely ARE organizations such as Amazon that are very good at this, but most organizations are not even close yet IMO. In my current engagement I am the agile coach for a large program, transitioning it from a waterfall approach. One of our goals is push-button deployment to any environment. However we will not be able to execute this against production without a lot of paperwork :( For instance, to make a change to their mission critical DB, they need approval from over 10 approvers first, many from other programs. Operations also needs 4 days notice for any non-emergency production change. Details of the change must be documented, including how to back the change out. And actually running of any change script is done by an independent group. This may sound ridiculous, but there are some very good business reasons why things are done this way. Agile it is not, and will not be in the foreseeable future. So DAD recognizes these realities, and suggests ways to minimize the bureaucracy. Long answer for that question,but essentially, yes, we strive to automate as much of the deployment pipeline as possible, and move to a Lean model if we can in order to deliver changes frequently, in which case the need for a Transition iteration could go away.
    ___
    I’ll answer the last question next. Transition is indeed a separate iteration, but the goal is more than deployment. You could have multiple go-lives during the iteration for example. The first one, and then a patch. During the iteration, you would do other activities like the ones that you mention in your comments. For instance, most companies that I have worked for do not unfortunately have a replication of their production system for testing, including production data and the ability to simulate real production variable load. Not to say that it is impossible because it is with techniques such as shared queues, but it is usually cost prohibitive. As a result it is typical to see DB tuning of indexes and such after the system has gone into production. Regarding defect fixes. Good agile practices fix defects as we go so in an “ideal” world we deploy with no defects. However this is seldom reality. Almost every large project I have been on has deployed software with known defects. If you wait to fix every defect in every story, it can really slow you down, and often it is actually a good business decision to defer certain defect fixes (depending on their likelihood and impact). Often the choice is to release a patch with defect fixes after the initial release. There are many other examples of non-construction type of activities that occur in Transition. On a recent project team members had to help with the training during the transition phase for instance.
    ___
    Concerning documentation, I certainly don’t mean to infer that documentation all happens at the end. In fact we describe the practice of “continuous documentation” in DAD. In my current project we use a wiki and it is continuously updated by all members of the team. Your documentation should of course be a byproduct of your work, not a separate exercise, and we should minimize documentation as much as possible. However, there is often special documentation required as part of an organization’s governance. Staged gate documentation for example, and sometimes it gets finished in Transition.
    ___
    The bottom line to all of this is that we definitely want to minimize the Transition work, ideally making it go away and deploying to customers in every Construction iteration, or even every day of every iteration! But where large agile projects need to work within a sub-optimal governance framework and interface with traditional projects and their methods, we need to make compromises to get the job done. DAD recognizes these realities and suggests ways to maximize your effectiveness.
    ____
    One last point. Some organizations may let you build their large complex projects in an agile fashion, but insist on one go-live at the end. Admittedly not preferred, but often happens. On one such project the Transition activities included extensive communications activities, both inside the company as well as outside customer users in preparation for the system, a massive data conversion and migration from a legacy batch environment to modern clustered on-line system, negotiations with the union as business processes and job descriptions changed with the new system, configuring firewalls and SSO, and last minute cleanup of defects. Believe me that Transition was an all consuming phase, extremely complex, and not a typical Construction iteration.

  3. January 19, 2012 at 8:39 am

    Hi Mark

    Thanks for the answers. A very interesting topic!

    >> There absolutely ARE organizations such as Amazon that are very good at this, but most organizations are not even close yet IMO.
    I experienced the same, but shouldn’t we teach and help them to get closer to that goal? And help organisations getting to the root of problems?

    >> This may sound ridiculous, but there are some very good business reasons why things are done this way.
    What are those business reasons?

    >> For instance, most companies that I have worked for do not unfortunately have a replication of their production system for testing, including production data and the ability to simulate real production variable load.
    Same hardware for testing as in production; I have never come across of that either, but I think you can solve that by getting a performance baseline in the test environment. With that you have an indicator of your system performance over time.

    >> As a result it is typical to see DB tuning of indexes and such after the system has gone into production.
    How are you measuring (before/after) those DB tunings in production. In my experience the production load is very unpredictable.

    >> There are many other examples of non-construction type of activities that occur in Transition.
    I am curious about those. Maybe topic for another blog post?

    >> One last point. Some organizations may let you build their large complex projects in an agile fashion, but insist on one go-live at the end.
    In almost every project there is 1 go-live at the end, which doesn’t mean you cannot deploy an earlier version and learn by doing it (maybe even deploy with a “feature switch”). Have you considered a strategy where you rollout in baby steps instead of 1 big bang?

    Thanks!!

  4. January 20, 2012 at 11:17 am

    Hi Peter, as a reply to some of your comments:

    >> What are those business reasons?
    – There are dozens of reasons why production changes need to be carefully controlled, often through a single source. In this way, we have audit trails of who did what, when, and why. If we didn’t have such controls in place a rogue programmer could introduce a change into the production system that could bring the organization to its knees, potentially without a way to back out the change. Examples of what could happen if controls were not in place include; unauthorized running of queries to access personal information such as banking details or medical history. Another example might be where data is shared by multiple business groups. Removing a column from a table should be discussed with these stakeholders before one of them does this.
    >> How are you measuring (before/after) those DB tunings in production. In my experience the production load is very unpredictable.
    – agree, load is unpredictable, but most businesses have regular patterns of spikes in activity, such as around thanksgiving. There are many tools available to monitor production performance before and after doing the tuning.
    >> I am curious about those. Maybe topic for another blog post?
    – Sounds like a good idea.
    >> In almost every project there is 1 go-live at the end, which doesn’t mean you cannot deploy an earlier version and learn by doing it (maybe even deploy with a “feature switch”). Have you considered a strategy where you rollout in baby steps instead of 1 big bang?
    – Yes, I have more than considered it, it is a goal of every agile project I am on. The challenge is to carve up enough functionality to be cohesive enough to be deployable independent of the remaining functionality, at least for large new applications. Once you have a decent amount of the application deployed, deploying frequent releases feature by feature becomes easier.
    – Indeed there is still a huge problem in our industry where it is considered healthy behaviour to batch up a huge amount of work and deploy infrequently, often only every year or two. There is a huge body of evidence that shows that large projects fail, yet we continue to fund them. Fortunately, anyone who adopts agile approaches understands this is wrong and breaks these projects up into smaller projects with frequent releases.
    – DAD acknowledges that ideally we deploy every few weeks, or even daily. But in reality, for new large projects, we first need to execute multiple Construction iterations to build up sufficient functionality before we deploy what we have done via a Transition iteration. Thereafter you should strive to deploy as frequently as possible.

  5. August 27, 2013 at 10:36 am

    Tom Morgan & I have been discussing this. First the question, then a blurb on why we are asking.

    What is the recommended method for estimating length of Transition phase, and how does that play into the release timeline versus release burndown?

    We develop a “release plan” which is based on velocity of the team during Construction, then we have different teams of people who perform tasks (often only once) during Transition. The size of those Transition work items is not the same as the Construction work items; the definition of Done for a release is not the same as Done for a Work Item …so … a “release plan” developed from the velocity to construct (define + design + build + test) a product is not going to be particularly useful for delivering the solution (software + docs + training + configuration + ops scheduling) into production. Our “Release Burndown” is in fact just a burndown useful for the release from Construction to Transition (smells like waterfall).

    I get the sense from terms in this blog post and from visual depiction in the book that Transition is considered one long (if the average Transition is 4.6 weeks and the average Construction iteration is two weeks) iteration. In other words, Transition work is not (typically) incremental and iterative? [Figure 20.4, though, indicates "one or more short iterations" for Transition.]

    Absent iterations, we are denied velocity, but the practical truth is that we couldn’t wait until Construction completed to estimate the length of Transition. We need lead time to prepare the business environment (people, not machines) months in advance of a release because we are but one of many releases with which they must contend. They need time to schedule people and facilities for training.

    How then ought a disciplined agile team (not yet in continuous delivery mode) estimate the portion of our release plan/timeline for Transition? History, expert opinion, W.A.G.?

    • August 27, 2013 at 11:30 am

      Thanks for your comments Skip. I have a number of thoughts here. Ideally the team that builds the solution transitions it to the stakeholders but as you know this often is not the case in enterprises where a separate group is responsible for the actual deployment. So as you suggest, the team’s velocity is not relevant for planning your transition. Although our surveys indicate that the average Transition phase is indeed around 4 weeks, this obviously is not optimal. I suspect that as an industry we will get better at this and that future surveys will see the average duration of Transition reduced. Certainly many organizations are moving to the continuous delivery lifecycle that DAD describes. Please don’t consider Transition to be one “long” iteration. If your reality is that you need a long time to roll something out, as in the case study of a POS rollout to stores across a country, then I would carve it up into many short iterations. Again, not ideal, and an uncommon release pattern to be sure. As far as estimating, although the agile community would throw up its arms at the mention of this, the truth is a typical large-scaled Transition phase can be deployed in a traditional manner. A lot of the tasks are by their very nature serial, and require up front planning and scheduling. One of the key reasons we use agile techniques is to deal with uncertainty. If you have a lot of uncertainty going into Transition, then clearly the work in Construction to mitigate the transition/deployment risk was not done. With transition risk properly mitigated, these phases can be executed in a predictable fashion in a traditional manner, which your ITIL and COBIT people will mandate anyway :)

      Regarding release burndowns, they indeed do burndown to the end of Construction, not until the end of Transition. As an aside, I find burndowns to be useful for scope-driven projects (i.e. the number of iterations will vary to deliver the minimally marketable work in the work item list). However, many projects are date-driven, and adjust the scope accordingly. In these situations, while a velocity chart can be very useful, certainly a ranged burndown is less appropriate.

  6. August 27, 2013 at 12:25 pm

    Thanks, Mark, for a valuable response and doggone quick, too! That answers our key question. We recognize some of the work currently done in Transition can be brought in earlier to shorten our delivery timeline. As the teams mature, we’ll begin to press harder for those changes.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 607 other followers

%d bloggers like this: