Failing “soft” is an Option in Agile

Recently, I noticed the following interesting post from BMC’s Paul Beavers: To Be Successful with Agile, Failure is More than an Option, it’s Necessary.

Paul highlights: “Failure from time to time is expected – trying, assessing, and adjusting are not optional”. This correlates with my experience as it’s in the stretching, and occasionally failing, in agile that creates the fastest learning environment and the highest end-result productivity. Of course, it’s best to fail soft rather than to fail hard. What I mean by this is that the rapid iteration cadence provides opportunities to fail soft at the iteration boundaries. These early failures help assure that the bigger and more consequential failure, failure to release, is a much lower probability.

As agile master Alex Yakima once commented while looking at an upcoming release plan and the few remaining iterations, “nope, not enough times to fail“. So we shortened the iterations (down to a few days in one or two cases) and made the release on time.

Editors update: I just saw this sign on the iteration tracking wall in an XP shop:

Fast failure is an option!

Release Planning Day 1 Narrative (AERP5)

Note: This is part of a series of posts (AERP1-n) highlighting the critical role that release planning plays in achieving enterprise agility.

Recently, I outlined a standard (or perhaps initial starting) agenda for Day 1 of such an event in the following graphic.

Here’s the hour-by hour narrative that should accompany this graphic.

Session I – Business Context

The kickoff to the program should be led by the most senior executives who have responsibility for the product, system or portfolio which is the subject of the session. The meeting planners (Project or release managers, engineering managers, facilitators, etc.) should reach as high in the organization as possible for the kickoff speaker. That individual should have the awareness of the company’s context in the market as this is a great opportunity to bring a key group of stakeholders (those dev managers, team leads, product owners etc. who are present) up to speed on what matters most to the business. This might include internal organization or other factors, as well as revenue, market share or other external indicators of the company’s position in the market. If there are new corporate initiatives underway, this is an opportunity to present those as well, but remember there is typically only one hour for such a discussion, but the next opportunity is only (insert Internal Release Schedule interval here) days away!

Session II – Product Vision

The next session is a presentation by the product managers of the current vision for the product and the tentative roadmap going forward. The main objective of this session is for the product managers to communicate to the teams the objectives (or themes) and the specific feature priorities for the upcoming release(s), which may be either an external release (general availability to customers) or internal release (potentially shippable, but in any case, fully evaluate-able, complete system increment). If there are multiple product managers for the system, each will likely need some time on stage, but time must be limited based on scope and size of the meeting. Also, if there are common requirements (standards, performance and reliability objectives, internationalization, etc.) that must be imposed on the subject system, those should be highlighted here as well and posted somewhere for easy access by the teams. At the conclusion of this session, product managers should provide a handout or post their feature priorities visibly as those are the driving elements for the upcoming sessions.

(Note: Taken together, this meeting delivers the first two elements (Vision and Roadmap) of the enterprise practice Lean Requirements at Scale (Chapter 17).)

Session III – Initial Release Planning: Development Team Breakouts

The next session is the longest (three to four hours) and perhaps most critical session of the two-day event. In this session, the component teams break out into separate meetings (typically team and tech leads, agile masters, product owners, etc) and draft their initial plan to achieve the objectives of the next upcoming release. (Note: Rarely does time and visibility allow planning more than one such release at such an event). In this session, the teams iterate through the following process:

• Brainstorm and estimate all the stories that are necessary to meet the objectives of the release
• Place the stories on iterations on the release plan in sequenced and prioritized order
• Factor in local dependencies as well as interdependencies with other teams
• Create a backlog of things that can’t be accomplished in the period and that can be postponed
• During this process, there will also be variety of discussions with product managers, system architects and other teams to better understand scope, priorities, necessary infrastructure development, potential shared code, etc.

This process continues until one of two things happens:
1) the objectives are met, or

2) the available resources for the release period are fully allocated

For example, if the teams have adopted the “standard” iteration and release pattern I proposed in a prior post, then a team’s draft release plan might looks as follows:

(Note: we’ll discuss the above picture in a “What’s Wrong with this Draft Release Plan?” future post. But for now, we have a draft plan.

Session IV – Initial Release Plan Review

The next session is an “all hands” session designed to develop a common understanding of “how we are going to deliver this release”. In this session, each team presents their draft release plan in front of all the other teams (15-30 minutes maximum, depending on number of teams) highlighting issues, bottlenecks, interdependencies and the like. During this review, all teams will be looking for any impact on their plan that the subject release plan may imply. The big questions that are being asked and answered in this session include:

• Is it overloaded (too many stories) or under loaded (too few)?
• What blocks or interdependencies may be present or implied?
• Does it respond to the vision and feature priorities presented?

And finally, the key question

• Is this a sensible plan from this team?

During this process, any blocks or issues that are raised that are outside the control of the team and which must be addressed before any commitment are placed on an issues parking lot.

Session V – Problem Solving/Draft Commitment

By the end of this session, it will start to become obvious to all in the room (including the senior product managers and executives) as to whether the objectives of the release are practical or impractical. Most typically, the latter answer presents itself as a tentative conclusion and much more work remains. (Note: while this may seem disconcerting, we would question the aggressiveness of any enterprise or process that brings together such resources in order to discover a release plan that is already “in the bag”.)

So in this likely case, the next session is designed for problem solving, scope management and tacking the hard issues. Perhaps during this period an agreement on a new scope can be reached and all the issues that have been raised can be addressed. But in all probability, this can’t be accomplished in the one or two hours left in this meeting. That’s why we have scheduled the session for two days – not one. Even then, the teams will likely have some important homework assignments for the evening.

The evening assignments and the day two program and narrative will be the subject of future posts.

A Standard Iteration and Release Pattern for the Enterprise?

I’ve discussed “around this topic” in the book and in various posts as well as the classroom environment, but it occurs to me that I’ve never directly proposed nor published a standard pattern for iterations and releases for the enterprise. While “standardization and agile” are unlike things in many agilists minds, I often fall back on the mantra of make routine that which can be routine in order for the company and teams to be able to focus their energy, creativity and initiative to flex about the things that cannot be made routine. (A daily stand-up is just such an example, meet routinely every day to discuss the events of the prior day and the upcoming day, so as to avoid having to schedule ad hoc meetings throughout the day).

Moreover, I’ve also recommended that agile teams focus on high frequency internal releases, and not worry so much in advance about what does or does not go to the customers or market, as there are a variety of outside factors and other stakeholders who make that determination. (Chapter 20 – Impact on Customer and Operations). Along with that guidance, we have seen or espoused other recommendations that include:

• Standardize on a short iteration length – my primary recommendation – two weeks
• Allow for one hardening iteration at the end of a release cycle. These are iterations devoted almost exclusively to technical debt or additional testing backlog (such as system and performance tests which may require resources not practical in the daily build environment, testing on secondary platforms, etc.), or perhaps finalization of documentation and support materials, etc.
• Push product to the market as frequently as practical – typically 60-120 days maximum

Given these inputs a standard pattern for internal releases can emerge as seen in the figure below. Note: some of these internal releases may morph to become external releases, but the teams do not typically have to worry about that too far in advance, especially if they are just starting down the agile enterprise path.

I’ve used this pattern successfully to kickstart a number of larger enterprises. With some experience, the enterprise will likely modify this pattern it to better suit its purpose over time, but it’s a great starting point.

Release Planning Day 1 Schedule (AERP4)

Note: this is one of a continuing series of posts, categorized under Release Planning.

In earlier posts, I’ve described the critical role that release planning plays in enterprise agility. I’ve described the motivations and objectives (AERP1 and AERP2) and planning and preparation (AERP3).

Over time, I’ve developed a somewhat standardized, two-day “program” for such an event which has served a number of teams quite well. This program provides a draft, tried and true agenda that your teams may want to start with. Of course, as with all things agile, there is no one size fits all, so you should feel free to try it as is, modify it, or throw it out and plan your own event. In any case, I’m confident it will be an important learning experience for the enterprise.
Here’s what Day 1 looks like:

Perhaps you can see from the end of the program, that one day doesn’t always quite get it done; that’s why we have reserved Day 2.

Meeting Deadlines – “Tiger Teams” vs. Agile Project Management Practices

Often times when introducing some of the basic agile project management practices to new teams (for example, the daily standup) some of the more experienced (ok, older) team members comment on the fact that these techniques look a lot like the techniques they used in the past when their projects were in deadline trouble. Indeed, most of us have similar experiences with challenged projects and when a project is struggling, and the deadline approaches, we would always fall back on some tried and true disaster recovery techniques. We’d form “tiger teams” that would typically work as follows:

• Quickly form a team of key, dedicated individuals, representing all the necessary disciplines
• Empower the team to cut scope as necessary in order to achieve the deadline
• Meet daily, first thing every morning (meet twice daily as the deadline approaches)
• Bring honesty and complete visibility to actual, objective (not planned) project status
• Commandeer outside resources as necessary
• Continue doing all the above until deadline achieved

We immediately see the parallels to agile project management, reflecting again that agile is not so much new, as a synthesis of some of most effective and highly tactical project management techniques from prior experience.

Parallels are as follows:

Tiger Team Action	Agile Team Principle
Form a team of key, dedicated individuals, representing all the necessary disciplines	Get the right people on the team. All necessary disciplines (define-build-test) must be present. Dedicate the team members for the life of the project (iteration or release).
Meet daily, first thing every morning (meet twice daily as the deadline approaches)	Daily standup. (Plus we have observed many teams doing twice daily stand-ups as iteration or release deadlines approach.)
Empower the team to cut prioritize and cut scope as necessary in order to achieve the deadline	Product Owner who is a team member, empowered to manage priorities and cut scope is systemic in agile
Bring complete visibility to actual, objective (not planned) project status	Empirical, demonstrable frequent outputs in hands of product owner and customer
Manage/commandeer outside resources as necessary	Commit resources, eliminate outside impediments that inhibit success
Continue doing all the above until deadline achieved	Repeat for every iteration throughout the project lifecycle.

In other words, the best way to keep projects out of trouble is to apply the same intensity at the beginning of the project as we used to do at the end. While this may seem intense for the routine day to day existence of the project, the actual result is sustainable and highly productive development process that avoids the enormous deadline pressures we used to see at the end, as the illustration below shows.

More on Date vs Scope – Solution: Staying Releasable?

My discussions with Paul at BMC continued, and I think we arrived at a reasonable conclusion as the thread below illustrates. (You might want to start at the bottom and read to the top.)

Paul,

Ah, I think you have found the keenest insight on the thread:

” being releasable within the iteration” (and release)

Indeed, when this is accomplished, the decision to release or not is either obvious or moot. You can release now, and again later if need be (subject of course to all the significant issues outside the organization in supporting a release). In general, the date vs scope problem is largely mitigated.

Of course this is based primarily on solid engineering practices, not management practices, and all teams struggle with staying releasable. I would have to say that I do see teams able to release almost weekly (each iteration is indeed a potentially shippable increment of functionality), but I will have to be honest and say that I have never seen a team or application of the scope of performance manager yet achieve that objective. So when it comes to TDD and continuous integration at your scope, you are pioneering new ground, so take a deep breath, and dive back in to “staying releasable”.

Dean

—–Original Message—–

From: Beavers, Paul

To: Dean Leffingwell

Dean,

Thanks for the response and the thought provoking ideas. Your explanation makes sense to me. To be honest what we strugglle with the most is still being releasable within the iteration. If we could fix that issue we would rarely be faced with the end game decision of to ship or not to ship. Seems there is some value in optimizing such that the decision doesn’t even need to be made.

Thanks,

Paul

Fixed Date or Fixed Scope at Enterprise Scale?

Well, the “fix the date and adjust the scope mantra” of agile is still subject to ongoing debate in some circles. In general, I stand by my suggestion that the easiest way to establish an agile, quality-based, software delivery rhythm is to fix ALL the release dates well in advance and adjust scope so that they are reliably met. Of course, there are exceptions to every rule. For further illustration, I thought you might benefit from an “up close and personal” discussion with some of the executives at BMC (provided with permission of course).

First, the question from Paul Beavers, a Development Director at BMC:

Dean,

“I would like your opinion on our thoughts / dilemma here. As you know we have implemented Scrum across the entire BPM organization, today it touches about 250 people.

Frequently, we find ourselves at the time in which we scheduled to release our products to manufacturing faced with a decision. Typically, the decision is do we ship the product at the scheduled time or do we put in one critical late breaking feature. Often times, this decision is around whether or not to fix a bug and slip the date or release with the bug. Often times we know the answer to the second question based on the severity of the defects. My belief is if we evaluate early enough and decide to add a hardening iteration and move the date, it is not particularly in violation of good agile practices, as long as the manner in which the date change is made is controlled and systematic. As I am sure you have encountered, there are purists who say ship on the originally planned date no matter what. My view on the matter is being Agile and following Agile principles does not allow us to relinquish responsibility for “thought” in each business situation.

I would like to understand your views on taking a hard line on the dates versus being flexible. What are the drawbacks to each approach?

I look forward to discussing this with you.”

Paul

============================

My response below:

From: Dean Leffingwell

To: Beavers, Paul

Subject: RE: A couple of ideas and advice related to Agile

Hi Paul,

I know you know that there isn’t a “right” answer to your question. Indeed, I have had some interesting discussions with Jim Highsmith on this very topic and the “fixed scope or fixed date debate” will go on for years to come. But here are some (please consume then in parallel, ie initially unrelated) thoughts for context:

1) I’m one of those that pushes hardest to meet the date because agile is dependent on a team meeting its near term commitments. (long term is so uncertain, something has to be relied upon). Moreover, holding the date forces effective scope management, so teams learn that they can’t just shove more scope into a scheduled release because the date is firm and the resources are committed. That forces prioritization on the part of product owners which is requisite for agile development, and yet product owners are very reluctant to prioritize until forced. In addition, prior to agile, teams haven’t been meeting their dates generally, so insisting on date commitments resets the standard of accountability in the organization and highlights one of the major benefits of agile. “we meet our commitments”. It also forces moving testing forward in the lifecycle. (“the end date is known and soon, I have to be testing now”). It also allows outside organizations, marketing etc, to depend on software delivery time-lines, often for the first time in their careers!

2) One question for the organization is what is “routine” and what is “exceptional”. If it is routine to slip the date and add features at the last minute, we know that doesn’t work because of the inevitable quality slippage, death marches, etc. , and that does not characterize agile. If we slip it one time and one feature, why not slip it the next time and add 3 features; pretty soon value delivery lags, and agile or not, the organization is not delivering reliably. But if its exceptional for a specific purpose, for an organization that routinely meets its commitments, and otherwise follows solid agile practices, then it can be highly useful at times to take the exception.

3) One of the most memorable agile mantras I’ve heard is to defer decisions (admittedly, design decisions form the speakers perspective) to the “last responsible moment”. That should give enterprises the flexibility to do the right thing, whatever that may be, and to decide later in the cycle than would be the case with waterfall methods.

Vignette 1, I once saw a great agile team extend one iteration to three weeks from two, because of interdependencies and conceptual integrity of the iteration. This is completely counter to all advice of keeping all iterations exactly the same length. And yet at the time, I thought it was brilliant, but only because they had shown that they could deliver reliably iterations every two weeks for almost six months first, and I had no fear that they would decide they liked 3 weeks, or 4 weeks, or 5 weeks better than two, because they had truly mastered agile. In this case, scope wins.

Vignette 2: I’m working with a development stage company right now, and the only thing I can tell them for certain is the date we will ship, even though we are clueless as to what that might actually be! In this case, date wins.

So no, I don’t take a hard line always on fixed date, but I’m about 80-90% biased that way, and nearly 100% biased early in an agile transformation.

Happy to talk more.

Dean