Agile Iteration. Agile Release Train. Agile Fractal.

As I’ve described at length in Agile Software Requirements, Scaling Software Agility and this blog, the Agile Release Train can provide tremendous benefits to the larger agile software enterprise. I, and many others, routinely apply it to: 1)   Align agile teams to a common mission 2)   Institutionalize program-level, product development flow.

However, being a man of oft-too-many-words, I struggle sometimes to describe the Agile Release Train in the simplest possible terms. A while back, I mentioned the release train as a fractal above the sprint/iteration. (A fractal is a geometric shape that can be split into parts, each a reduced size copy of the whole). Maybe that’s the best way to think about the release train. Let’s try it.

There is no debate that an agile iteration/sprint has a common and simple pattern: 1. Plan. 2. Commit. 3. Execute. 4. Demo. 5. Adapt (retrospect). It looks like this: The release train has more teams and longer iterations (uber sprints, composed of multiple, aligned team sprints)  but the pattern is exactly the same, it’s just that it is at the next level of program scale. It looks like this:

Does that simplify things a bit?

Agile Release Train: Develop Synchronously, Release Whenever.

The software development paradigm called the Agile Release Train (introduced in Scaling Software Agility and better elaborated in Agile Software Requirements) is being put to effective use in a number of larger agile, software enterprises. It is often put in place between six months and a year after the team-level agile transformation. (That’s when the entropy and local optimization of all that team-level empowerment can become an impediment of its own).

As I’ve described, the loftier goals of the ART are to “align teams to a common mission” and “institutionalize product development flow.” What that translates to is “get all the teams pulling in the same direction and keep them there” and “implement development practices based on synchronization and cadence (uber-sprinting) that bring the same agile content management and continuous system integration discipline to the program as you have done with the teams”.

While conceptually, it’s not that complex a process, there are a number of misconceptions about the train that require clarification.

The first is the behavior implied by the name Agile Release Train. As illustrated in the Scaled Agile Framework Big Picture graphic below, development occurs on a fixed cadence of about 8-12 weeks (often 4 dev sprints plus a hardening sprint if applicable).

Each larger interval results in a product or full system release, or Potentially Shippable Increment. (Note: of course, our goal is to have  PSI’s every darn day, but in the larger program, that’s not a practical reality). Though the PSI cadence is highlighted, the intent is that teams are free to release software at any time the market demands it, or whenever it benefits the enterprise to do so. However, since the name is “Release Train”, and the cadence and the Scaled Agile Framework Graphic are overloaded, some assume that you can and must release only on the fixed time boundaries. That can create some unwarranted resistance to adoption of the train. Comments like “we can’t adopt ART because our customers don’t want releases that often” or “we can’t adopt the train because we have to release more often than the train cadence” are not uncommon.

But you can and should synchronize development. Since this is a common misunderstanding about the train, I typically use the graphic below to illustrate the conceptual firewall between development cadence and release cycles.

To summarize from the graphic:

1)   Development occurs in constant length intervals, each characterized by a plan-commit-execute-demo-retrospect model with forced asset synchronization at the two-week sprint cadence synch points.

but

2)   Any team, or the entire program,is free to release any product, component, or system to anyone whenever they want, subject only to the program/enterprises release quality and governance criteria.

In other words, Develop Synchronously, Release Whenever!

Next post: The Agile Release Train as an agile practice fractal above the teams.

Agile Release Train Metrics

I’ve had an On-again, Off-again, love-hate relationship with software metrics throughout my career. The On part is:

• of course we have to measure ourselves, less we  a) not advance the science of software engineering in any credible way, and b) not provide any professional, quantitative input to the fiduciaries and key stakeholders who ultimately determine the economic benefit of our work (and indeed decide to pay us or not)

the Off part is:

• a)   why don’t any of these metrics we’ve traditionally applied actually seem to work?, and b) can’t all of these be used by management to drive behaviors that may eventually be counterproductive, and/or c) be manipulated by teams to some similar, ill result?

But the On part of this debate within myself continually informs me that “measure we must” , particularly if we want to be credible with our agile model in the enterprise setting. So, I took my best shot at project, process, and an enterprise “balanced scorecard approach” to metrics in Chapter 22 of Scaling Software Agility. I reviewed it today and it still seems credible (at least to me).

Recently, however, I’ve been repeatedly faced with suggesting metrics that can work well in the context of each PSI/release in an Agile Release Train. That’s a simpler discussion, because there, context is known, and the measurement problem is focused on a specific program, with specific teams, in a specific technical and business context. To that end, I’ve recently been presenting the following set of metrics to the release train programs and Agile Working Groups I’m involved in.

A set of reasonable metrics for an agile release train

At each PSI, each team self assess and report on the above, and the aggregate can be rolled up into a program level summary, which should establish some sense of health and progress for that train.

Most of these are fairly obvious to agile teams. I described least obvious one, “percent of business value” achieved in this earlier post.

Hope this helps at least somebody.

Agile Release Train Community of Practice

As those of you familiar with Scaling Software Agility, Agile Software Requirements or this blog know, the Agile Release Train is the mechanism I like to recommend whenever an enterprise needs to harness a significant number of agile teams (5-10-15) to a common mission. And in the larger enterprise, that’s pretty often. As I described in Chapter 15 of ASR, in more abstract terms, the ART is used to:

1. Drive strategic alignment
2. Institutionalize product development flow

The agile release train, is implied in the Big Picture:Scaled Agile Delivery Model, as seen below:

A number of large software enterprises are using this or a similar mechanism (an uber sprint?) to build larger scale, fully agile programs. In so doing, a number of new best-practices-at-real-agile-scale,  are starting to emerge.

My colleague Jorgen Hesselberg, is organizing a small community of practice to explore both challenges and patterns of success. I know some of the likely participants, and I can assure you there will be some lively discussions about some of the largest agile enterprise initiatives. Topics for this COP are likely to include:

• organizing trains around value streams
• preapring for the release planning event
• running the event
• release planning at scale (50-100 largely co-located devs and testers) and super scale (100-200 distributed, mutli-site, concurrent planning)
• roles and responsibilities
• release train governance (keeping the train on the tracks, PMO involvement, etc)
• metrics
• coordinating epics across multiple trains

If you’d like to participate, contact Jorgen at jorgen.hesselberg@navteq.com.

If this comes together, I’ll certainly be blogging about the process and results of this strategically interesting, agile-at-scale COP as soon as they become available.

Agile Portfolio Planning and Business Epic Kanban System Overview

My colleague and “agile ninja”, Chad Holdorf, has been working with agile at enterprise scale for some time now. He has adopted the Big Picture from my Agile Software Requirements book, which he describes as a “Scaled Agile Delivery Model” (I like that name a lot; I might even adopt it). Recently, he has been working at the portfolio levels of the enterprise to effect lean and agile thinking there too, and to provide a systematic way of moving projects from the portfolio (enterprise backlog) into development via a set of Agile Release Trains. He’s using the enterprise backlog model I described on my blog and the paradigm of the business epic kanban system , which I describe in Chapter 23 – Investment Themes, Epics and Portfolio Planning, of ASR. He has just published a short video/blog post that summarizes this system brilliantly. Check it out at: http://www.scaledagiledelivery.com/2011/04/22/agile-portfolio-planning/.

In addition, as agile tooling is a necessary condition for success at enterprise scale, he’s been collaborating with Catherine Connor at Rally (as have I) on the development of their new portfolio planning tool called “Stratus”. Here’s what Rally says about Stratus:

“Rally is previewing Project Stratus, a new application that elevates Agile planning and tracking to roadmap and portfolio levels. Based on collaboration with Rally customers, Project Stratus focuses at the PMO level on:

• Keeping development aligned with strategic business priorities
• Visualizing and managing projects and large features across the entire value stream
• Analyzing the roadmap to optimize delivery of value while minimizing risk
• Strengthening feedback loops between the PMO and development teams”

Now ideally, any method (with tooling) that really supported enterprise-level, agile portfolio planning would accomplish a number of objectives:

1. make the backlog of potential and current portfolio projects visible
2. provide wip limits so as to keep the full system in maximum productive flow
3. provide a persistent, single version of the truth in the enterprise (avoid dueling spreadsheets/tools)
4. provide visibility for how current development activities correlate to the enterprises current investment themes
5. provide a capacity-based, systematic way of driving new epics into development

I think you’ll see from Chad’s video, that this system has the potential to accomplish all these objectives.

One last comment: Great job Chad and Catherine! This is clearly “new knowledge” that can help most every agile enterprise.

Scrum of Scrum Ground Rules

In the last post, I described an impressive Scrum of Scrums meeting I attended this week. It’s a pretty big Agile Release Train that has 14 cooperating teams. After just a few months of agile, this program really seems to be “on their game”, and this team clearly is developing some significant “ba”.

I also snagged a photo of their “ground rules for the Scrum of Scrum meeting”. But the photo isn’t clear, so I’ll just list the rules here (with their permission):

• “Like my daily Scrum, this meeting is meant to be fast and short.
• I will be present on time and I will come prepared.
• I will first state my Scrum Team name.
• I will talk about my team, not individuals on the team.
• I understand that problems can and should be raised during this meeting, but I will not discuss solutions until after everyone has had a chance to go thorugh their standard report.
• After all teams have reported, the focus will shift to any issues, problems or challenges that were raised. These will then be discussed and resolved or added to the Scrum of Scrums parking lot for future discussion.”

Thanks guys! (you know who you are).

On “Ba” at a Scrum of Scrums

I was a “chicken” today at one of the more effective Scrum of Scrums for an Agile Release Train that I have witnessed. Big program. 14 teams. The uberScrumMaster (Release Team Manager) started the meeting at 9 am, facilitated the meeting and ran a pretty tight ship.

Fourteen teams (2-3 remote) reported in a standard format (see pic) in about 20 minutes total.

A few “meet afters” ( we need to talk about this more) were raised.

In the standard report, one of the ScrumMasters narrated the following;

–       “yesterday, we reported that we were blocked (something was put in our way) by another team.

–       It wasn’t true. It was we that had blocked ourselves and, yes, we blocked everybody else too!

–       We are really sorry and we fixed it just as soon as we could .”

This conversation happened part in jest and fun, and part in dead seriousness. This team of teams has “ba”. (the energy of an effective, self-organizing team. See below.).

After the standard report, there were 3-4 “meet afters” discussed. The uberScrumMaster took notes and his laptop was projected in the room so everybody can see the work and agreements in process.

The meeting adjourned at 9:25 AM , with all questions asked and answered.

Make no mistake, this team of teams has significant challenges ahead (don’t they all?), but it is just so satisfying when you see it all come together. And they will handle the challenges. I am so proud of these people. It is so rewarding for any of us to be a part of a high performing team.

Again I am reminded, that with just a little leadership (which these folks and their managers, who were not present at this meeting, of course) clearly exhibit, you can trust the teams EVERY TIME.

Quick note on “Ba” below.

========

What is Ba?

Ba- is the Zen of Scrum, a shared “context in motion and the energy that drives a self-organizing team:

–       “Dynamic interaction of individuals and organization creates synthesis in the form of a self-organizing team

–       The fuel of ba is its self-organizing nature-  a shared context in which individuals can interact

–       Team members create new points of view and resolve contradictions through dialogue

–       New knowledge as a stream of meaning emerges

–       This emergent knowledge codifies into working software

–       Ba must be energized with its own intentions, vision, interest, or mission to be directed effectively

–       Leaders provide autonomy, creative chaos, redundancy, requisite variety, love, care, trust, and commitment

–       Creative chaos can be created by implementing demanding performance goals. The team is challenged to question every norm of development

–       Time pressures will drive extreme use of simultaneous engineering

–       Equal access to information at all levels is critical

(source. Hitotsubashi: On Knowledge Management)

Release Predictability Metric

I’ve been using a “Release Predictability Metric” (RPM) on a number of larger agile programs. The goal of this key metric is to help the teams achieve a level of program predictability that their enterprise stakeholders can count on, while still allowing for taking on the reasonable risks that are necessary for innovation , as well as stretching to the maximum potential achievements in a PSI time box. In describing it to a colleague, I realized I had never blogged about it, so I’m posting this excerpted content from the new book to address that oversight.

=====================

Measuring Release Predictability

If you ask top executives what they would most like to see out of the software development process, many will answer “predictability”. And that is one of the many challenges in agile. We can reliably predict quality (by fixing it and adopting effective technical practices) and date (by fixing it) and cost (by fixing the team size and the PSI date) – but we can’t actually predict functionality – at least in the longer term. If we did, we’d be right back in the iron triangle that has served us so poorly in the past. Moreover, if we predicted and controlled functionality long term, then we’d have to temporarily ignore the variability and new opportunities the market presents. That isn’t agile.

However, as professionals, we must be able to provide our enterprise with a reasonably reliable predictor of upcoming events, at least near term, as well as some sense of the future product Roadmap that we intend to execute.

When implemented properly, the Agile Release Train can provide just such a predictability measure, at least for the next PSI (or maybe two). That gives the enterprise from 3-6 months of visibility into upcoming release content – enough to plan, strategize and support with market communications, release launches, etc. The release objectives that we established during release planning are our primary means to do this.

At release planning time, each objective is given a business value, assigned by the business owners for that program or subdomain. During each release retrospective, the teams meet with their business owners to self-assess the percentage of business values they achieved for each objective. This can be done both at the team and program level. For example, a program might rate its accomplishments as follows:

Actual Vs Plan Release (PSI) Objectives

Release Objectives Process Control Band

In the example above, the program accomplished 79% of its release objectives. The questions arises – how good, or bad, is that? To answer this, we must return to our lean principles and the context for the enterprise program itself.

On the surface, at least, it might appear that accomplishing 100% of release objectives is the only worthy goal. Closer analysis, however, tells us differently. In order for a team to routinely accomplish 100% of its release objectives, they must either

1)    drive all risk out of the plan by eliminating or curtailing innovation and risk taking

2)    back off so far on objectives so as to assure completion

Neither of these optimizes the economic impact of our efforts. To achieve that, we need to operate successfully in some acceptable process control band, so that the program has reasonable predictability, and yet allows for the variability, “optionality”, and stretch goals inherent with software development.

In our experience, a program that can reliability achieve most of its release objectives is a trusted program that is an extraordinary asset to the enterprise. In this case, the release predictability measure should fall in a process control band something like that in the following figure over time.

Managing uncertainty with a release (PSI) objectives process control band

In this figure, Team B is predictable, even though they do not routinely hit 100% of their release objectives. They can’t, otherwise there would be no room for innovation and risk taking. However, the enterprise can depend on the fact that they will do most of what they said they would do. Team A, however, is all over the map. It’s hard to manage any program or enterprise with the characteristics of Team A. You simply can’t depend on them to deliver anything like what they predicted.

By creating and measuring this predictability measure at every PSI, the enterprise can eventually achieve the right balance of predictability and risk taking, thereby achieving the optimum economic outcomes.

My Agile Denver Presentation from Last Night

Last night I presented to the local Agile Denver group. This was a variant of my Agile 2010 presentation, but one with a heavier focus on the scalable requirements (backlog) model. The title is:

Scaling Software Agility:
Agile Software Requirements

The content and flow is:

1.Lean and Scalable Requirements Model

Applying the Model

2. The Agile Release Train

3. Rearchitecting with Flow

4. Agile Portfolio Management

I promised to post it, so here it is:

Agile Software Requirements (Agile Denver).pptx

Thanks to Brad Swanson and Bob Hartman for arranging this opportunity.

Presentation: Rearchitecting Enterprise Class Systems

I’ve been using this presentation to provide overview context for some larger enterprises who have interest in the topic of rearchitecting  large scale systems in an agile manner. Of course, it isn’t a trivial topic as the problem itself is quite complex, and this solution requires a “trifecta”  of three advanced practices:

① Lean and Scalable Requirements Model (more)

② The Agile Release Train (more)

③ An Architectural Epic Kanban System (more)

The presentation is here: Rearchitecting Enterprise Systems.pptx

This presentation pulls these three elements into a more cohesive overview, albeit it one that still requires some voice over explanation.