Enterprise Agility–The Big Picture (5): User Stories and the Iteration Backlog

Note: In the post Enterprise Agility: The Big Picture, we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In prior posts, we’ve discussed Teams, Iterations , the Agile Product Owner and types of Backlog. To see the full series, select the “Big Picture” blog post category. In this post, we’ll describe User Stories and the Iteration Backlog [callout (5)].

big-picture-5-user-stories-and-iteration-backlog

Big Picture 5- User Stories and the Iteration Backlog

In the last post, we described three types of backlog that are common in the agile enterprise. These are:

The Iteration (or Story) Backlog – holding user stories and the like that are generally intended to be implemented by a component or feature team in their code baseline in the context of an iteration

The Release (or Feature) Backlog – holding features that deliver system-level value that typically affect multiple component teams and typically span iterations

The Portfolio (or Epic) Backlog – This backlog is a placeholder for capturing and discussing the larger scale initiatives that an enterprise has, or intends to have, underway. Epics often affect multiple component teams, multiple systems, and even multiple products. Epics typically span iterations, releases, and sometimes years!

We mentioned that, of these, the Iteration (story) Backlog is by far the “hardest working backlog” in the enterprise. That is because every team has one and they use it in managing their day to day activities. It is the central repository for all their work, both work in process (scheduled) and future work (unscheduled). It consists primarily of User Stories.

Where User Stories Come From

In an earlier post, we described the Product Owner’s critical role in developing, owning and managing the Iteration Backlog. In its most basic form, this involves identifying, prioritizing and elaborating user stories.

  • Identifying user stories can be done at any point in time and it’s an oversimplification to say that only the product owner can do this. Indeed anyone on the team should be able to identify or suggest potential user stories, and most teams allow anyone to post a potential story in the backlog. So the Iteration Backlog is really a repository of ideas staged for future prioritization.
  • Prioritizing user stories is indeed the responsibility of the Product Owner. While it is a collaborative effort – based on discussions with the team, product managers, customers and the Product Owners vision – no matter the source and the nature of the debate, in the end, the Product Owner makes the call. They are the authority for both what stories get put in the iteration and what the relative priorities are within an iteration. This latter point is important because unless a team is loafing, not every story will be accomplished as planned – no team’s estimates are perfect- and the metaphor of landing the iteration is really shorthand for “achieving the objectives of the iteration by implementing at least the highest priorities stories”.
  • Elaborating User Stories. I’ve often noted that agile development is an extremely and inherently disciplined process. As an example, if a developer/tester pair takes responsibility for completing a story within an iteration boundary, it’s fair to expect that the story is reasonably well understood before they make such a commitment. Achieving this understanding by elaborating the story is a shared responsibility of the PO and the team, but the Product Owner must take the lead and see that stories are sufficiently elaborated to be properly estimated and implemented in a short iteration. Generally, this has to be done just-in-time, but in advance of the iteration boundary in which the story will be implemented.

(Note: for a deeper and more meaningful treatment of user stories, See Chapter 17 of Scaling Software Agility or Mike Cohn’s Agile Estimating and Planning).

User Stories Have State

Tracking user stories through their state lifecycle is how teams manage and assess the status of the iteration. In order to do so, many teams assign various states to user stories and use the states to track progress. For example, Rally Software’s agile project management tool default state model calls out six states:

Backlog – story has been identified, but not yet defined

Defined–a story is sufficiently elaborated so that it can be estimated and implemented

In-Progress –one or more team members has “taken responsibility” for the story and it is in the process of being coded and tested

Completed–coding and testing has been completed, but the story has not yet been accepted by the product owner

Accepted–story has been accepted by the product owner (usually with QA support) into the new baseline

Blocked–progress on the story is blocked for some reason. Blocks are often a key topic at the daily standup and if the story is of high priority, then resolving the block is of high priority too.

There’s More to the Iteration Backlog than User Stories

While the primary content of the Iteration Backlog should be user-value stories, in practice, most teams use the story metaphor to track all the work they need to accomplish in an iteration. In this way, they can estimate and keep track of everything they need to do to be complete the iteration. Thus, there are various “types” of stories that are all prioritized, estimated, and managed together. In addition to true User Stories (those that deliver direct value), other types of iteration “stories” include:

  1. Systems Architecture–refactors and design “spikes” that evolve the product architecture to support new user stories
  2. Development infrastructure–necessary work on the development environment, tooling and the like
  3. Overhead–training, education, company meetings, etc. that require attendance and consume team capacity
  4. Defects–defects (or suites of defects) that need to be fixed in the current iteration

On User Stories and Tasks

For more detailed tracking of stories, teams typically decompose stories into individual tasks that must be accomplished in order to complete the story. Indeed some agile training uses the task object as the basic estimating and tracking metaphor.

However, I much prefer the tracking focus be at the story level (see post the Simple Visible Iteration Kanban Board for a sample, visual Iteration backlog kanban board) and as it are better aligned to business rather than individual objectives. Tasks are a means to an end- basically a work breakdown structure that facilitates estimating and the taking of individual responsibilities- teams will use them naturally but they shouldn’t be the primary objective of the iteration. Note that when you add tasks to the hierarchy (epic, feature, story and task), you actually end up with four levels and that is sufficient for most enterprises, though 5, 6 and even 7 levels are not uncommon for larger systems and enterprises (is that the simplest thing that can possibly work?).

At implementation time, Stories Must Fit in a Single Iteration

Prior to the iteration boundary, not much thought need to be given to how “big” a story is. However, by definition, once bound to an iteration, stories must be sized and estimable so as to be “complete-able” in the course of the iteration. They are atomic- they must be completed in whole (good!) or carried forward to the next iteration (bad). Doing so typically involves splitting larger stories into iteration-size bites, while keeping the focus on value delivery. This learned behavior drives the extreme incremental delivery model that is integral to agile. Estimating and sizing user stories is certainly an art that takes time for the team to master. Mike Cohn’s User Stories Applied: For Agile Software Development is the best text I am aware of on this important topic.

Back to the Big Picture – Tracking the Iteration

With this model, the iteration is composed of all the stories and the state of the stories in aggregate represents the state of the current iteration. The iteration can be tracked in the large via a “burn down chart” (continuous measure of work remaining vs. time available in the iteration) and/or by assessing the individual states of story completion as a sample agile project management tooling screenshot shows below.

With this understanding of defining and tracking the Iteration, we can move on to the Release in the next post.

4 thoughts on “Enterprise Agility–The Big Picture (5): User Stories and the Iteration Backlog

  1. This is a fantastic post that should be read multiple times. I can’t articulate enough in words how important it is to understand the concepts listed above. Failure to do so will lead to numerous frustrations… For example…

    Identifying – By default, PM is accustomed to defining priorities. If the PM organization desires full control, they will fight to prevent others from creating stories. What they fail to realize is that this behavior creates a lack of visibility situation that is prevalent in the development world. Traditionally development “ear marks” a % of time for support and R&D. PM only gets the left over to scope in their priorities. This is inherently flawed. Agile is about visibility. PM should be open to all and any suggestions. Development should be open to all visibility. With the right PO, the appropriate balance of technical debt vs. quality vs. new features vs. architectural changes, etc. will occur

    Prioritizing – agree with everything stated. Key area to watch out for is that without a higher level understanding of the big picture, POs will gravitate towards prioritizing their own silos and may lose sight of the need for a bigger picture priority.

    Elaboration – given past posts, etc. I know you agree with the need to have clear and definitive acceptance criteria. May want to mention this for folks that don’t read the blog end to end. A key activity during elaboration is defining acceptance criteria for it’s how the team will ultimately be judged!

  2. This comment focuses the rest of the sections…

    User Story State
    The only state I see missing is “Not Accepted”. The reason this is important is to give teams historical context for how they executed. While you can move the story from the current iteration to the next, you may lose context that’s important for analysis in a large organization. The information analyzed should never be used to “judge” the agile release train. Instead it should be used by the agile release train to improve

    Size of a Story
    I would actually go a step further and declare that a story should never be more than 1-2 days if possible (assuming a two week iteration). The #1 reason I have seen stories fail is because of granularity. Bottom line, the bigger the story, the more difficult the acceptance criteria. Most new teams tend to fail with the large stories – anything can go wrong and with a large story, you fail it 100% – there is no partial completion.

    Iteration Tracking
    We actually track by done, done, done. When is development done? Testing? Doc? I can’t say we have perfected things yet but it seems to work OK and it keeps the “are you done with task A?” type of questions outside of the daily SOS that is more productive if driven at the user story level vs. task level. We let the teams themselves deal with the task detail.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s