Enterprise Agility-The Big Picture (9): The Agile Product Manager

Note: In the post Enterprise Agility: The Big Picture, we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In prior posts, we’ve discussed Teams, Iterations , the Agile Product Owner, Backlog, User Stories and the Iteration Backlog the Release , Vision and Release Backlog and The Roadmap. In this post, we’ll discuss the Agile Product Manager, [callout 9] below.

Big Picture 9 - Agile Product Manager

I’ve blogged off and on the “agile product manager vs. agile product owner” roles over the last few months so this is not a new topic and I won’t repeat it all here. Many of these posts are categorized under the Product owner/Product Manager category on this blog. We have also touched on the Product Manager in the Big Picture post on the Agile Product Owner and the follow on post. In addition, I described some of the basic Product Manager responsibilities in this article a few years back.

Based on blog hits and search criteria, this topic is pretty relevant right now so it’s a good time to elaborate further on the responsibilities of this role within the context of the Big Picture.

Product Manager/Business Owner/Business Analyst?

Throughout this series, I’ve used the term “Product Manager” instead of Business Owner or Business Analyst, even though those terms may be more familiar in some enterprises. Regardless of organizational structure or title, an effective business owner must exist and they must drive the vision either directly, or through the product management/business analyst organization. An effective Business Owner/Product Manager should exist for each major domain of the solution or the agile teams will be filling in the gaps, possibly with mixed results, depending on their expertise in the various domains. For consistency, we’ll continue to use the term “Product Manager” from here forward, but the reader may wish to translate that into the terms of their enterprise.

Responsibilities of the Product Manager

Agile or not, the Product Manager must fulfill the following responsibilities:

1. Stay abreast of the latest industry trends
2. Understand the changing needs of the market and the customer base
3. Maintain a solid understanding of the current solution

Using this data, the Product Manager’s primary responsibility is to then

1. Articulate a clear direction for addressing gaps and opportunities
   

The Agile Product Manager

Like most every other role in the software enterprise, the Product Manager’s role evolves as the company transitions to agile development methods. The first decision is whether or not the Product Manager assumes the role of the agile Product Owner and thereby takes on the additional responsibilities for iteration planning, story elaboration and prioritization, demo and acceptance. As we’ve noted before, that is probably not practical within the enterprise (see Responsibilities of the Agile Product Owner vs. Enterprise Product Manager and Role of the Product manager ). The differing responsibilities for the two roles are then as highlighted in the following table.

Responsibilities: Agile Product Manager and Agile Product Owner

Agile Product Manager in the Big Picture

As can be seen in the table and as is implied in the Big Picture, we see that the Product Manager owns the Vision and Release (Feature) Backlog and as its implementation in the Release and the
Roadmap. The Product Owner is a charter member of the Agile Team, and owns the User Stories and Iteration (story) Backlog and the implementation via Iterations. Working together, the Product Manager and Product Owner steer the agile enterprise.

To fulfill these responsibilities, the Agile Product Manager:

Owns the Vision – in collaboration with the business owners and the Product Owners, the Product Manager sets the Vision and the prioritized feature set which further describe how the Vision may be fulfilled in the implementation.

Drives the Release Objectives and Priorities through Release Planning – The Product Manager plays a key role in the Release Planning process as well, whereby they have routine and periodic, face-to-face opportunities to communicate objectives directly to the agile teams.

Updates and Maintains the Roadmap – One result of this process is the Product Roadmap, which serves as the plan of record for coupling the Vision to Implementation Timelines. The Product Manager uses the Roadmap to communicate the Product Managers “big picture” to the stakeholders inside and outside the enterprise.

That’s it for the Agile Product Manager. In the next post in the Big Picture Series, we’ll get back to the execution model and describe the role that the Release Management Team plays in helping assure the successful implementation of all that Vision.

Note: A Special thanks to Mauricio Zamora of CSG systems, who contributed some content and insight for this post.

Enterprise Agility-The Big Picture (8): The Roadmap

Note: In the post Enterprise Agility: The Big Picture, we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In prior posts, we’ve discussed Teams,  Iterations , the Agile Product Owner, Backlog, User Stories and the Iteration Backlog the Release and Vision and Release Backlog. In this post, we’ll discuss the Roadmap, [callout 8] below.

The Big Picture 8-The Roadmap

For anyone following this series closely, you may have noticed a subtle shift at some point when the Roadmap icon (8) magically appeared behind the Vision (7). While I’ve been trying not to overload the Big Picture with unnecessary detail, it became clear during my explanations that the Roadmap is integral to the Big Picture and you can’t effectively explain enterprise agility without describing this key element.

When we described the Vision in the last blog post, it was presented as time-independent; the Vision is intended to provide a sense of the objectives of the product or system without any binding to time. This is appropriate as the objective is to communicate the gestalt of “what this thing is we are about to build” and overloading with timelines will likely derail the discussion of the “what”.

However, in order to set priorities and plan for implementation, we need an additional perspective that enhances the understanding. This is the purpose of the Roadmap. The Roadmap is not a particularly complicated thing, nor is the mechanical maintenance of it difficult. For example, a typical Roadmap might be communicated in a single graphic as follows:

Sample Product Roadmap

The Roadmap consists of a series of planned release dates, each of which has a theme and a prioritized feature set. While it is a simple thing mechanically to represent the Roadmap, figuring out the content is another matter entirely. The topic of what the team plans to ship and when can be a deep, fascinating and occasionally contentious topic in agile and we can’t cover it all here. However, the easiest way to think about the Roadmap is that it is an output, rather than an input to the Release Planning process. Fortunately, we covered Release Planning fairly extensively in the Release Planning blog series as well as in the Enterprise Agility Big Picture (6) The Release so there is some Roadmap guidance there for those who follow this Big Picture model.

But before you do all that clicking and reading, here are some summary guidelines for the role of the Roadmap in the context of the Pig Picture:

• The Roadmap is an output, rather than an input to the Release Planning Process
• The next release may be Internal or External. In either case, the dates and themes for the next release are fixed. The features are prioritized and variable.
• The Roadmap is a “plan of record” and is subject to change as development facts and customer needs change
• The teams generally commit to only the features in the next upcoming release. Releases beyond the next represent only the team’s current best estimate.

And perhaps the most important guidance is this:

• Even within the next upcoming release and even though the team has committed to the objectives of the release, the feature set cannot be guaranteed. If it were, then you would have fixed scope-fixed time-fixed resources for an extended time period and that fails the agile acid test. However, it is a reasonable expectation that a team that has committed to the release will:
  1) meet the date
  2) meet the theme or objective, and
  3) deliver most of the features, and certainly the highest priority ones with the requisite quality.

  Doing anything less would be unprofessional and belie the power, discipline and accountability of our agile enterprise model. Moreover, it will eventually threaten our own empowerment, as failure to deliver will inevitably cause the implementation of various controls to “help us”!

That’s it for the Roadmap in the Big Picture. We’ll touch upon it again in our next post, the Agile Product Manager.

Upcoming September 10, 2008 Webinar – Principles of Agile Architecture

For those interested in the Agile Architecture Series, I’ll be presenting a webinar on this topic on September 10, 2008 at 1:00PM Eastern time. The webinar Principles of Agile Architecture: Intentional Architecture in Enterprise Class Systems is described below:

“As Agile development practices cross the chasm to the enterprise, many interesting debates continue about the role of systems architecture. As the continuous refactoring of emerging-only architectures becomes less practical as the system size and complexity grows, there is a natural desire to minimize unnecessary refactoring as well as to build systems from components that work in a like manner. This defines a role for “Intentional Architecture” in agile, enterprise-class systems development. In this Web seminar, learn about seven core principles that can be applied to architecting these large scale systems without sacrificing the principles or benefits of true software agility.”

This is part of a series sponsored by Rally Software, in collaboration with Sticky Minds and Better Software Magazine, designed specifically for the Enterprise Agilist. This webinar series is designed around specific topics of interest to the larger enterprise adopting agile methods. The description from the website is below:

“Join leading Agile experts, coaches, and authors—including Jean Tabaka, Dean Leffingwell, Stacia Broderick and Ryan Martens—as they paint a picture of successful Enterprise Agile adoption from a team’s first project to enterprise-wide transformation. During this Web seminar series, you will learn proven organizational transition approaches and best practices around Agile project and product management, Agile architecture and quality management. Development, management, and product teams will gain both an understanding of what it means to transition to Agile as well as specific best practices for their role in the organization.”

The series is now available for registration at: http://www.bulldogsolutions.net/SQE/SQE09242008/calendar.aspx?bdls=15887.

And it is FREE!

More on the Big Picture (3)- Product Owner vs. Product Manager

In the post Enterprise Agility–The Big Picture (3): Role of the Product Owner, I discussed the differences between the Product Owner and the Product Manager. In the past, I’ve discussed the fact that, at least in the larger enterprise, these are typically NOT the same individual and this is clear from the Big Picture Itself.

Big Picture 3- Product Owner and Product Manager

The reasons are straightforward. Simply, there are typically not enough Product Managers, nor are they necessarily inclined by training or interest to spend some/much/most of their time working directly with the product teams. The math itself is part of the problem. I’ve seen enterprises with up to 400 practitioners operating with as few as 4-6 Product Managers. It simply isn’t practical to assume that they will be able to support the Agile Manifesto Principle: Business people and developers must work together daily throughout the project. Even if they were so inclined, there simply aren’t enough hours in the day. And if they did spend most of their time with the teams, they become further removed from the customer the enterprise is required to serve. That can’t be good. (For more on this topic, check out the Product Owner/Product Manager category on this blog).

Moreover, the effective Product owner exhibits must exhibit a degree of technical competence in order to deserve and earn the respect of the team. As I noted previously,

“the foundation for effective decision making is driven by the technology of the implementation. In addition, continuous refactoring is integral to agile, and since the product owner owns the backlog, prioritizing refactors vs. value stories is a critical skill that requires a technical foundation.”

This blog post has raised some interesting comments which those interested in the topic may want to peruse at Enterprise Agility–The Big Picture (3): Role of the Product Owner.

One in-depth comment in particular, takes some issue with my recommendation (which I still support) that the Product Owner reports into the same line management as the team. To save you time, I’ve incorporated some of those comments from Malamo below:

“I want to expand a bit on the concept of trust. Agile forces a very interesting and powerful change with the Product Mgt and Development relationship. For years, it’s always been the same story – PM asks for too much and Development delivers too little. It’s been a conflict since the beginning of time and it’s always the other person’s fault. For the context of this post, I am going to declare one big assumption: that the product owner is properly skilled and has all the leadership characteristics listed above. The one I will add is that they should be a natural leader – more than anyone else, I believe the product is the person most responsible for keeping the team focused, motivated and engaged.

A good product owner changes the conflict mentioned above – completely. Their alliance is with the team and as a result, the conflict switches from PM vs. Development to PM vs. PO. The PO is the person responsible for ensuring product management is happy with the results. I have seen numerous situations where the PO has passionately defended the team’s performance. It’s a wonderful thing…

The product owner is also responsible for fostering and maintaining a solid relationship with the PM. If this doesn’t happen, it will create significant issues because ultimately the true stakeholders will start to question the ability for agile to deliver better results. The PO becomes the bridge between PM and Development. The PO understands with specificity what it takes to develop SW.

As a result of this shift in conflict, I personally believe all POs should report under the Product Management organization and NOT under development. Nothing changes from what Dean posts above – it’s all still critical. The PO should be collocated. The PO’s alliance is with the team. However, the dotted line (or direct line) relationship to the PM creates a needed balance of power.”

Enterprise Agility-The Big Picture (7): Vision and Release Backlog

Note: In the post Enterprise Agility: The Big Picture, we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In prior posts, we’ve discussed Teams, Iterations , the Agile Product Owner, Backlog, User Stories and the Iteration Backlog and the Release. In this post, we’ll discuss two closely related topics, the Vision and Release Backlog, [callout (7] below.

The Big Picture 7-Vision and Release Backlog

The Agile Vision

In agile, the traditional product requirements documents (PRD), system specification, software requirements specifications (SRS) and the like are typically eliminated. In their place, agile enterprises take a leaner approach better suited to the last-responsible-moment, delayed decision making and artifact-light development practices of the agile enterprise. However, since the PRD and SRS documents no longer exist to specify system behavior, communicating the Vision for the product or system to the development teams becomes even more critical. Doing so is management’s responsibility, because no matter how empowered and energized the agile teams may have become, it is management’s responsibility to set strategic direction. In our Big Picture model, management delivers the vision to the team face-to-face in the periodic Release Planning events.

The Vision answers big questions for the system, application or product under development, including

• Where are we headed with this thing?
• What problem does it solve?
• What features and benefits does it provide?
• For whom does it provide it?
• What performance, reliability, etc. does it deliver?
• What platforms, standards, applications, etc. will it support?

In addition, the enterprise is too large to assume that all the global development teams will natively understand the various constraints and “ilities”, such as reliability, accuracy, performance, quality, etc, that are imposed on the system, so these must be communicated as well. These nonfunctional (example: performance) and common requirements (example: accessibility standards, common technologies) must be communicated and documented in a central repository where all teams can readily access them. They are an adjunct to the Vision and are every bit as critical. Your practices may be lightweight and agile, but the system still has to work.

An Agile Vision Can Take Many Forms

I’ve seen agile teams take a variety of approaches to communicating the Vision. (Note: I covered many of these in more detail in Chapter 17 of SSA– Lean requirements at Scale: Vision, Roadmap and Just-in-Time Elaboration).These include:

• Draft Press Release
• The “Very Preliminary Data Sheet“
• Jim Highsmith’s “Product Vision Box“
• RUP-Like Vision Document. (One very agile team once told me, somewhat defensively, about why they still loved their RUP-derived Vision template: “We write to make sure management understands what we are about to build”)

Note: It doesn’t even have to be that structured

In the last release planning session I facilitated, there wasn’t an opportunity for the four product mangers to collaborate prior to the release planning session. Even if there were, it’s doubtful that they could have necessarily come up with a harmonized, force-ranked feature set anyway. (Question: which product manager wanted their number one feature to be placed fourth on the release backlog?). Instead, we allowed each product manager approximately 45 minutes on stage. Their format was regularized as follows: 10 minutes context, 10 minutes vision and 20 minutes describing feature priorities. Each presented a briefing deck, which had a list and description of the top 10 features from their perspective. This was then provided as a handout to the team. Based on this context, the teams then went into the more detailed planning session.

Clearly, this was not the forced-rank prioritization we like to see in agile and it was up to the teams to decide what to do about the fact that four product managers each had separate priorities. But software, like life, can be a little messy. Most importantly, this model seemed to work really well, in part because the Product Managers were part of the process and they had compete visibility into what the teams could and could not achieve in the time box.

The Prime Content of the Vision Document is a Set of Features

No matter the form, the prime content of the vision document is a prioritized set of features. In my earlier book Managing Software Requirements (perhaps attaining agile wisdom means we don’t have to throw out everything we learned prior), I described features as “services provided by the system that fulfill a user need”. Features are “Big Picture” system behaviors that can be described in a sentence or two and which are written so that customers can actually understand, debate and prioritize them. In Managing Software Requirements, I posited that by managing the level of abstraction, a system of arbitrary complexity (from the space shuttle to the spellchecker on this editor) can be described in a list of 25 or so features. That still works for agile teams describing a Vision today and features can still be used as the primary placeholder for user value.

(Note: in many agile environments, the concept of the feature is sometimes replaced by the word “epic”, meaning “bigger than an iteration”. However, I find the word feature to be in more common usage, more consistent with prior practices and more natural to the stakeholders and their clients. It also has a natural correlation to agile feature teams which do the work of implementation. However, there is no standard usage model. Also, I’ve reserved the word “epic” in the Big Picture for the next higher, portfolio level.)

Undelivered Features Fill the Release Backlog

We introduced the Release Backlog in a prior post. In a manner similar to the Iteration (story) backlog which contains stories, the Release Backlog contains the prioritized set of features that have not yet been implemented. Like stories, features can be scheduled (in a release) or unscheduled (waiting for future attention). They are prioritized and estimated. Estimates at this scale are coarse grained and imprecise. For example, if teams use ideal developer days for stories (1, 2, 4, 8 etc), then features could easily be 10-50x that number. Better, at this coarse level, features might be sized solely on a relative scale (this feature is 2X bigger than that other one we just completed) and this prevents any temptation to over-invest in feature elaboration and estimating. If and when a feature reaches a priority such that it hits a release planning boundary, it will be broken into user stories as those are the estimation and implementation “currency” for the agile teams.

Note: Cohn and others often refer to this backlog as the “Product Backlog”.

On Decoupling the Tooling and Management Practices of Agile Features and Stories

Because of their fewer numbers of objects and higher level of abstraction, tooling and managing a release backlog composed of features (Level 2 on the  Big Picture)  is not quite as demanding as managing the team’s iteration backlog with stories and tasks (think thousands of user stories over time). In some reasonably agile large enterprises, I have seen the Release Backlog continue to be maintained in a traditional requirements management tool such as Doors or RequisitePro. While this may trigger a gag reflex in some agile circles, in my experience, the decoupling of features from the story breakdown through release planning means that the processes and tooling for maintaining these backlogs need not be the same, at least initially.

Moreover, some agile transformation agents have discovered that attempting to undo all the requirements practices of large, traditional enterprises could actually slow down the effective agile transformation for the development teams. Dev team progress can be stalled as the company debates these Level 2&3, guttural change initiatives. Allowing the Product Managers and Business Analysts to continue to use their existing tooling can therefore provide a much needed degree of flexibility. The wholesale replacement of a company’s Level 2&3 practices for requirements discovery, prioritization and tooling is therefore not a battle that must be fought immediately.

Of course, in the end it will be easier to have all the agile requirements objects (epics, features, stories and tasks) in a single, scalable agile project management tool such as Rally Software or Version One. This new class of agile requirements/agile project management tooling also allows assignment of the agile requirements objects to iterations and releases and provides for the ability to track feature completion across releases and teams via rollup reporting mechanisms. This can be a critical element in helping large numbers of teams deliver complex systems in an agile manner. Also, these new tools are starting to provide support for market-based prioritization, collaborative, on-line collection of user requirements, integration with the help desk and more advanced portfolio management features, so it is likely they will eventually take up residence in the portfolio planning offices (Level 3) as well.

For now however, when it comes to change management challenges of the substantive enterprise, the agile transformation specialist is advised to pick his or her battles carefully!

Upcoming SSA Class in Boulder, Sep. 11, 2008

I’ll be delivering only two public courses of Scaling Software Agility: Best Practices for Large Enterprises this fall. Both will be held at Agile University in Boulder, Colorado. The first will be held September 11, 2008. You can see the abstract and register for the course here. While the course is based on the book, the material is constantly updated based on my ongoing experiences helping some of the world’s larger software companies transform themselves to become an agile enterprise.

In addition to the delivered content, the small group setting provides an opportunity for informal discussions of some of the real world challenges that attendees face. And since most attendees face significant challenges of agile adoption at scale, the shared learning environment facilitates experience sharing and practical problem solving in the enterprise context.

Hope to see you there.

Dean

Enterprise Agility-The Big Picture (6): The Release

Note: In the post Enterprise Agility: The Big Picture, we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In prior posts, we’ve discussed Teams, Iterations , the Agile Product Owner,  types of Backlog and User Stories and the Iteration Backlog. In this post, we’ll discuss the purpose of this entire endeavor, the Release. [callout (6)]

Big Picture 6-The Release

In the last few posts, we’ve described how agile teams cooperate to produce increments of software in iterations. While the goal of each iteration is to produce a “potentially shippable increment” of software, teams-especially larger-scale enterprise teams-find that this is a “tall order” and it may simply not be practical to achieve a shippable increment at each iteration boundary.

Not Every Iteration Can or Should be Released

There are a number of internal, technical reasons as to why an iteration may not be releasable:

• Accumulation of technical debt (needed refactors, test automation, accumulated defects). No iteration is perfect and even good teams often build up some technical debt
• Need to create specialty documentation and certifications that may not be practical or economical at each iteration
• Need to test the application across the “matrix of death”, i.e. compatibility checking for the hundreds of variants that are likely necessary to cover all platforms, devices, interfaces etc
• Need for more extensive performance, reliability or compliance testing,  for which the tools and resources may not be available or practical for each team

These are some of the reasons we have included a hardening iteration in our Big Picture model.

Moreover, there are good business reasons why not every increment, even if it were potentially shippable, should be shipped to the external customer:

• Potential for interference with a customer’s existing purchasing contracts and support agreements
• Potential for customer overhead for user training
• Potential for disrupting customer’s existing installation

This is the reason many agilists have recommended the Two Levels of Planning and Tracking as I described in Chapter 10 of SSA. (Note: XP explicitly calls out the Release and Iteration pattern. Scrum, per se, has only the single Sprint construct, but in practice many Scrum teams have evolved a model of releases following a Sprint-Sprint-Sprint-Release Sprint (hardening Sprint) pattern, so I don’t think there is much debate in the agile community about this basic two-level construct).

A Release can be Internal or External

In Chapter 18-Systems of Systems and the Agile Release Train and Chapter 21-Impact on Customers and Operations, I described a number of constructs and constraints that cause teams to focus more on a continuous stream of Internal releases (truly potentially shippable increments) than External Releases (General Availability- release to market, upgrade install base, push to distribution, etc.). This focuses the team on things they can control – including fixed internal release dates, quality criteria, etc.- rather than what they cannot control – timing of announcements and PR, impact on install base, fixed trade show deadlines and the like. This may best be illustrated by the “firewall” construct in the following graphic:

Firewall between Internal Releases and GA

So while the Big Picture labels it simply as a “Release”, it is with the understanding that the release can be either Internal or External and the general cadence is driven by a periodic and fixed Internal Release calendar.

Releases are Driven by a Theme and Feature Content

Be it internal or external, all releases follow the same basic pattern. They are driven by a Theme or Set of Objectives, a label that clearly communicates the business objective of the release along with a prioritized feature backlog. In the same manner that stories are sized to fit in iterations, features are sized to fit in a release. This helps assure that each release is driven by a set of objectives that deliver holistic (even if modest and incremental) customer value. At release planning time, features are decomposed into stories. With appropriate automated tooling support, the status of features can be tracked via accumulated story completion.

Planning the Release is a Seminal Event in the Enterprise

I’ve blogged extensively about the criticality of period, face-to-face Release Planning events and I won’t repeat it here. But in any case, every release has a kickoff Release Planning session that the enterprise uses to set the company context and to align the teams to common business objectives for the release. The teams, in turn, plan the release and provide an estimate of the feature set that will be delivered in the release. This is the team’s commitment to the release, and they endeavor to meet their commitment by satisfying the primary objectives of the release, even if it turns out that not every feature makes the deadline.

We’ll describe how the release content is driven by a Vision and a prioritized feature set (the Release Backlog) in the next post.

Principles of Agile Architecture – A Whitepaper

The whitepaper, Principles of Agile Architecture: Intentional Architecture in Enterprise-Class Systems, has now been posted on the Resource page. This whitepaper grew out of a series of blog posts wherein we were trying to define some governing agile principles that teams could use  to guide their agile, system-level architectural practices. My coauthors are Ryan Martens, Rally’s CTO and founder, and Mauricio Zamora, Executive Director from CSG Systems, both of whom are experienced enterprise agilists, architects and thought leaders in their own right.

Enterprise Agility–The Big Picture (5): User Stories and the Iteration Backlog

Note: In the post Enterprise Agility: The Big Picture, we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In prior posts, we’ve discussed Teams, Iterations , the Agile Product Owner and types of Backlog. To see the full series, select the “Big Picture” blog post category. In this post, we’ll describe User Stories and the Iteration Backlog [callout (5)].

Big Picture 5- User Stories and the Iteration Backlog

In the last post, we described three types of backlog that are common in the agile enterprise. These are:

The Iteration (or Story) Backlog – holding user stories and the like that are generally intended to be implemented by a component or feature team in their code baseline in the context of an iteration

The Release (or Feature) Backlog – holding features that deliver system-level value that typically affect multiple component teams and typically span iterations

The Portfolio (or Epic) Backlog – This backlog is a placeholder for capturing and discussing the larger scale initiatives that an enterprise has, or intends to have, underway. Epics often affect multiple component teams, multiple systems, and even multiple products. Epics typically span iterations, releases, and sometimes years!

We mentioned that, of these, the Iteration (story) Backlog is by far the “hardest working backlog” in the enterprise. That is because every team has one and they use it in managing their day to day activities. It is the central repository for all their work, both work in process (scheduled) and future work (unscheduled). It consists primarily of User Stories.

Where User Stories Come From

In an earlier post, we described the Product Owner’s critical role in developing, owning and managing the Iteration Backlog. In its most basic form, this involves identifying, prioritizing and elaborating user stories.

• Identifying user stories can be done at any point in time and it’s an oversimplification to say that only the product owner can do this. Indeed anyone on the team should be able to identify or suggest potential user stories, and most teams allow anyone to post a potential story in the backlog. So the Iteration Backlog is really a repository of ideas staged for future prioritization.
• Prioritizing user stories is indeed the responsibility of the Product Owner. While it is a collaborative effort – based on discussions with the team, product managers, customers and the Product Owners vision – no matter the source and the nature of the debate, in the end, the Product Owner makes the call. They are the authority for both what stories get put in the iteration and what the relative priorities are within an iteration. This latter point is important because unless a team is loafing, not every story will be accomplished as planned – no team’s estimates are perfect- and the metaphor of landing the iteration is really shorthand for “achieving the objectives of the iteration by implementing at least the highest priorities stories”.
• Elaborating User Stories. I’ve often noted that agile development is an extremely and inherently disciplined process. As an example, if a developer/tester pair takes responsibility for completing a story within an iteration boundary, it’s fair to expect that the story is reasonably well understood before they make such a commitment. Achieving this understanding by elaborating the story is a shared responsibility of the PO and the team, but the Product Owner must take the lead and see that stories are sufficiently elaborated to be properly estimated and implemented in a short iteration. Generally, this has to be done just-in-time, but in advance of the iteration boundary in which the story will be implemented.

(Note: for a deeper and more meaningful treatment of user stories, See Chapter 17 of Scaling Software Agility or Mike Cohn’s Agile Estimating and Planning).

User Stories Have State

Tracking user stories through their state lifecycle is how teams manage and assess the status of the iteration. In order to do so, many teams assign various states to user stories and use the states to track progress. For example, Rally Software’s agile project management tool default state model calls out six states:

Backlog – story has been identified, but not yet defined

Defined–a story is sufficiently elaborated so that it can be estimated and implemented

In-Progress –one or more team members has “taken responsibility” for the story and it is in the process of being coded and tested

Completed–coding and testing has been completed, but the story has not yet been accepted by the product owner

Accepted–story has been accepted by the product owner (usually with QA support) into the new baseline

Blocked–progress on the story is blocked for some reason. Blocks are often a key topic at the daily standup and if the story is of high priority, then resolving the block is of high priority too.

There’s More to the Iteration Backlog than User Stories

While the primary content of the Iteration Backlog should be user-value stories, in practice, most teams use the story metaphor to track all the work they need to accomplish in an iteration. In this way, they can estimate and keep track of everything they need to do to be complete the iteration. Thus, there are various “types” of stories that are all prioritized, estimated, and managed together. In addition to true User Stories (those that deliver direct value), other types of iteration “stories” include:

1. Systems Architecture–refactors and design “spikes” that evolve the product architecture to support new user stories
2. Development infrastructure–necessary work on the development environment, tooling and the like
3. Overhead–training, education, company meetings, etc. that require attendance and consume team capacity
4. Defects–defects (or suites of defects) that need to be fixed in the current iteration

On User Stories and Tasks

For more detailed tracking of stories, teams typically decompose stories into individual tasks that must be accomplished in order to complete the story. Indeed some agile training uses the task object as the basic estimating and tracking metaphor.

However, I much prefer the tracking focus be at the story level (see post the Simple Visible Iteration Kanban Board for a sample, visual Iteration backlog kanban board) and as it are better aligned to business rather than individual objectives. Tasks are a means to an end- basically a work breakdown structure that facilitates estimating and the taking of individual responsibilities- teams will use them naturally but they shouldn’t be the primary objective of the iteration. Note that when you add tasks to the hierarchy (epic, feature, story and task), you actually end up with four levels and that is sufficient for most enterprises, though 5, 6 and even 7 levels are not uncommon for larger systems and enterprises (is that the simplest thing that can possibly work?).

At implementation time, Stories Must Fit in a Single Iteration

Prior to the iteration boundary, not much thought need to be given to how “big” a story is. However, by definition, once bound to an iteration, stories must be sized and estimable so as to be “complete-able” in the course of the iteration. They are atomic- they must be completed in whole (good!) or carried forward to the next iteration (bad). Doing so typically involves splitting larger stories into iteration-size bites, while keeping the focus on value delivery. This learned behavior drives the extreme incremental delivery model that is integral to agile. Estimating and sizing user stories is certainly an art that takes time for the team to master. Mike Cohn’s User Stories Applied: For Agile Software Development is the best text I am aware of on this important topic.

Back to the Big Picture – Tracking the Iteration

With this model, the iteration is composed of all the stories and the state of the stories in aggregate represents the state of the current iteration. The iteration can be tracked in the large via a “burn down chart” (continuous measure of work remaining vs. time available in the iteration) and/or by assessing the individual states of story completion as a sample agile project management tooling screenshot shows below.

With this understanding of defining and tracking the Iteration, we can move on to the Release in the next post.

Revised Big Picture Graphic and Posts 1-4

I’ve had a number of off-line comments about the Agile Enterprise Big Picture graphic and the semantics I’ve used to describe it so I keep revising it to make it clearer, and hopefully, simpler. Here’s the latest:

Big Picture Updated with Backlog Constraint

The changes may appear to be subtle but they are material.

The series is here.