Agile Enterprise Requirements Information Model – Subset for Agile Project Teams

In a recent post, A Lean, Scalable Requirements Information Model for the Agile Enterprise, we introduced the requirements information model that underlies the requirements-related elements of the Big Picture series. While presenting the model in UML form may seem a bit of a geek fest, it’s the best way to describe the model in relatively unambiguous terms. In a follow on post, we described What’s Lean and Scalable about the Model and promised to later describe the model from the perspective of Agile Project Teams, Agile Programs and the Agile Enterprise. In this post, we’ll describe the model as it applies to Agile Project Teams.

But first, here’s a look at the requirements meta-model.

Agile Requirements Information Model

The Model Subset for Agile Teams

In the Big Picture series, the Agile Project Team (Agile Team for short) is highlighted below:

Agile Teams in the Big Picture

The critical role of the Agile Team itself within the agile enterprise has already been described in this post.

In accordance with the lean and scalable principles from the last post, it is a) the simplest thing that could possibly support the needs of all stakeholders and b)  particularly sensitive to the needs of the team members, in that the Agile Team is concerned with only a small subset of the requirements model. Moreover, that subset is quintessentially agile in that the artifacts described are consistent with most agile training (at least Scrum and XP) as well as common practice. (In other words, it isn’t mucked up with administrative overhead, manual traceability, reporting, detailed requirements-cram down, or any of the other ways enterprises can unnecessarily burden the teams!).

The elements of the model that the teams base their work on are extracted from the model and illustrated below:

Since most of the work (90-95%) involved with delivering enterprise software in an agile manner is done by these teams, using only these simple artifacts and relationships, it’s worth the effort to define these items and relationships more explicitly. We’ll do that in the following sections.

Backlog Item

First, we note that Story, pictured in the center, is a “kind of” Backlog Item. (The triangle indicator connecting Story to Backlog item is the UML generalization relationship, indicating that the thing the arrow points to is a generalization of the special case of the pointing thing. In other words, in this case, Story is a special kind of Backlog Item. This also implies that there are other kinds of backlog items as well).

The term Backlog was first introduced by Scrum where it was described as Product Backlog. In our enterprise model, Product can be a pretty nebulous thing as teams may be working on APIs, subsystems or whatever so there are multiple types of backlogs in the Big Picture. Therefore, our use of the term Backlog is more generalized than in Scrum. In the Big Picture, we described the particular backlog we are talking about here as the Iteration (Story) Backlog as can be seen below.

This Backlog is the one and only, definitive, source of work for the team. In this case, this particular Backlog (Iteration (Story) Backlog) holds all the work items (represented as Stories) that the team has identified. It is local to them. It is their repository for all identified work items and the contents are typically of little concern to others in the enterprise. This is a workflow “separation of concerns” which provides the team with a local work sandbox. They manage it, tool it and put things in and out of it as it suits their needs in meeting their iteration objectives. If “a thing to do” is in there, it is likely to happen. If it isn’t, then it won’t.

Note: Within the team, maintenance and prioritization of the Backlog is the responsibility of the Product Owner, who in turn is a full (or maybe half) time resident of the Agile Team. The role of the Product Owner in developing, prioritizing and maintaining the backlog is outside the scope of this post. For more on that role, see posts in the Product Owner and Big Picture 3: Role of the Product Owner category.

Story

The team’s Iteration (Story) Backlog consists of all the work items the team has identified. In the meta-model, we call these work items Stories because that’s what most agile teams call them. (Strictly speaking, “work items” is probably a better term, but we aren’t trying to fight agile gravity with this meta-model!) So for our purposes, we’ll define Story simply as

A Story is a work item contained in the team’s iteration backlog.

User Stories

While that definition is simple, it belies the underlying strength of agile in that it is the User Story that delivers all the value to the user in the agile model. Indeed, the user story is inseparable from agile’s goal of insane focus on value delivery. In the meta-model so far, the User Story is not explicitly called out, but rather is implied (as a shorthand) by the Story class.

To make the User Story explicit, we need to extend the simple model a little as seen below:

With this small addition, we now see that the backlog is composed (primarily) of User Stories and anything that isn’t a User Story is an Other
Work Item. Other Work Items include things like refactors, defects, support and maintenance, infrastructure and team overhead. We’ll discuss the rationale for calling out these Other Work Items specifically later, but for now they appear primarily to help the team’s better estimate the time they actually have to deliver User Stories.

I’ve described the User Story briefly in the post Enterprise Agility–The Big Picture (5): User Stories and the Iteration Backlog, but since this key construct does most of the work in the agile model, it’s worth investing some more words here.

User Story Basics

User Stories are the agile replacement for what has been traditionally expressed as software requirements statements or Use Cases and they are the true workhorse of agile development. Developed initially within the constructs of Extreme Programming, they are now endemic to agile development in general and are taught in most Certified Scrum classes as well. In our model, we’ll define a User Story as:

A User Story is a brief statement of intent which describes something the system needs to do for the user.

As commonly taught (at least in the last few CSM Scrum courses I have seen), the user story takes a canonical (standard) form of:

As a <role> I can <activity> so that <business value>

(Note: While searching for the origin of this nifty form, I received the following email from Mike Cohn ” It started with a team at Connextra in London in 2003 and was mentioned at XP2003. I started using it then and wrote about it in my user stories book that came out in 2004″.  Thanks, Mike.)

In this form, User Stories can be seen to incorporate both elements of the problem space (the business value delivered) and the solution space (the activity that the user does with the system). For example: “As a Salesperson, (<role>) I want to paginate my leads when I send mass emails (<what I do with the system>) so that I can quickly select a large number of leads (<business value I receive)”.

LOTS has been written on applying User Stories in agile development. The good news is that it is now so common that I don’t have to elaborate further here, as that is not the intent of this post nor the meta-model. For that, I refer you to the many books on Extreme Programming and User Stories.

Acceptance Test

Ron Jeffries, one of the creators of XP, described what has become my favorite way to think about User Stories. He used the neat alliteration, Card, Conversation and Confirmation to describe the three elements of a User Story. Where:

• Card represents the 2-3 sentences used to describe the intent of the story.

Note: In XP and agile, stories are often written manually on physical index cards. More typically in the enterprise, the “card” elements is captured as text and attachments in agile project management tooling, but teams often still use cards for early planning and brainstorming.

• Conversation represents fleshing out the details of the intent of the card in a conversation with the customer or product owner. In other words, the card also represents a “promise for a conversation” about the intent.
• Confirmation represents the Acceptance tests which will confirm the story has been implemented correctly.

With this simple alliteration and XPs zealousness for “all code is tested code” we have an object lesson in how quality is achieved during, rather than after, actual code development.

Relative to our model, however, we have represented the Acceptance Test as an artifact distinct from the (User) Story itself as seen below:

There are many reasons why we did this, which I won’t belabor here, but in any case the model is explicit in its insistence on the relationship between the Story and Acceptance Test as follows:

• In the 1 to many (1..) relationship – every story has one (or more) Acceptance Tests.
• Done when passes. A story cannot be considered complete until it has passed the Acceptance Test(s).

In this manner, the model explicitly supports the agile “all code is tested code” quality mantra. Every User Story has at least one Acceptance Test which must be passed before the Story can be considered to be done. Very direct. Very explicit. Very cool.

Tasks

And finally, we find our way to the Task model element:

As you can now infer from the graphic, stories are implemented by Tasks. Tasks are the lowest granularity in the model and represent specific work items to be performed by specific team members to accomplish the story. In our context the definition of a Task is:

A Task is a small work item, either stand alone, or necessary for the completion of a Story

Tasks have an owner (the person who is going to do the task) and are estimated in hours. The burn down of task hours represents one form of iteration status. As implied by the 1-to-many relationship expressed in the model, there is often more than one task necessary to deliver even a small story. For example:

In other cases, it’s common to see a mini-lifecycle coded into the Tasks of a Story. For example:

Story 51 – Select photo for upload
Task 51.1 – Define acceptance test     Juha, Dean, Bill
Task 51.2 – Code story                            Juha
Task 51.3 – Code acceptance test       Bill
Task 51.4 – Get it to pass                        Juha and Bill
Task 51.5 – Document in user help    Cindy

In most cases, Tasks are “children” to their associated Story. However, for simplicity, our model also supports stand alone Tasks. With this construct, a team need not create a Story simply to parent such an item as “order 8 Gig more memory for the new file server”.

Well, that’s it for our description of the meta-model as intended to be used by Agile Project Teams. In an upcoming post, we’ll look a little higher in both the Agile Requirements Information Model and the subject organization and describe how to apply the model to Agile Programs.

Still More on Agile Product Owner vs. Agile Product Manager- A New Whitepaper

The Agile Product Owner/Product Manager category on this blog continues to get a lot of interest from readers. In the past, I’ve highlighted my opinions on the differing nature of the Agile Product Owner and Agile Product Manager roles at enterprise scale. This has stirred some interest and a little controversy and a number of others are opining on the topic as well.

I just noticed a new whitepaper Living in an Agile World: the Strategic Role of Product Management when Development goes Agile from Luke Hohmann (Enthiosys) and Steve Johnson (Pragmatic Marketing). I think this whitepaper does the best job to date of describing the yin and yang of APO (Agile Product Owners) and APM’s (Agile Product Managers). Moreover, both authors are experts in the field of software product management so they speak with knowledge and credibility from the PM role. I doubt that it’s particularly import to the authors, but I agree with most of the premises and conclusions of this article.

Here’s a grabber:

When you adopt agile development methods, you encounter new concepts, new artifacts, new planning methods, and new roles and relationships. It seems that agile teams do everything in a new way. And, as you attempt to integrate agile into your existing systems, you’ll also attempt to map these new concepts to your old, familiar concepts. And a product manager is now called a Product Owner… right?

Wrong!

Companies adopting agile methods know that product teams need a voice representing the customer. Developers need personas, market problems, and most of all, a prioritized list of requirements. Agile methods advocate a role called product owner to support the product team with customer and market information. Since the closest equivalent to product owner in most companies is the product manager, it seems natural to equate the two. But product owner and product manager are not the same. In fact, a product owner’s responsibilities are just a small subset of product management.

It continues to be my experience that “equating the two” does not work well at all at enterprise scale and a more refined set of roles must emerge. But I refer you to the article itself for a more meaningful discussion. The article is already receiving both supportive and (so far) mildly critical comments from readers, so be sure and check the comments section as well. Of course, comments are always welcome here, too.

What’s Lean and Scalable about the Requirements Information Model for the Agile Enterprise?

In the last post, we introduced a Lean, Scalable Requirements Information Model for the Agile Enterprise. This model, based on the Big Picture, is being developed in collaboration with Juha-Markus Aalto, Director of Operational Development for Nokia S60 Software.

This model serves as the requirements-related semantic underpinnings for the Big Picture of Enterprise Agility and I promised to describe the model further in upcoming posts. Before I dive deeper into the model, though, it’s useful to spend a few moments describing why the model is “Lean” and “Scalable” as advertised.

What Makes a Software Process Lean?

Lean Software is a philosophy of approach that is characterized by:

• Eliminating all non-value added processes, steps, delays and artifacts
• Minimizing time to market; maximizing value delivery

Lean Software development has been characterized by Mary and Tom Poppendieck in a number of books. In their most recent book, the principles of lean software are described along with the myths that prevent lean thinking. Here’s a view from Implementing Lean Software Development: From Concept to Cash.

• Principle 1: Eliminate Waste- Myth: Early Specification Reduces Waste
• Principle 2: Build Quality in- Myth: The Job of testing is to find defects
• Principle 3: Create Knowledge – Myth: Predictions Create Predictability
• Principle 4: Defer Commitment – Myth: Planning is Commitment
• Principle 5: Deliver Fast – Myth: Haste Makes Waste
• Principle 6: Respect People – Myth: There is one best way
• Principle 7: Optimize the Whole – Myth: Optimize by decomposition

What Makes a Software Process Scalable?

Compared to lean, I know of no such convenient published definition. However, based on my experience in prior (RUP) and current (agile and scaling agile) software process initiatives, I’d think that an agile software process is scalable when:

• Principle 1: It’s the simplest thing that could possibly support the needs of all stakeholders, including the team, program and enterprise level. (Scaling to support hundreds and thousands of practitioners).
• Principle 2: It supports agile’s insane focus on customer value delivery at all levels.
• Principle 3: It is particularly sensitive to the needs of the team members. (They write all the software; maximizing their velocity maximizes the productivity of the whole).
• Principle 4: It is lightweight and adds no unnecessary overhead when applied to the enterprise. (Doesn’t slow development as it scales).
• Principle 5: It is configurable and can be customized where appropriate. (Every enterprise is unique).
• Principle 6: It drives and supports best practices of building a quality software foundation. (Without inherent quality, nothing scales).

What’s Lean and Scalable About this Model?

With that background, we can turn to attempting to defend our “labeling claims” of a Lean and Scalable information model. Of course, it’s the application of the model, rather than the model itself, that makes it lean and scalable. Therefore, in the table below we’ll describe how the model is intended to be used in the context of the Big Picture. We’ll also indicate the principles of Lean (L) and Scalable (S) that the model supports. Here we go:

 

Organization	Model elements	What’s Lean?	What’s Scalable
Agile Team	Story Task Acceptance test Single (iteration) backlog	All that is needed for the team. (L1) Just in time story elaboration (L1, L4) User Story construct (L7) Simple backlog. (L1, L4) Acceptance tests assure quality foundation. (L2)	No limit to number of define/ build/ test) agile teams. (S1, S4) Extremely efficient for the team. (S3) Hooks to program. (S1) Acceptance tests assure quality foundation. (S6)
Agile Program	Feature Feature (release) backlog Nonfunctional requirements (constraints)	Features described at high level of abstraction. (L1,L4) Does not over constrain the solution space. (L1) Single, localized backlog. Introduce non-functional requirements only as and when necessary. (L1)	Programs driven by Features, not requirements. (S2) Hooks to stories below.(S1, S3)
Agile Enterprise	Epic Strategic Product Themes Portfolio backlog	Lightweight, hi level descriptions of the problem not solution space. (L2, L3) Single (portfolio backlog) (L1) Modifiable (L5)	Epics hook to Features. (S1,S2, S6) Traceable and supportive of agile epic-based estimating and planning. (S1, S5)

Hopefully, we haven’t overly-belabored this simple point. But if we are to depend on this model in the enterprise, we must be confident in the fact that is both Lean and Scalable!

A Lean, Scalable Requirements Information Model for the Agile Enterprise

For those of you who have been following the Big Picture series, you may have noted that one element of the picture is the “requirements information model” that flows down the right side. This element of the model describes agile terms for expressing system behaviors at various levels of abstraction. (See below).

The Big Picture, Highlighting the Requirements Model

In summary form, the model implies the following requirements-related elements and relationships as illustrated below.

For Big Picture purposes, this is an adequate explanation which helps the enterprise to understand the new labels for “requirement like things” in enterprise agile methods and the way teams use these new things to describe intended system behavior.

However, while the Big Picture supports “hand waving” descriptions of this complex problem, it does not define the exact nature of each element nor the critical relationships between them. For this purpose, my colleague, Juha-Markus Aalto, Director of Operational Development for Nokia S60 Software Unit, and I have been working on a semantic meta-model for agile enterprise requirements information management. The model also serves as the technical underpinnings for the requirements elements of the Big Picture. The need for the more definitive model is driven by two factors:

1. The need for clear definition of each artifact type, so teams can be coached as how to use the appropriate element (ex: feature, user story, task) at the appropriate time
2. The need to make explicit the relationships amongst the elements of the model (user stories, acceptance tests, features, etc.) to support tooling automation, which will provide enterprise- level support for status reporting and portfolio estimating and planning.

Last week, Juha-Markus presented this model at Object Days at the Tampere, Finland, University of Technology. His full presentation can be found here.

Lessons Learned at (truly) Enterprise Scale Agile Adoption

The purpose of Juha-Markus presentation was not the model itself; rather it was to describe how this model supports the enterprise level agile practices being deployed in the development of Nokia’s S60 Software Platform and Applications. Nokia’s S60 Platform is a software platform for mobile phones that is currently amongst the leading smart phone platforms in the world. The Platform is built by an extensive network of software teams who are applying Scrum to increase the productivity and quality of their local efforts. S60 in turn, is built on top of the Symbian OS, further coupling the efforts of the Nokia development teams to the many software practitioners of this critical supplier. Nokia S60 teams’ experiences in scaling agile across a large number of intensively collaborative software teams represents some of the most advanced agile thinking and practices, at the greatest scale, of any that I am aware of.

These experiences serve as an excellent object lesson for others in (really) Scaling Software Agility. For more on this topic, I refer you to Juha-Markus’ presentation which will provide a fuller understanding of some of the challenges the S60 teams are addressing with their new agile model.

The Requirements Information Model

The purpose of this post, however, is to introduce the requirements information model that we have been collaborating on. The most basic form of this model (more in future posts), as represented in UML form, is shown below.

A Lean, Scalable Requirements Information Model

When compared to the Big Picture, it’s obvious that the Big Picture graphic is an oversimplification of the reality of large scale, agile system development. While the elements of Epic, Feature, Story, Task and Backlog are described in the Big Picture and elaborated in supportive posts, the elements of Strategic Product Theme and Acceptance Test have not. These are described below:

• Strategic Product Theme – this information model adds this, new and highest level element to the information hierarchy. Themes provide a well-defined set of core value propositions that in turn, drive portfolio investment. They define the key strengths of the enterprise, deliver on strategic initiatives and solutions, and drive long-term competitiveness. Themes, in turn cause the creation of Epics, Features, and Stories that deliver on the themes value proposition. (In the Big Picture, Themes can be considered to be the various “slices” in the portfolio management pie chart at the upper left)
• Acceptance Test – moving all the way to the bottom of the stack, the model also highlights the critical role of the acceptance test in agile development. In turn, this encapsulates two key agile quality principles: 1) there is always at least one acceptance test for every story (in agile, all code is tested code) and b) a Story cannot be considered “done” until it passes the test.

Lastly, the Non-Functional Requirement is a primary element in this model, which reflects the role these special requirements play in the development of enterprise class systems. (These were called out as auxiliary elements of the “Vision + in the Big Picture.)

Summary

That summarizes the primary elements of the lean, scalable, requirements information model. To the new or smaller team Agilist, this model may seem relatively complex and perhaps not very agile, or at least not “the simplest thing that can possibly work”. However, experience has shown that thinking about the solution space in these varying levels of abstraction is in fact, a simplification of the problem at enterprise scale and encourages agile thinking throughout the organizational hierarchy. I’ll describe why in an upcoming post.

Enterprise Agility – The Big Picture (14b): More on Agile Portfolio Management and the Legacy Mindset

Note: This is part of a continuing series (see the Big Picture Category on this blog) wherein we introduced an overview graphic intended to capture the essence of enterprise agility in a single slide. In a series of continuing posts, we have been working our way from the bottom (stories, iterations and the like) to the top where the enterprise vision and portfolio management resides. This post is one in a miniseries describing the last big icon on the upper left – Agile Portfolio Management.

The Big Picture - Agile Portfolio Management

In the last Big Picture Post, we introduced the portfolio, along with the challenges that agile teams are likely to find as the impact of their new and more productive models wends its way to the Project Management Office (PMO), which in some cases can be the “mother ship of potential agile impediments.” We also introduced the excellent DTE energy case study: Establishing an Agile Portfolio to Align IT Investments with Business Needs. This case study is an example of how an agile enterprise can first recognize, and then successfully address the significant changes necessary to allow the emergence of true enterprise agility benefits.

Since that post, I’ve been thinking that I would expound on the legacy mindsets that called out in the case study. But I’ve decided I can’t. They are just too well done and so symptomatic that I decided just to repeat them here. However, the accompanying descriptions and paraphrasing are my own. These are based on my own experience from two perspectives; 1) then, as an executive, managing portfolios and carrying some elements of these mindsets, and 2) now, as an agile executive attempting to change those very same mindsets!

The legacy mindsets called out in the DTC case study are:

• Widget engineering – The belief that software development is a repetitive, controlled and manufacturing-like business, rather than Research and Development. “Draw it up and build it like you drew it,” goes the thinking.
• Order Taker Mentality – Also known as “build what we tell you to build”. Founded on the belief that the customer is always right, seemingly all-knowing and under the assumption that they actually know what their requirements really are. The teams should just build it.
• Maximize utilization- The belief that if all resources aren’t fully utilized on paper then they won’t be fully utilized in practice. “They’ll just be idlers, wasting their time, waiting for their next assignment.” goes the thinking. “Fully assign them or lose them”.
• Control through data – The belief that
  with the right kind of data – earned value metrics, design reviews, requirements and test plan inspections – we can tell where we are on the project. And then if we still can’t tell where we are, we’ll just ask for more detailed data until we can.
• And we can plan out a full year of projects- Conveniently disregarding our last 20 years or so of experience in failing to predict projects a year in advance, we assume it’s a failure of our planning, not a failure of the basic paradigm. “If we only planned in more detail, we could really get it right this year.”
• Get it Done – The belief that our best case plans can be reality if we only try hard enough.
  Also known as: “That was the plan we agreed to, now execute it.” And “when the going gets tough, the tough get going”. And lastly, the infamous “we know it’s impossible, that’s why we hired you for this job.”

Perhaps it’s obvious from my elaborations that I tend to get somewhat passionate about this topic. Perhaps it’s because my guilt shows through. Surely, I have been on the other side of this fence before iterative and incremental, and now agile, development methods proved their extraordinary worth. So, I know as well as any that if these legacy mindsets are not addressed, they will kill the agile enterprise initiative as surely as a fixed content/fixed schedule 60 hr/week death march.

Here’s why. The table below shows the manifestation of these problems in practice (as described by DTE along with a few of my own) and the real problems they create for the nascent agile teams.

Mindset	Manifestation	Problems
“widget engineering”	Fixed schedule, functionality planning Big Up Front Design/Analysis (BUFD)	Long range plans Resources fully committed up to a year in advance Analysis paralysis
“order taker mentality”	Build what we tell you to build We are the boss of you	False agreements. No buy in. Misses innovation contribution from teams Failure to meet expectations Mistrust No empowerment, lower motivation
“Maximize utilization”	Resources committed long range 100% allocation before “what if” Key resources assigned to multiple projects	No time to think or innovate Dedicate resources to task or lose resources Thrashing – lost productivity of most valuable resources Exhaustion, burnout
“we can plan out a full year of projects”	Detailed wbs, earned value metrics, fully loaded Gantt charts	Plans are immediately obsolete, but not treated that way Earned value metrics don’t reflect actual progress
“Control through data”	Fine grain reporting and overhead Milestones that are not reflective of process or progress	Reporting overhead slows actual development Annoys and de-motivates the teams
“Get it done”	Belief that best case plans must succeed	Deferred recognition of plan vs. actual Late discovery and re-negotiation Loss of credibility, mistrust

You’ll probably recognize many of these mindsets, manifestations and the problems they create for the development teams. Many enterprises do. In one recent presentation I made highlighting the DTE case study, one PMO executive said “just delete DTE and put in [our company name here] and there you have it”!

Well enough crabbing and I’m sorry if I offended anybody. I just thought it important to walk through this pain field one last time, making sure we understand the problem so that the imperative for a solution is really, really apparent.

In the next post in this series, I promise to move to a more productive and proactive set of ideas to address these problems. One giant step further to actually becoming the agile enterprise.