Agile Requirements Information Model: More on Non-Functional Requirements (System Qualities?)

Note: This is part of a series on a Lean and Scalable Agile Enterprise Requirements Information Model. Refer to that series for additional context.

In an earlier post, Agile Requirements Information Model (5) – For Agile Programs (b): Nonfunctional Requirements, we introduced Non-Functional Requirements as “Backlog Constraints” and highlighted the important role these types of requirements play in building enterprise-class systems.

They appear in the model here:

Agile Requirements Information Model - Basic Form

Agile Requirements Information Model - Basic Form

as well as in the Big Picture:

Big Picture Graphic (Updated Jan 2009)

Big Picture Graphic (Updated Jan 2009)

One set of comments from Ryan Shriver (http://theagileengineer.com) triggered a deeper look. Ryan commented:
“I like how you related Non-Functional requirements to backlog items, I think the concept of constraints make sense. May I propose a way to further investigate the components of Non-Functional requirements?”

Here is Ryan’ specific post on the topic:

http://www.theagileengineer.com/public/Home/Entries/2009/1/19_Modeling_Non-Functional_Requirements.html

In this post, Ryan suggests that a far better label for Non-Functional Requirements is “System Qualities” and that resonates with me. He goes on to note:

  1. A Non-Functional requirement is expressed as a System Quality. It’s the center of this tiny universe. I prefer the term Qualities over Non-Functional requirements because it says what they are, not what they’re not (if that makes sense).

  2. A Quality can have a Scale, which captures “what” is being measured (the units of measure)

  3. A Quality can have a Meter, which captures “how” the measurement is made (method of measurement)

  4. A Quality can have many Constraints (failure levels), so long as each has a unique label (i.e. Tolerable, Product Failure and SLA penalty could be examples)

  5. A Quality can have many Targets (goal levels), so long as each has a unique label (i.e. Budget Goal, Stretch Goal and Wish could be examples)

  6. 6A Quality can have many Benchmarks (previous results), so long as each has a unique label (such as date taken, conditions when results recorded as examples)”

I’m giving some serious thought to just renaming the NFR box in the model to be” System Qualities” as that communicates more directly what these things are and why we care. Of course, I would still describe them as Non-Functional Requirements and Design Constraints, because that’s the way so many people (like me, for example, in Managing Software Requirements: First and Second Editions)  have always described them in the past. What do the readers think of this semantic change?

Ryan also pointed me to a long thread on Mike Cohn’s blog which has a deeper discussion, mostly around whether these types of requirements can be expressed in measurable and objective form, and whether they should be expressed using the canonical user story form (as a…).

Tom Gilb, who is an authority on system requirements practices, also opines in comments to the post. You can see Mike’s post and a host of those comments here.

Thanks for the tips Ryan!

Updated Big Picture Graphic

The Big Picture has evolved quite a bit over course of the Big Picture Series. Here’s the latest tip revision of the graphic. Refer to the series itself for elaborations.

5/09 Update: Added Portfolio Managers.

Changed “Release (PSI)” to “Release Increment” to better reflect the intent. No one understood what a PSI (Potentially Shippable Increment) was, and it doesn’t exactly roll off the tongue.). Also, some are using the word “Train” (as in Agile Release Train) for this milestone, because the connotation of the word “Release” is overloaded and can cause unnecessary palpitations. (as in “You’ve got to be kidding, we can’t release a product every bla bla bla”). I’ve been thinking about the term “Releasable Increment” for this event also.

Big Picture (Updated 5/09)

Big Picture (Updated 5/09)

Updated Agile Requirements Information Model

For those of you following the Agile Requirements series, you will have already noted that the model has been changed and extended over the course of the series. For reference purposes, the updated “basic” information model is below:

Basic Agile Requirements Information Model (Updated Jan 22, 2009)

Basic Agile Requirements Information Model (Updated Jan 22, 2009)

As the series has progressed, the model has been further elaborated with additional subtypes and a few new model elements. The current, fully elaborated, “extended” model is here.

Extended Agile Requirements Information Model (Updated Jan 22, 2009)

Extended Agile Requirements Information Model (Updated Jan 22, 2009)

You may also note the new, optional use case model element. I’ll describe the application of use cases in the context of this agile requirements model in an upcoming post.

What, No Use Cases in the Lean and Scalable Agile Requirements Model?

Just the other day, I received the comments below (read the excerpt below the dashed lines) from Jennifer Fawcett, of www.agileproductpowner.com. Jennifer worked with me at Rational and Proquo (as an extremely capable agile product owner) so we have a long history together.

In her comments, Jennifer asked why there was no mention of use cases in the requirements information model.  Perhaps I haven’t introduced them here yet because:
a) I’ve primarily used use cases myself in the context of RUP-based development, or
b) I’ve seen use cases specifically deprecated in a CSM course. I recently saw a slide that basically said “don’t use use cases, because they are too detailed, exist indefinitely and users don’t understand them”!, or
c) I don’t see use cases applied very frequently in agile teams myself (probably because they are not part of XP and Scrum which are the dominant agile methods), or
d) Given my extensive RUP background, I might have a pro-use case bias. (The book I wrote prior to SSA is entitled Managing Software Requirements: A Use Case Approach.)

However, when building systems of scale, there is no tool quite so powerful as a use case for exploring the interactions amongst users, the systems, and the subsystems of the solution. Moreover, the use-case technique is the best way I know to help identify all the alternate scenarios that trip us so often when it comes to system level quality and readiness. (Example: When I’m uploading a big photo from my phone and the battery is low and there is an incoming call, what is the graceful failure path?) This is especially the case when you are building complex hardware and software systems, where system-spanning Epics, Features and Stories bob in and out of hardware and software like a surfacing porpoise.

I’ve been thinking about adding use cases to the requirements information model for some time. In thinking about it, I am also reminded of the fact that use cases have been well applied in other agile models, including: Alistair Cockburn (Writing Effective Use Cases (Agile Software Series)); in the Open Unified Process, the Eclipse Foundation/open source/agile RUP variant, which is even Use-Case Driven; and in the context of the Dynamic System Development Method (DSDM). So I have come to the conclusion that not including them in the agile model was myopic of me and potentially deprived the agile system builder (Ok, well maybe just the reader of my blog…) with guidance towards one of the best available system analysis constructs.

Therefore, I’m going to introduce use cases as an optional element of the agile requirements information model in an upcoming post.

In the meantime, since it was Jennifer’s comments that triggered this reconsideration, and since she took the time to describe the benefits of use cases in agile development and to provide some lightweight agile guidance, I’m including her comments verbatim below: (thanks Jennifer!).

=============================

“Hi Dean, I noticed that your meta-model is void of Use Cases! That got me thinking “Hmmm…I’m working in an agile organization as an agile product owner, but I still write Use Cases when I feel the need to! I must be still living my former dark, SDLC (pre agile) life…!”

But, maybe not!

How about exploring how Use Cases can help as a powerful tool in the agile requirements meta-model? Below is some experience I’ve had with agile use cases.

First and foremost, I employ use cases as my primary elaboration technique for the simple agile attitude of constant communication. Our team includes software, firmware, and hardware developers. At our mature stage of the game, all stories inevitably touch all developers, across all disciplines. While we would like to think we all agree, inevitably, we don’t. We are constantly having discussions like: “Should the logic be built into the end device, or the software platform”. “Should the data be pushed or pulled?” “What’s the persistence model, and how long/where should the data persist?” “What are the hardware memory requirements for this story?” Use cases help flush out answers to these questions, and help bring a starting “agreement” among developers as to the initial direction of implementation. That said, we are agile, and it’s all subject to change!

Here’s how we are using use cases in our agile process, and why they work for us.

Why use cases work in our agile process
1. Initial Agreement to Final Agreement
Many Epics or Stories start with a picture or an idea. It’s usually a UI that Product Management has sketched, which defines what the user sees, and infers how the user shall interact with the system. This is often (but not always) accompanied with a story. Defining how the system behaves as a result of the user interaction is the challenge. Use cases provide the perfect forum to analyze and define this initial to final user to system interaction.

2. Statement of Record
Like I said, developers do not always agree (nor do humans for that matter). Use cases provide that simple statement of record for when our brains fail to remember how we thought through the main success scenarios and alternate success scenarios, as well as variations and open issues.

3. Developers Love Them!
A story is not DONE until it’s tested. Most “good” developers write their unit test before they even start laying down code. (some write them after). Use cases give developers a framework for their unit tests before they get going. That’s not the only reason they love them. Developers are often anal (forgive me, I love developers for this reason). Use cases give developers the opportunity to capture details that they feel are important to the story. Careful though…don’t let them get caught in “analysis-paralysis”. As scrummaster, allow discussions, and come to initial agreement honoring the fact that we don’t know what we don’t know, and move on.

4. Business groups gain early understanding
My philosophy is: “Publish everything, and have no secrets.” Our Wiki is golden. By publishing the use cases, all areas of the business gain early access and an understanding of how the system will behave. Stories can provide this same benefit, however, use cases give other parts of your business a “more than they need to know”, aspect, thus covering the proverbial ass and further opening up conversations. Also, this helps get documentation into the agile process early and often.

5. The perfect framework for user acceptance testing
Many conversations happen during the agile development lifecycle. If use cases capture the initial to final agreement per sprint and/or release, what a perfect framework for user acceptance tests (Agile Product Owners LOVE THIS!)

Best Practices for Agile Use Cases
Interview Stakeholders
Before you jot down a word, interview Product Management and the development team leaders. Knowing that the team will most likely not agree, this helps in gaining an initial understanding of what the open issues and what discussion points should take place.

Make them lightweight
No one wants to read use cases that are dozens of pages long. You’ll gain better readership by keeping them short. No longer than two pages should be sufficient to get going. (two may even be too long!). Capture just enough information to get the heads to nod and move on. Make sure you communicate in a language that is common to all stakeholders.

Keep detailed UI design out of it
I struggle with this one, as I am a fond believer in UI detail, but that’s not the goal. Having an exhibit or two with screen shots is fine. Making your use case into a detailed design document is not.

Present/collaborate with your team
Use the daily meet-after opportunity to elaborate and gain feedback. This daily opportunity allows for quick turnaround. Depending on the story, developers often need time to noodle on a use case. Asking them daily until confidence is built helps hedge your bets.

Design them for change.
A simple intro page with a change traceability table will most likely serve you in the future. (post mortems, refactors, etc.)

Write your Use Cases Just-In-Time
After interviewing stakeholders, most Product Owners can kick out a use case in an hour or so. This should happen JIT prior to Iteration Planning. Having a light-weight use case ready at the iteration boundary provides the team a clear framework for tasking and estimating.

All this said, stories are part of our daily agile process too! Not all stories are elaborated with use cases. Some are as clear as a sunny Colorado day. Use cases are brought into the agile process primarily when we want to provide that more detailed analysis of user/system interaction, and gain agreement across the system and developers. I simply employ use cases as another tool in my agile toolbox.

Cheers, and thanks for listening!”

–Jennifer Fawcett

Agile Enterprise Requirements Information Model (6b) – Subset for Portfolio Management: Epics

Note: This is the next in a series of posts describing, in more rigorous terms, A Lean, Scalable Requirements Information Model for the Agile Enterprise (AE RIM) that underlies the Big Picture Series. In this series, I’ve been collaborating with Juha-Markus Aalto, Director of Operational Development for Nokia S60 Software Unit, where a variant of the model is being developed and applied on a very large scale development project. In the last post, the Agile Requirements Information Model for Agile Portfolio: Strategic Product Themes, we initiated our discussion of the two model elements at the Portfolio Level – Strategic Product Themes and Epics, with a discussion of Strategic Product Themes. In this post, we’ll conclude this level with a discussion of Epics.

But first, for context, here’s an updated version of the Big picture with the Portfolio level highlighted:

capture-big-picture-portfolio1Agile Enterprise Big Picture with Portfolio Level Highlighted

And here’s a look at the basic form of the updated requirements information model, with the portfolio level requirements items highlighted.

Basic Requirements Information Model with Portfolio Elements Hihlighted

Basic Requirements Information Model with Portfolio Elements Highlighted

Epics

Epics represent the highest level expression of a customer need as this (more obviously hierarchical) graphic shows.

capture-next

Derived from the portfolio of Strategic Product Themes, Epics are development initiatives that are intended to deliver the value of the theme and are identified, prioritized, estimated and maintained in the Epic Backlog. Prior to release planning, Epics are decomposed into specific Features, which in turn, drive Release Planning in the Big Picture.

Release Planning in the Big Picture

Release Planning in the Big Picture

Epics may be expressed in bullet form, as a sentence or two, in video, prototype, or indeed in any form of expression suitable to express the intent of the product initiative. (I was in recently in a meeting where an (otherwise) clever agilist, pushing back on what he considered to be delaying and excessive epic-level documentation requirements, told his PMO management “you can even tell us in interpretive dance – we’ll take it from there”. Note, however, that this is not the recommended selling approach!)

With Epics, clearly, the objective is Vision, not specificity. In other words, the Epic need only be described in detail sufficient to initiate a further discussion about what types of Features an Epic implies.

Epics, Features and Stories, Oh My!

It’s apparent by now that Epics, Features and Stories are all forms of expressing user need and (hopefully) benefit, but at different levels of abstraction. Since this might be a less-than-satisfactory discriminator for the technical types, let’s look at an example.

In the Strategic Product Theme post, we hypothesized that past Strategic Product Themes for Gmail might have included: (by the way, I have no insights or contacts at Google; I’m just making this example up from their published information)

  1. Voice and video chat from within gmail
  2. Outlook integration
  3. Personalization
  4. Mail for Mobile
  5. Group chat

One can also imagine that Personalization of a user’s Gmail desktop is a significant initiative – one that Google might hope would create a strategic differentiation with Yahoo, Outlook and others – and one that could evolve over years. In other words, Personalization is a reasonable example of a well-stated Strategic Product Theme – you get the general idea, but you have no idea what exactly to do next!

In order to communicate better what to do next, prospective Epics, such as “Emoticons” and “User Background Desktop Themes“, could have then been identified, Then, the teams responsible for “User Background Desktop Themes” might well have brainstormed a multiple-stage implementation process, with the first releasable Feature being “apply a standard theme from Gmail theme catalog“, followed by “Allow users to further customize a chosen standard theme“. In the first iteration, the teams might have decided to create Stories to refactor the existing code. For example: “implement a framework where selected themes could be cataloged, selected and applied” and also a user value story “as an alpha tester, I can select and apply the first, prototype standard desktop theme so that I can provide early feedback on the concept”.

If this were the case, then the requirements information model for this example would be.

capture-combo1

Requirement Hierarchy Example

Discriminating Epics, Features, and Stories

Still, we admit that there is no scientific way to determine whether a “thing you know you want to do” is an Epic, Feature or Story, but perhaps the following table of discriminators will help:

 

Type of Information Description Responsibility Time frame & Sizing Expression format Testable
Strategic Product Theme BIG, hairy, audacious, game changing, initiatives.
Differentiating, and providing competitive advantage.
Portfolio fiduciaries Span strategic planning horizon, 12-18+ months.
Not sized, controlled by percentage investment
Any: text, prototype, PPT, video, conversation No
Epic Bold, Impactful, marketable differentiators Program and product management, business analysts, business unit managers 6-12 months.
Sized in points.
Most any, including prototype, mockup, declarative form or user story canonical form No
Feature Short, descriptive, value delivery and benefit oriented statement. Customer and marketing understandable. Product Manager and Product Owner. Fits in an internal release (PSI) , divide into incremental sub- features as necessary.
Sized in points.
Declarative form or user story canonical form.
May be elaborated with system use cases.
Yes
Story Small atomic. Fit for team and detailed user understanding Product Owner and Team. Fits in a single iteration.
Sized in story points.
User story canonical form Yes

We also note in this table that Epics are indeed estimable (sizable). I covered that topic in an earlier, Big Picture post: Enterprise Agility – The Big Picture (13 continued): Estimating Epics.

Types of Epics

In earlier versions of the model and in the Big Picture we have indicated that there are two primary type of Epics to be considered:

Epics in the Big Picture and in the Model

Epics in the Big Picture and in the Model

  1. User Epics (Consumer Epics for many enterprises and earlier versions of the model) reflect those initiates that directly affect the deliver benefits to the end user.
  2. Technology (Architecture in earlier versions of the model) Epics reflect those initiatives that are necessary to adopt new technologies, or major architectural refactors that are necessary to create the architectural runway an enterprise needs to avoid excessive refactoring.

In our Gmail example, Personalization represents a User (Consumer) Epic. Perhaps an Epic such as “Ajax-enable across all supported browsers” could represent a significant Technology Epic.

Looking Ahead

Now that we have described Epics, we’ve introduced and elaborated on all the current model elements of A Lean, Scalable Requirements Information Model for the Agile Enterprise, Highlights (well, if you could call them that…) include the User Story and Acceptance Tests covered in the Agile Enterprise Requirements Information Model – Subset for Agile Project Teams, Features covered in Agile Requirements Information Model (4) – Subset For Agile Programs (a): Features and Feature Backlog, Nonfunctional Requirements (Backlog Constraints) and System Validation Tests covered in Agile Requirements Information Model (5) – For Agile Programs (b): Nonfunctional Requirements, Strategic Product Themes covered in Agile Enterprise Requirements Information Model (6a) – Subset for Portfolio Management: Strategic Product Themes

In a near term post, I’m going to recap the changes to the model in an updated graphic which recasts the Basic and Advanced forms of the information model.

Also, based on comments and feedback (like “whatever happened to use cases, can’t I use them in agile?), I also intend to discuss use cases in the context of agile development and this agile requirements information model. After all, if you are addressing enterprise class challenges, you may wish to have ALL the effective tools at your supposal. So I suspect I’ll be on this thread for a bit longer.

Agile Enterprise Requirements Information Model (6a) – Subset for Portfolio Management: Strategic Product Themes

In this blog series, we’ve been describing a Lean, Scalable Requirements Information Model for the Agile Enterprise (AE RIM) suitable for extending the basic requirements practices of agility to the largest software enterprises (hundreds and thousands of agile practitioners). This model represents the requirements substrate for an even broader model, a model for implementing the basic organizational, process and requirements practices of an agile enterprise as represented in the Big Picture Blog Series. In this series, I’ve been collaborating with Juha-Markus Aalto, Director of Operational Development for Nokia S60 Software Unit, where a variant of the model is being developed and applied on a very large scale project.

To date, we’ve described the basic model, what’s lean and scalable about the model, the model subset for agile project teams, and the subset for agile programs, including an appropriate treatment for those critical Nonfunctional Requirements that govern so much of a systems behavior.

In these next two posts, we’ll address the last, and highest level of the model, how it applies to agile portfolio management, represented at the top of the big picture graphic below.

Big Picture with Portfolio Level Highlighted

Big Picture with Portfolio Level Highlighted

But first, let’s look at the basic model, with those remaining elements highlighted.

capture-model-with-epic-and-spt-highlihted1

(Note 1: the model has evolved and been expanded over the course of this series. The above reflects the current, basic model. We’ll provide an update, including the basic and expanded model in a near-future post.
Note 2: We also note that the model, which has a more specific, product context than the Big Picture uses “Strategic Product Theme” instead of the Big Pictures Investment Themes, but I haven’t decided whether or not to do anything about it yet.)

We see from the model that the portfolio elements are Strategic Product Theme and Epic. In this post, we’ll address the Strategic Product Theme.

Strategic Product Themes

Strategic Product Themes represent the set of initiatives which drive the enterprises investment in systems, products and applications. The set of Themes for an enterprise, (or more likely a business unit within an enterprise), establishes the relative investment objectives for the entity as the pie chart below illustrates:

Strategic Product Themes Portfolio Mix

Strategic Product Themes Portfolio Mix

These Themes drive the Vision for all product teams and new Epics are derived from this decision. While the derivation of these decisions is outside the scope of this blog series, the responsibility for these decisions lies with those who have fiduciary responsibilities to their stakeholders. In most enterprises, this happens at the business unit level based on annual or twice annual budgeting process, which in turn, determines the amount of funds available for each business unit to invest in development.

Within the business unit, the decisions are based on some combination of:

  1. Investment in existing product offerings – enhancements, support and maintenance
  2. Investment in new products and services – products that will enhance revenue and/or gain new market share in the current or near term budget period
  3. Investment in futures – product and service offerings that require investment today, but will not contribute to revenue until outlying years.

Balancing these investments is difficult and we should appreciate the enormous pressures and conflicting priorities that are constantly present to those who make these decisions for our enterprise. But make them they must or the enterprise will wander though time with an unfocused strategy. And we can all guess how that will work out.

The result of the decision process is a set of Themes – key product value propositions that provide marketplace differentiation and competitive advantage. Themes have a much longer life span that Epics, etc. and a set of Themes may be largely unchanged for up to a year or even more.

Communicating Strategic Product Themes

While determining the investment mix is no small feat, communicating the decision via Themes is fairly straightforward, as Themes are very high level and are easily represented in bullet form. For example, one could imagine the Google Gmail team deciding on the following Themes in some recent period:

  1. Introduce voice and video chat from within gmail
  2. Outlook integration
  3. Personalization
  4. Mail for Mobile 2.0
  5. Group chat from within mail

Why Investment Mix Rather than Backlog Priority?

As opposed to Epics, Features and Stories, Strategic Product Themes are not contained or represented in a Backlog (they are not “a kind of Backlog Item”) as the model shows.

capture-themes-are-not-backlog

The differences are subtle, but important.

  • Backlog Items are designed to be addressed in priority order. Strategic Product Themes are designed to be addressed on “a percentage of time to be made available basis.” For example, the lowest priority Story on an iteration backlog may not be addressed at all in the course of an iteration and yet the iteration could well be a success (meet its stated objectives and be accepted by the product owner). However, if the lowest priority (smallest investment mix) Strategic Product Theme is not addressed over time, the enterprise may ultimately fail in its mission as it is not making its actual investments based on the priorities it has decided.
  • Strategic Product Themes also do not share certain other Backlog Item behaviors. For example, as critical as they are, they are not generally testable, as their instantiation occurs first through Epics and then finally, via actual implementation in Features and Stories, which have the specificity necessary to be testable.
  • If the enterprise is focused, there are only a few Themes active at any one time, so they don’t require a backlog management tool (spreadsheet or agile project management tool) to capture and maintain them.

A Look Ahead

That concludes our discussion of Strategic Product Themes, which are the portfolio drivers that lead to the creation of Epics that directly affect the course of software development. In the next post we’ll (hopefully) conclude our current discussion series of the AE RIM with a discussion of Epics.

Lean & Kanban Conference Rescheduled – May 6th-8th

The  new conference focusing on the application of Lean and Kanban methods for software development – Lean & Kanban 2009 has been rescheduled to May 6-8 in Miami. This innovative conference will featuring a full day of presentations with separate tracks for Lean and Kanban, plus a full day of open space and a half day of lightning talks.

“Learn the latest thinking in applying Lean techniques like Kanban to shorten cycle time and deliver more value from your software projects. Mix with the experts and practitioners using these ideas every day in organizations around the world. Be part of our community. Be part of the next wave of software management and leadership.”

It’s a smaller, informal conference with Keynotes from lean thought leaders including, myself,  Alan Shalloway, and David J. Anderson, so it’s a great opportunity to learn about the latest techniques for improving productivity and value delivery as well as to hob knob with the presenters.

I’ll be delivering a keynote at the conference focusing on Lean and Scalable Requirements Practices for the Agile Enterprise that is the subject of the Agile Requirements and Big Picture blog series. This will be my first public presentation on the topic and I look forward to an informative and interactive discussion of these ideas and real world practices. Other speakers will include:

I expect this conference to be innovative, informative and perhaps even a little controversial around the edges. Just my cup of tea!

Hope to see you there.

–Dean

Agile Requirements Information Model (5) – For Agile Programs (b): Nonfunctional Requirements

Note: This is the fifth in a series of posts describing, in more rigorous terms, A Lean, Scalable Requirements Information Model for the Agile Enterprise (AE RIM) that underlies the Big Picture Series. In this series, I’ve been collaborating with Juha-Markus Aalto, Director of Operational Development for Nokia S60 Software Unit, where a variant of the model is being developed and applied on a very large scale development project.

In the last post, the Agile Requirements Information Model For Agile Programs we described those elements of the model, Features and Feature Backlog that are relevant to the Program Level of the Big Picture. (see graphic).

capture-big-picture-with-program-highlights1

Agile Enterprise Big Picture with Program Level Highlighted

In this post, we complete that discussion with a discussion of Non-functional Requirements. For reference, the basic information model is below, with the elements relevant to agile programs highlighted.

capture-model-for-nfr-post

Non-functional Requirements in Agile Development

From a requirements perspective, the User Story form of expression used by the agile team is used to describe the functional requirements of the system; those system behaviors whereby some combination of inputs produces a meaningful output (result) for the user. With all due respect to that nifty agile invention, however, little has been described in agile as to how to handle the Non-functional Requirements (NFRs) for the system.

Traditionally, these were often described as the “ilities” – quality, reliability, scalability, etc. – and the thought of “don’t forget the ilities” is intended to remind us that these are important and critical elements of system behavior. Indeed, they are often as or more import as the sum total of all the functionality. For if a system isn’t reliable (crashes) or marketable (failure to meet some imposed regulatory standard) or scalable (doesn’t support the number of users required) then, agile or not, we will fail just as badly as if we forgot some critical functional requirement.

Non-functional Requirements have been explored and described in texts such as Managing Software Requirements: A Use Case Approach (Leffingwell), and it isn’t necessary to re-explore all that territory here. However, as a reminder of all these considerations, I like to think that teams who do a good job of understanding and addressing these requirements are SCRUPLED, in that they responsibly consider at least the following types of NFRs:

  • Security, Licensing, Installation– access privileges, security requirements, installation and licensing requirements
  • Copyright, legal notices and other items – required corporate representations and legal protections
  • Reliability – defects, Mean Time Between Failures, availability
  • Usability – ease of use requirements – presentation design guidelines, UE standards, accessibility standards, training standards, sheets, help systems etc.
  • Performance – quantitative performance requirements
  • Localization and Internationalization – foreign language operating systems, localization enablement, specific localizations
  • Essential standards
    industry, regulatory
    and other externally imposed standards
  • Design Constraints – other constraints on the system or development technologies. Mandated programming languages and standards, platforms, common components, etc.

Aren’t These Non-functional Requirements Just Backlog Constraints?

In one perspective, all of these items can be considered to be constraints on new development, in that each eliminates some degree of design freedom on the part of those building the system.

For example: “yes, we need to implement Single Sign-On (SSO) functionality in our accounting suite, (a new Feature) but we still have to be compliant with Sarbanes Oxley (constraint) “. Example: “For compatibility with our partners, we have agreed to implement SAML-based SSO (constraint) for all products in the suite”. And “every product in the suite has to be internationalized (constraint), but only the Order Entry module must be localized to Korean (constraint) for this release.”

So in the information model, we have modeled them as such:

Association Between Backlog Items and NFRs

Association Between Backlog Items and NFRs

We see that a) some Backlog Items may be constrained by zero or more Non-functional Requirements (0…*) and b) Non-functional Requirements apply to zero or more backlog items (0..*).

Once identified, ALL relevant Non-functional Requirements must be captured and communicated to ALL teams who may be affected by the constraints. One way to do this is to create a repository of these items in the requirements or agile project management tool the team is using. While our model is tooling independent and doesn’t address the “where this stuff is persisted”, we’ll just call them Backlog Constraints and represent them pictorially in the Big Picture as follows:

A Backlog with its Backlog Constraints

A Backlog with its Backlog Constraints

Ok, well maybe this looks like a “bucket of backlog constraints”. But then again, maybe that’s what it is!

Testing Backlog Constraints

When we look at the long list of list of potential Backlog Constraints – Security, Copyright, Reliability, Usability, Performance , Localization, Essential standards and Design Constraints – the question naturally arises as to whether these constraints are testable.

If we adhere to the agile mantra that “all code is tested code”, then the answer is assuredly yes, as most of most all of these constraints (Performance for example) can be objectively tested. Indeed, system compliance to these requirements is just as critical as it is to meeting functional requirements. However, up until this point the model hasn’t reflected testing Backlog Constraints as explicitly as it has called out testing the functionality implemented in Features and Stories via Acceptance Tests. But that is easily remedied in the extension below:

capture-introduce-system-validation-tests

Rather than calling these tests Acceptance Tests (a word generally used in agile to indicate the acceptance of a Feature or Story by the customer or Product owner) and further overloading that term, we’ve called them System Validation Tests. This is intended to better describe how this set of tests help assure that the system is validated to be in compliance with its Nonfunctional Requirements. The multiplicity (1..* and 0..*) further indicates that

  • Not every NFR has a validation test. For example, some design constraints such as “Program in Python” simply aren’t worth testing (other than perhaps by one-time inspection or acknowledgment).
  • However most are (..*) and one would expect that most nonfunctional requirements have at least one objective test associated with them. Moreover, some NFRs (“Compatibility with IE 8 Beta”, for example), might require an entire test suite to assure conformance.
  • Every system validation test should be associated with some NFR (1..*) (otherwise there would be no way to tell whether it passes!), and
  • Some System Validation tests could assure compliance with more than one NFR.

Do all Backlogs Have Such Constraints?

In earlier versions of the Big Picture, I described most non-functional requirements as an addendum to the Vision. That was a simplification however, as the information model and the Big Picture illustrates three backlogs: the Epic, Feature and Story backlogs, each of which can have associated Backlog Constraints.

For example, internationalization and regulatory requirements may affect many or all project teams, so these NFRs live at the Feature level. However, I recently saw another circumstance whereby a very complex system, which was in turn composed of many applications, platforms and subsystems, needed a substantive investment in overall performance and reliability. So in this case, this was an Epic level Backlog Constraint (“substantively increase performance and reliability “) and it fit conveniently right next to all the other Epics that warranted investment. So the model supports, but does not require, Backlog Constraints (sets of NFRs) at all levels of the hierarchy.

The Big Picture, now updated with Backlog Constraints buckets, looks like this:

  

 

Big Picture Updated with Backlog Constraints

Big Picture Updated with Backlog Constraints

With the addition of System Validation Tests, we now have elevated NFRs (called Backlog Constraints) to be first class citizens of the lean and scalable Agile Enterprise Requirements Information Model, thereby reemphasizing the importance of these types of requirements when building large scale systems in an agile manner.

Conclusion

This completes the discussion of Non-functional Requirements, which we are describing as sets of Backlog Constraints which govern development of new Backlog Items. In turn, this completes the discussion of the Agile Enterprise Information Model for Programs. In the next post, we’ll describe the last element of the hierarchy, the Portfolio level.

Agile Requirements Information Model (4) – Subset For Agile Programs (a): Features and Feature Backlog

Note: This is the fourth in a series of posts describing, in more rigorous terms, the  Lean, Scalable Requirements Information Model for the Agile Enterprise (AE RIM) that underlies the Big Picture Series. As I’ve noted, I’ve been collaborating in this series with Juha-Markus Aalto, Director of Operational Development for Nokia S60 Software Unit, where a variant of the model is being developed and applied on a large scale (very large- affecting thousands of practitioners) development project.

In the last post, the Subset for Agile Teams we described the model as it applies to Agile Project Teams. In this next post, we move up one level in the Big Picture (see graphic below) and describe how the model is applied at the Program Level.

Agile Enterprise Big Picture with Program Level Highlighted

Agile Enterprise Big Picture with Program Level Highlighted

But first, for convenience, we’ll take a quick look again at the base model.

Base Requirements Information Model

Base Requirements Information Model

Note: as the model is being elaborated in earlier posts, it is being refined with additional model elements and relationships, but I’ll continue to use the summary version (base) model here for simplicity.

The Program Level.

As can be seen in the Big Picture, we assume that a significant number of Agile Teams are collaborating in a Program building an enterprise-class System of substantial complexity. We described the teams use of a quintessentially agile model subset (Story, Task, Backlog Item and Acceptance Test) in the last post.  At the Program Level, we’ll introduce the additional elements of Feature Backlog Item (in this case, Feature backlog item) and Non-Functional Requirement.

capture-program-elements-highlighted

Feature

In describing the features of a product or system, we leave the world of small team agile (where most everything is a Story) and take a more abstract and higher level view of the system of interest. In so doing we have the security of returning to a more traditional description of system behavior. There is certainly some comfort in knowing that while agile changes most everything, it needn’t change absolutely
everything we have learned prior, including what has been well described in software requirements texts (for example: Managing Software Requirements: A Use Case Approach: Leffingwell and Widrig). In that text, Features were described as:

Services provided by the system that fulfill a user need.

They lived at a level above software requirements and bridged the gap from the problem domain (understanding user needs) to the solution domain (specific requirements intended to address the user needs) as the graphic below shows:

"Requirements Pyramid" from Managing Software Requirements, by Leffingwell and Widrig, Addison-Wesley 2003.

"Requirements Pyramid" from Managing Software Requirements, by Leffingwell and Widrig, Addison-Wesley 2003.

We also posited in that text that a system of arbitrary complexity can be described with a list of 25-50 such features. (Note, however, that in the Big Picture, Features are sized to fit within release boundaries, so that a release can be described in terms of a list of just a few features).

That simple rule of thumb allowed us to keep our high level descriptions exactly that, high level, and simplified our attempts to describe complex systems in a shorter form. Of course, in so doing we didn’t invent either the word “Feature” or the usage of the word in that text. Rather, we simply fell back on industry standard norms to describe products in terms of, for example, a Features and Benefits Matrix that was often used by product marketing to describe the capabilities and benefits provided by our new system. By applying this familiar construct in agile, we also bridge the language gap from the agile project team/product owner to the system/program/product manager level and give those who operate outside our agile teams a traditional label (Feature) to use to do their traditional work (i.e. describe the thing they’d like us to build).

Even more conveniently, we also provide a surface to organize agile teams around – the Feature Team, which is an agile team organized for the purpose of delivering a set of features to end users – see Larmon & Vodde-Scaling Lean and Agile Development, Highsmith- Agile Project Management and Leffingwell-Scaling Software Agility. (Note: in my latter text, though, I described teams more commonly as agile component teams, but noted that these teams could be organized around Features, Components, Subsystems, or whatever made sense to the system context).

Whether by serendipity or intent, the ability to describe a system in terms of its proposed features and the ability to organize agile teams around the features gives us a straightforward method to approach building large-scale systems in an agile manner.

The Feature Backlog

Returning to the Backlog and Feature model elements:

capture-backlog-and-feature-highlights

We can see two obvious implications.

  1. Features are realized by Stories. At release planning time, Features are decomposed into Stories, which is the team’s implementation currency. (In a large system, there may tens and even hundreds of Stories necessary to realize a Feature)
  2. Planned Features are stored in a Backlog, in this case the Feature (Release) Backlog.

In backlog form, Features are typically expressed in bullet form, or at most, in a sentence or two. For example, you might describe a few features of Google Mail something like:

  • Provide “Stars” for special conversations or messages, as a visual reminder that you need to follow-up on a message or conversation later.
  • Introduce “Labels” as a “folder-like” conversation-organizing metaphor.

Expressing Features in Story Canonical Form

As agilists, however, we are also comfortable with the suggested, canonical Story form described in the last post:

As a <role> I can <activity> so that <business value>

Applying this form to the Feature construct can be also to help focus the Feature writer on a better understanding of the user role and need. For example:

As a modestly skilled user, I can assign more than one label to a conversation so that I can find or see a conversation from multiple perspectives

Clearly there are advantages to this approach. However, there is also the potential confusion from the fact that Features then look exactly like Stories but are simply written at a higher level of abstraction. But of course, that’s what they really do represent!

Testing Features

In the last post, we also reintroduced the agile mantra all code is tested code and illustrated the point with the base model:

capture-stpry-with-acceptance-test

This illustrates a Story cannot be considered Done until it has passed one or more acceptance tests.

At the Program level, the question naturally arises as to whether or not Features also deserve (or require) Acceptance Tests. The answer is, most typically, yes. While story level testing should assure that the methods and classes are reliable (unit testing) and the stories suits their intended purpose (functional testing) , the fact is that the Feature may span multiple project teams and many, many (tens to hundreds) of stories.

While perhaps ideally, each project team would have the ability to test the Feature at the system level, the fact is that that is often not practical (or perhaps even desirable – after all, how many teams would we want to continuously test the same feature!). Many individual project teams may not have the local resources (test bed, hardware configuration items, other applications) necessary to test a full system. Moreover, at scale, many teams are developing APIs, infrastructure components, drivers and the like based upon the architecture of the solution, and they may not even have an understanding of the full scope of the system Feature that drove their Story to exist. (If you don’t believe or agree with this, test the hypothesis by asking some newly hired teammates working on a new feature or component of a really big system to describe how or why a system-level feature works……).

In addition, there are also a myriad of system-level “what if” considerations (think unified process alternate use-case scenarios) that must be tested to assure the overall system reliability and some of these can only be tested at the full system level.

For this reason, Features typically require one or more functional acceptance tests to assure that the Feature meets the user’s needs. In practice, many of these tests may be implemented at a higher level in the hierarchy, at the Big Picture System Team level. (see below)

system-team1

To reflect the addition of Acceptance Tests to Features, it is necessary to update the information model with an association from Feature to Accetpance Test as the grpahic below shows.

Revised Information Model, Adding Acceptance Tests to Features

Revised Information Model, Adding Acceptance Tests to Features

In this manner, we illustrate that every Feature requires one or more Acceptance Tests, and a Feature cannot be considered Done until it passes.

Next- Nonfunctional Requirements in Agile

Well, that covers the Feature model element at the Program level. However, we have yet to explore another critical Program model element, the Nonfunctional Requirement. Since this post is already way too long, I’ll leave that for the next post in this series.