Announcing an Upcoming Book on Agile Requirements

As readers of this blog know, I’ve been using my previous book, Scaling Software Agility: Best Practices for Large Enterprises, as a source reference to help guide large-scale enterprises in becoming more agile in their software development projects. In the course of many engagements with these large enterprises, a recurring theme keeps popping up, “How do I handle requirements at scale in an agile environment?” It’s a good question that deserves a thorough treatment of the subject and, hopefully, some practical answers to that question. I’ve been addressing some of the answers in various blog posts and whitepapers (see Agile Requirements category and A Lean and Scalable Requirements Model for Agile Enterprises.) Also, I’ve written fairly extensively on the requirements topic in the past, in Managing Software Requirements: First and Second Editions, from Addison-Wesley.

While thinking about this issue for the last few years, I’ve accumulated a lot of practical approaches to the question of managing requirements for big systems in an agile world. In addition, several agilists have been thinking about this problem as well and have made substantial contributions to the art. Although I tried to duck the issue, it just wouldn’t go away, so I’ve decided to write another book (with help from collaborators such as Mauricio Zamora, Pete Behrens, Jennifer Fawcett and others) that focuses specifically on this matter. I’ve tentatively titled the book Agile Requirements: Lean Requirements Practices for Teams, Programs and the Enterprise and, with help from my co-author, Don Widrig, it should be published by Addison-Wesley sometime early next year.

As in all things agile, it isn’t completely nailed down yet but here is the Table of Contents that I’m working from now. A lot of the book exists in draft form, so don’t hesitate to contact me if you’re interested in helping me shape this book for final publication. In future posts, I’m going to push various whitepapers and snippets to gather early feedback, and to help anyone who might actually find this stuff helpful between now and the time the book is published.

===============

Preliminary Table of Contents

Agile Requirements: Lean Requirements Practices for Teams, Programs and the Enterprise

by Dean Leffingwell, with Don Widrig

Forward

Preface

About the Author

Front Matter

·      Preface

·      How to Read this Book

·      Acknowledgements

Part I – Overview: The Big Picture

1.             A Brief History of Software Requirements Methods

2.             The Big Picture

3.             Agile Requirements for the Project Team

4.             Agile Requirements for the Program

5.             Agile Requirements for the Portfolio

Interlude Case Study: Tendril Residential Energy Ecosystem

Part II – Agile Requirements for the Team

6.             User Stories

7.              Stakeholders, User Personas and User Experiences

8.             Estimating and Velocity

9.             Iterating

10.          Acceptance Tests

11.          Role of the Product Owner

12.          Requirements Discovery Toolkit

Part III – Agile Requirements for the Program

13.          Vision, Features, and the Solution Roadmap

14.          Role of the Product Manager

15.          Release Planning and Execution

16.          Nonfunctional Requirements

17.          Requirements Analysis Toolkit

18.          Use Cases

Part IV – Agile Requirements for the Portfolio

19.          Systems Engineering

20.          Business Modeling

21.          Epics and Agile Portfolio Management

22. Epics and Enterprise Architecture

Is Scrum Lean?

In my last post, I described some work I’ve been doing with lean thinking in some larger software enterprises – not so much for teams themselves, who generally find Scrum and/with/or XP to be adequate for their needs – but to other levels in the organization – project office, sales, marketing and distribution – where value stream analysis, reduction of waste and elimination of delays in time to market resonate better than the agile manifesto or the methods themselves. In addition, lean provides a broader set of both philosophies and practical tools to apply in  making their departments and organizations more agile.

In that post, I noted that a bigger question naturally arises:

“Do we have two different, complementary or even competing philosophies at work – lean for the enterprise and agile for the teams? Or is agile really a “software instance of lean”, making the philosophy of approach and language of discussion, one harmonious construct, where agile is the lean development practice subset of an overall lean enterprise approach. “

XP and Lean?

I also quoted Kent Beck in noting that:

“I think that Agile and Lean are strongly related, but that they are two different ideas. Lean aims to achieve efficiency through eliminating waste and respecting people. Agility is a by-product in lean as rapid cycles are required to identify and eliminate waste. Agile software development aims to meet the evolving needs of customers through the early and continuous deployment of valuable software.

The values, principles, and practices of the two approaches are different, even though complementary.”

With XP, the linkage is indirect – well correlated and complementary for certain- but it’s clear that Kent doesn’t see XP as being derived from, or based on, lean.

What About Scrum?

A few years back, I came across a presentation that Jeff Sutherland on the The Roots of Scrum. Jeff also pointed me to a video of his actual, physical presentation, which can be found on InfoQ.

In this presentation, Jeff notes that there are many roots of Scrum, including such things as the advent of OO methods (rapid development and less painful refactoring), process and productivity research (empirical methods work best when process is to complex to be predicted), and even evolutionary biology and complex adaptive systems(central command and control will kill it!), but right at the top of the list are some of the philosophies concurrent with, but not the same as,  the development of lean.

Jeff notes that the “Godfathers of Scrum” are Takeuchi and Nonaka (authors of The New, New Product Development Game. This is my personal favorite, root philosophy, of all things agile and includes the first use of the words “Scrum”, “Rugby” and “Sashimi” in a software context that I am aware of (also see my earlier post). Takeuchi and Nonaka are also the authors of The Knowledge Creating Company and Hitotsubashi on Knowledge Management.

Just a few excerpts from these works:

From the New, New, Product Development Game:

“leading companies show six characteristics in managing their new product development process:

1.            Built-in instability.

2.            Self-organizing project teams

3.            Overlapping development phases.

4.            Multi-learning.

5.            Subtle control.

6.            Organizational transfer of learning. “

From Jeff’s presentation and Hitotsubashi on Knowledge Management:

Ba- the Zen of Scrum (shared “context in motion” and the energy that drives a self-organizing team):

  • “Dynamic interaction of individuals and organization creates synthesis in the form of a self-organizing team
  • The fuel of ba is its self-organizing nature a shared context in which individuals can interact
  • Team members create new points of view and resolve contradictions
    through dialogue
  • New knowledge as a stream of meaning emerges
  • This emergent knowledge codifies into working software
  • Ba must be energized with its own intentions, vision, interest, or mission to be directed effectively
  • Leaders provide autonomy, creative chaos, redundancy, requisite variety, love, care, trust, and commitment
  • Creative chaos can be created by implementing demanding performance goals. The team is challenged to question every norm of development
  • Time pressures will drive extreme use of simultaneous engineering
  • Equal access to information at all levels is critical.”

Does this look like the good Scrum teams you know?

Of course, this background describes the roots of Scrum from certain philosophies, practices and authors perspectives contemporaneous with development of lean thinking and concurrent development, but does not specifically place lean in the roots of Scrum.

Additional Background

In my lean research, I’ve been going back to some of the earlier and quintessentially lean works, including: Lean Thinking from Womack and Jones, The Toyota Way by Jeffrey Liker, Lean Software Strategies by Peter Middleton and James Sutton (both founding members of Lean Software and Systems Consortium), to better understand both lean, and its application in the software development context.

In this review, I’ve been struck again and again by common principles – not just like principles or complementary principles – but virtually identical, fundamental, and immutable principles that provide the cornerstone values and philosophies of both lean and agile approaches. I’ve become more and more convinced that agile is a software instance of lean– purpose built for the unique challenges, intangibles and thought-based work of developing software as fast as efficiently as possible – but as lean as lean can be in that context.

Next Post

With respect to Scrum, in particular, the similarity is just too hard to ignore. I’ll describe that in my next post Scrum in the House of Lean?

Are Scrum and XP Lean? (Part 1)

As I mentioned in my last post, I’m working with a number of large-scale agile transformations, most deploying Scrum. However, while Scrum drives agility and efficiency for the software development team, Scrum (or XP) doesn’t have so much to offer to the line managers, directors and VPs engaged in managing development, project, program, and product management and all the other surrounds that are necessary in the larger enterprise. And we all know that a set of agile teams, by themselves, does not assure an agile enterprise.

To address these larger issues, I’m using the principles and tools of lean – value stream analysis, flow, pull, kaizen mind, limiting work in process, etc. – because these principles apply equally well to a program office processing incoming requirements as they do to manufacturing lines. In addressing this larger problem, you have to consider the value stream in total, from the sales office where an order originates – to the development team – to production and distribution. As you know, that can be a long and arduous process in the enterprise, resulting in unacceptably slow time to market, even if the teams themselves are agile.

So the question arises, do we have two different, complementary or even competing philosophies at work – lean for the enterprise and agile for the teams? Or is agile really a “software instance of lean”, making the philosophy of approach, and language of discussion, one harmonious construct, where agile is the lean development practice subset of an overall lean enterprise approach.

Based on my personal experiences with lean manufacturing a decade or so back, I’ve always seen agile development in lean terms, and it certainly seems to be a “lean instance” to me.  Many others do to. For example, Ryan Martens recently blogged on the topic at Agile and Lean Software Development – an Oxymoron?  However, some of the experts comments he received back were not so simpatico.

With respect to XP, Kent Beck commented on Ryan’s post:

“I think that Agile and Lean are strongly related, but that they are two different ideas. Lean aims to achieve efficiency through eliminating waste and respecting people. Agility is a by-product in lean as rapid cycles are required to identify and eliminate waste. Agile software development aims to meet the evolving needs of customers through the early and continuous deployment of valuable software.

The values, principles, and practices of the two approaches are different, even though complementary.”

Now personally, I’ve never considered the elimination of waste as the primary driver behind lean (I rather think of flow, pull, kaizen, reduced wip, faster cycle times – which of course tends to happen when you reduce waste!). However, in the popular book Lean Thinking: Banish Waste and Create Wealth in Your Corporation (Womack and Jones), eliminating “muda” (waste) is the primary theme, so that there can be no doubt of the objective from their expert perspective. Larman and Vodde however,  disagree in noting “the pillars of lean are not tools and waste reduction”.

No matter whether elimination of waste is or isn’t the primary objective of lean, it’s clear that Kent Beck does not see XP as a subset of lean, as they have “different values, principles, and practices”.  And who would argue with Kent on the philosophy and roots of XP?

As for Scrum, however, which is the primary method being use in the enterprise projects I’m involved with, its roots are quite different, as you’ll see in my next post.