Rearchitecting with Flow

I’m happy to report that the final chapters of the agile requirements book are largely converging. It now appears as it might actually be finished someday. In the meantime, for those of you who have been interested in the topic of agile architecture, I’ve completed the next draft of Chapter 21 – Rearchitecting with Flow and posted it here for public review and comment.

I think it provides some innovations that might help  larger agile enterprises  reason about and implement system-level architecture in a lean and agile manner.

I’d like to thank the architects and enterprise agilists at F-Secure Corporation, including KE Tan, Gabor Gunyho, Santeri Kangas, Ari Inki and James Cooper,  for their substantial contribution to this Chapter. Even more importantly, they are putting a variant of this system to work right now so we’ll have more feedback over the next few months  However, any errors in thinking, logic, design or description are solely my own.

Advertisements

Prioritizing Features

In the last post, I described a system for estimating features that will be in my upcoming book. This post, prioritizing features, is that post’s evil twin. Together, it should be provide a reasonable strategy to estimate cost and schedule for a feature, and a systematic way to pick  which features will deliver the highest value first, based on the cost of delay. For context, you may want to browse the Big Picture category and the Agile Requirements category.

Introduction

One of the largest challenges that all software teams face is prioritizing requirements for implementation and delivery to the customers. This is certainly a challenge for every agile team at iteration boundaries, and it rises to even greater importance when prioritizing features at the program level. Here, small decisions can have big impacts on implementation cost and timeliness of value delivery.

There are a number of reasons why prioritization is such a hard problem:

  1. Customers are seemingly reluctant to prioritize features. Perhaps this is because they simply “want them all”, which is understandable; or perhaps they are uncertain as to what the relative priorities are; or perhaps they cannot gain internal agreement.
  2. Product managers are often even more reluctant. Perhaps this is because that if they could only get them all, they wouldn’t have to prioritize anything and more importantly, they would be assured of receiving all the ultimate value[1]
  3. Quantifying value is extremely difficult. Some features are simple “must haves” to remain competitive or keep market share. How does one quantify the impact of keeping market share, one feature at a time?

To assist us in our efforts, we often attempt to provide a Return on Investment (ROI) per feature, by predicting the likely increase in revenue if a feature is available. Of course, investment in determining ROI is most likely a false science, as no ones crystal ball is an adequate predictor of future revenue, especially when you attempt to allocate revenue on a per-feature basis. This is compounded by the fact that the analyst that does the work is likely to develop a vested, personal interest in seeing that the feature is developed. Plus, any product manager or business analyst worth their salt can probably make a case for a great ROI for their feature – otherwise they wouldn’t have worked on it to begin with.

In agile, however, the challenge of prioritization is immutable – we admit up front that we can’t implement (nor even discover) all potential requirements. After all, we have typically fixed quality, resources and delivery schedule. Therefore, the only variable we have is scope. Effective prioritization becomes a mandatory practice and art; one that must be mastered by every agile team and program.

Of course, prioritizing requirements is not a new problem. A number of authors have described reasonable mechanisms for prioritization. Our favorites include

  • Agile Estimating and Planning [Cohn. 2006]
  • Software Requirements [Wiegers. 1999.]
  • Software by Numbers [Deene and Cleland-Huang [2004].

For those for whom this topic is potentially a determinant of program of success or failure of the program, we refer you also to these bodies of work. While we will take a different approach in this post (and upcoming book), there is certainly no one right way to prioritize and teams will benefit from differing perspectives on this unique problem.

Value/Effort as an ROI proxy – A First Approximation

As we alluded to above, we have traditionally prioritized work by trying to understand the relative Return on Investment, which is the relationship between potential return (value) divided by the effort (cost to implement) for a feature. At least the model was simple:

If we could simply establish value and cost – if not in absolute terms, then at least relative to other features – then we have a way to prioritize based on economics. After all, who wouldn’t want to deliver a higher ROI feature before a lower ROI feature? That seemed to make totally intuitive and (apparently) economical common sense.

What’s Wrong with our Value/Effort ROI Proxy?

However, based on more complete economic framework, it turns out that relative ROI (as we’ve too simply defined it above) is not an adequate proxy for prioritizing value delivery.  Recently, one of the more thoughtful and rigorous economic views to prioritizing value delivery (sequencing work in flow terms) is Reinertsen’s Principles of Product Development Flow [Reinertsen 2009] that we introduced in an earlier post. These principles describe methods of sequencing work based on the mathematics and underlying economics of lean product development.

We’ll use those principles to describe an enhanced method for prioritizing features. As we do so, we’ll discover a deeply seated flaw in our first assumption – the assumption that a high relative ROI project should naturally have precedence over a lower ROI project.

Instead, what need to understand is the way in which the economics of our program may be dramatically affected by sequence. For example, the potential profit for a particular high ROI feature could be less sensitive to a schedule delay than a lower ROI feature. In this case, the lower ROI feature should be implemented first, followed by the higher ROI feature. This may not make intuitive sense, but we’ll see that it does make economic sense.

Prioritizing Features Based on the Cost of Delay.

Since prioritizing value delivery is the key economic driver for a program, we’ll need a more sophisticated model to produce better returns. To build the model, we’ll make a small detour through some of the fundamentals of queuing theory. After all, as we described in another post, prioritizing value delivery in software development is just a special case of queuing theory. So applying those principles will should create a solid foundation for critical decision-making.

Introducing Cost of Delay

As Reinertsen points out “if you only quantify one thing, quantify the cost of delay.”[2], so we’ll need to be able to estimate the CoD as part of our more economically-grounded approach. (More on that shortly). Fortunately, however, we don’t have to only quantify one thing, and as we have already outlined an estimating strategy, we’ll actually be able to quantify two things, the feature effort estimate, as we have described in the last post, and the Cost of Delay. Together, we should have what we need.

In achieving product development flow, Reinertsen describes three methods for prioritizing work based on the economics of Cost of Delay.

Shortest Job First

When the cost of delay for two features is equal, doing the Shortest (in our case, smallest) Job First, produces the best economic returns, as is illustrated in the Figure below.

Shortest Job First

Fro

Indeed the impact is dramatic, as doing the smallest job first has economic returns that are many times better than doing the larger job first. So we arrive at our first conclusion:

If two features have the same Cost of Delay, do the smallest feature first.

High Delay Cost First

If the size of the jobs are about the same, then the second approach, High Delay Cost First, illustrates the effect of prioritizing the jobs with the highest cost of delay. Again, the economics are compelling as the figure below illustrates.

High Delay Cost First

Of course this makes intuitive sense in this case as well (not that intuition has always led us to the correct conclusion). For if CoD is a proxy for value, and one feature has more value than another and it’s the same effort, we do it first; we knew that already from our ROI Value/Effort proxy. So we have our second conclusion:

If two features have the same effort, do the job with the highest cost of delay first.

Weighted Shortest Job First

Now that we have seen the impact, we understand that these two conclusions are quite sensible when the size or CoD of two features are comparable. Of course, we are not manufacturing widgets here. The Cost of Delay and implementation effort for different software features is highly variable. Plus, often they have weak or no correlations (i.e., some valuable jobs are easy to do, some are hard); that’s just the way it is with software. In the case where the CoD and effort for a feature are highly variable, then the best economics is achieved when we implement them in order of the Weighted Shortest Job First.

In this case, we calculate the relative priority weighting by dividing the CoD by the size estimate. If the CoD and job sizes vary greatly, then the differential economics can be dramatic, as illustrated in the figure below.

Weighted Shortest Job First

This, then, is our preferred approach for software development:

If two features have different size and CoD, do the weighted shortest feature first.

Estimating the Cost of Delay

This seems like a promising decision model for prioritizing features. It is based on solid economics and is quite rational once you understand it. However, we have excluded one small item – how does one go about calculating the cost of delay for a feature? If we aren’t careful, we could fall into the analysis trap we mentioned above – overinvesting in calculating size estimates for features – plus overinvestment in calculating CoD – could lead to too much overhead plus a potential bias by those doing the work. We need something simpler.

We suggest that CoD – so critical to our decision-making criteria – is in turn, an aggregation of three attributes of a feature, each of which can be estimated fairly readily, if compared to other features. They are user value, time value, and information discovery value.

  • User value is simply the potential value of the feature in the eyes of the user. Product managers often have a good sense of the relative value of a feature, (“they prefer this over that”) even when it is impossible to determine the absolute value. And since we are prioritizing like things, relative user value is all we need.
  • Time value is another relative estimate; one based on how the user value decays over time. Many features provide higher value when they are delivered early, and differentiated in the market, and lower value as features become commoditized. In some cases time value is modest at best (“implement the new UI standard with new corporate branding”). In other case, time value is extreme (“implement the new testing protocol prior to the school year buying season”), and of course there are in between cases as well (support 64 bit servers).
  • Information discovery value adds a final dimensionone that acknowledges that what we are really doing is software research and development. Our world is laden with both risk and opportunity. Some features are more or less valuable to us based on how they help us unlock these mysteries, mitigate risk, and help us exploit new opportunities. For example, “move user authentication to a new web service”, could be a risky effort for a shrink-wrapped software provider that has done nothing like that in the past, but imagine the opportunities that such a new feature could engender.

With these three value factors, user value, time value, and information discovery value, we have the final pieces to our prioritization puzzle.

Feature Prioritization Evaluation matrix

Now, we can integrate all this information into an evaluation spreadsheet that we can use to establish the relative priorities for a feature, based on both effort and CoD. An example is shown in the table below below.

Cost of Delay Effort WSJF
User Time Info. Total
Feature A 4 8 8 20 4 5.0
Feature B 8 4 3 15 6 2.5
Feature C 6 6 6 18 5 3.6
Legend:
Relative weighting Scale: 10 is highest, 1 is lowest.
Weight is calculated as Total Cost of Delay divided by effort

Table 1:  Spreadsheet for calculating weighted shortest job first

In our example, it is interesting to note that that Feature B – the job with the highest user value (8) and the highest feature/effort rating (1.3) –  is actually the job that has the lowest weight, and therefore should be implemented last, not first. The job that has the lowest feature value (Feature A) actually produces the highest actual return on investment, so long as it is implemented first. So much for our intuition!

All Prioritizations are Local and Temporal

Reinertsen points out another subtle implication of WSJF scheduling. Priority should be based on delay cost, which is a global property of the feature, and effort, which is a local property of the team that is implementing the feature. In other words, a job with a lower relative feature value may require little resource from a specific team, and therefore should be implemented ahead of another higher priority feature, if that feature requires more resources for that team. This means that all priorities are inherently local[3].

This occasionally flies in the face of the way we often do things, whereby management sets a global priority for a project, which is to be applied across all teams. In that case, a lower priority task for a high priority project may take precedence over a high priority task for a lower priority feature. And availability of scarce resources doesn’t even enter into the equation. We see now that this approach simply doesn’t make economic sense.

In addition, we note that our model is highly sensitive to the time element – priorities change rapidly as deadlines approach. For example: “implement the new testing protocol in time for the school year buying season” could have a time value of “1-2” on the winter prior to the next school year start, but could easily be a “10” in May of the next year.

One conclusion of the above is that priorities have to be determined locally, and at the last responsible moment. That is the time when we can best assess the cost of delay and the resources available to work on the feature.

Fortunately, in our big picture model, we prioritize features on Release ( or PSI) Planning boundaries. Therefore the cadence we established for our release train will serve us well here, so long as we take the time to reprioritize at each such boundary.

When we do so, we can be confident that our priorities are current – taking into account then-current availability of resources and then-current cost of delay, as well as being based on solid economic fundamentals.

Achieving Differential Value – the Kano Model of Customer Satisfaction

Along with his colleagues, Noriaki Kano, an expert on the field of quality management and customer satisfaction, developed a model for customer satisfaction that challenged many traditional beliefs.  Specifically, the Kano model challenges the assumption that customer satisfaction is achieved by balancing investment across the various attributes of a product or service. Rather, customer satisfaction can be optimized by focusing on differential features, those “exciters” and “delighters” that increase customer satisfaction and loyalty beyond that which a proportional investment would otherwise merit. The Kano model is shown in the figure below.

Kano Model of Customer Satisfaction

The model illustrates three types of features

  • Basic (must have) features – features that must be present to have a viable solution.
  • Linear features – features for which the investment is directly proportional to the result. The more you invest, the higher the satisfaction.
  • Exciters and delighters. These are the features that differentiate the solution from the competition. They provide the highest opportunity for customer satisfaction and loyalty.

The primary insight from the Kano model is the position and shape of the lower and upper curves.

The shape of the basic curve is telling. Until a feature is simply “present”, the investment is proportional to the result, but satisfaction remains low until a threshold is achieved. Thereafter, however, additional investment produces a less than proportional reward. The center point of this curve gives rise to what is often described as the Minimum Marketable Feature (MMF). In other words, in order for solution to be considered viable, it must contain the requisite set of MMFs. However, enhancing or “gold plating” any MMF will not produce a proportional economic return.

The position and shape of the exciters and delighters curve tells the opposite story. Because these features are unique, compelling and differentiated, even a small investment (the area on the left) still produces high customer interest and potential satisfaction. Additional investment produces still more, and proportionally more investment produces still higher satisfaction. This is where we get the greatest leverage for our investment.

Prioritizing Features for Differential Value

Given that we have already described a full-fledged Weighed-Shortest-Job-First prioritization model, the question arises as to what additional benefit we can derive from Kano’s thinking? There are three strategic takeaways:

First, when competing on features in an existing marketplace, teams should place relatively high user value (and therefore a relatively high cost of delay) on features required to reach minimal parity. This leads us to rule #1:

Differential Value Rule #1 – Invest in MMFs, but never overinvest in a feature that is already commoditized.

Thereafter, the strategic focus should move to placing higher user value on differentiating features – those that will excite and delight the users, those for which competitive solutions have no answer, and those for which an incremental investment produces a nonlinear return. This leads us to rule #2:

Differential Value Rule #2 – Drive innovation by having the courage to invest in exciters.

Finally and most subtly, when we are forced to engage in feature wars with competitors that may already be ahead of us, it may not make sense to put all our investment into MMFs. After all, our competitors will keep investing too; what makes us think we can catch up? Instead, it may be better to slight some narrow category of MMFs and, instead, focus some amount of investment on exciters – even if we have not reached full basic feature parity.

Experience has shown that customers can be relatively patient with developers when they can a) reasonably anticipate the appearance of adequate MMFs and b) see the promise of the differential value of the exciters that the team is bringing forward from the backlog. This leads us to our third and final rule of feature prioritization:

Differential Value Rule #3 – If resources do not allow you to compete with the current rules of the game, play a different game.

Summary

In this post we’ve provided a method for prioritizing features based on a weighted Cost of Delay. It’s not so different from prior models, but it delivers superior economic returns, and it includes attributes of estimate size (cost), potential return and adjustments for risk (information discovery). We’ve also described the Kano Model which leads to an appropriate bias for investment in exciters and delighters. It may just be common sense; or perhaps its common sense buttressed by the real, yet subtle,  economics of product development flow.

Comments welcome.


[1] There was once a Dilbert cartoon on this topic. The developer says to the customer, “please tell me what your priorities are so I’ll know what I don’t have to work on.”

[2] Principle of Product Development Flow E3: if you only quantify one thing, quantify the Cost of Delay

[3] Principle of Product Development Flow:F18: Prioritizations are inherently local.

Reference: Reinertsen, Don.  The Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing. 2010.

Estimating Features

In various categories of this blog, we’ve described the role that Features play in defining system behavior at the program level of the Big Picture.

Program Level of the Big Picture

In writing the upcoming  agile requirements book, I needed to pull together various hints and tips I’ve provided on estimating features, into a more comprehensive view. This post is a preview of some of that work. I’m pushing it now so as to elicit comments (any maybe help someone actually estimate features in the meantime).

Introduction

Estimating effort at higher levels of abstraction (features and epics) builds on the basic estimating model teams used for estimating user stories, so a brief recap may be useful:

  • Teams typically estimate user stories using an abstract, relative estimating model based on story points. Story points can be measured in the abstract (unit-less but numerically relevant) or as Ideal Developer Days (IDDs), with the abstract measure being the most common.

(Note: For simplicity, many teams start with a basic model whereby a single story point is estimated to be one ideal developer day (IDD). Teams often allow 8 IDD’s per two week iteration, leaving two days for planning, demos, and other project and non project related overhead. So for many teams, a story point eventually equates to an IDD, which tends to normalize the team to a common baseline and simplifies cost estimating as well.)

  • The aggregate amount of story points that a team can deliver in the course of one iteration is the team’s velocity. Velocity represents the amount of user value (number of story points) that a team can deliver over any particular time box.
  • When a team’s size changes, or vacations or holidays occur, they simple adjust the expected velocity accordingly.

After some number of iterations, teams will generally have a fairly reliable velocity. That allows them to make intelligent commitments for each iteration. It also provides the basic mechanism we need for estimating at the program level.

Estimating Features

Depending on where the item is in the program backlog, and how important the estimate is, a feature estimate can go through a series of refinements as the figure below shows:

Successive Refinement of Feature Estimates

Preliminary:  Gross, Relative Estimating

For initial backlog prioritization, the product management team may simply need a rough estimate of the effort to implement a feature, even before discussion with the teams.  If so, they can simply use the same, relative estimating model – the “bigness” of each backlog item is simply estimated relative to others of the same type. This provides a simple and fast estimating heuristic that can be used for the initial (but again, only relative) scoping and prioritization of work.

Refined: Estimate in Story Points

At some point, it may be useful to refine the effort estimate by converting the feature effort estimate into story points. With story points the teams can start to reason about potential cost and schedule impact. For this, we need tracking information that allows us to convert features into story points. Fortunately, many agile project management tools support the feature-to-story hierarchy and the teams can leverage this to build some historical data. A simple, comparison of the new feature to the expended story points for a similar size feature provides this first refinement. This is still a fairly gross estimate, as it depends on comparing the new feature to like features for which there is historical data, but at least it’s in a currency that can be used for further analysis.

Final: Bottom-up, Team-Based Estimating

The estimates so far require only a minor time investment and they can be done by the product management team in isolation. That can be appropriate, based on the stage of the feature.

However, for any meaningful estimate, the fidelity of the estimate must be significantly improved by having the estimating done by the teams. In any material program, there will be multiple teams, and they may or may not be affected by the new feature. Sometimes, only they know whether their module, feature, or component is impacted. Therefore, only they can actually determine a more responsible estimate. They will typically have their own history of like features – and the story points it required to complete them – in their project management repository.

However, since ad hoc requests for estimating interrupts the team from their daily iteration activities, the estimating task is most efficiently done on a cadence. Many teams have a periodic  story/feature preview meeting, which is designed, in part, for this purpose.

Estimating Cost

Once the feature has been estimated in the currency of story points, a cost estimate can be quickly derived. While the development teams themselves may not have ready knowledge of the cost a story point, at the program level it is fairly straightforward to calculate one. Simply, look at the burdened cost per iteration timebox for the teams who provided the estimates, and divide that by their velocity. This gives an estimate of the cost per story point for the subject teams affected by the new feature.

Additional work may be necessary when teams are distributed in differing geographic locations, as that can result in a highly varying cost per story point for individual teams.

Estimating Delivery Time

Finally, given an understanding of what percentage of the team’s time the program is willing to devote to the new feature, we can also use historical data to predict how long it will take to deliver. For example, if Feature A was implemented in about 1,000 story points and took three months; and Feature B is a little bigger, then Feature B will take a little over three months, assuming a comparable time allocation.

As a further refinement, the program can also look at the current available velocity of the affected teams, make some assumptions about percentage time allocation, and derive a better time and schedule estimate from there.

Conclusion

That’s it for this post and I’d really appreciate any comments people have on the feature estimating model I’ve outlined above.

The next post will most likely be focused on prioritizing features, or maybe I’ll get back to the kanban model for architectural epics, depending on the wind and the weather.