An Agile Architectural Epic Kanban System: Part 2 – The Model

In the last post, I described the context and objectives for a kanban system for identifying, analyzing and implementing architectural epics in the agile enterprise. This work is being developed in collaboration with Santeri Kangas and others of F-Secure Corp., who are contributing to the model and applying it in their agile development shop. In  this post, we’ll introduce the proposed model.

For additional context for this work in process, architectural epics are first class citizens of the agile enterprise in the Big Picture Series (see below)

And the epics themselves are described in the Lean and Scalable Requirements Model:

In the last post, we described the role of system architects and “intentionally emergent architecture” in enterprise class agile development. We also described four primary business factors that drive new, cross cutting, architectural epics:

  1. New product or service opportunities.
  2. Changes in technologies that affect multiple products and services.
  3. Some architecturally-centric problem within the existing solution set.
  4. Common infrastructure and avoidance of over-investment.

In order to assure that we address the development of architectural runway in a lean and agile manner, we proposed using a kanban system. We also described the three primary objectives for such a kanban system:

  1. To make architectural work in process (AWIP) visible
  2. To establish AWIP limits to help assure product development flow
  3. To drive an effective collaboration with the development teams

Introducing the Architectural Epic Kanban System

Of course, we probably wouldn’t have invested this time if we didn’t have any idea how such a system might work, and indeed, we have been developing it for the last few months. And, as readers of this blog already know, I like to express concepts visually first, so the reader doesn’t have to pour through a ton of text just to build their own mental model for what it is, and how it works.

So, here is another big picture graphic that highlights the relevant elements of the proposed system.

Architectural Epic State Descriptions

The underlying assumption for the kanban system is that epics go through a series of four states, each characterized by different activities on the part of the architecture and development teams and correspondingly increasing levels of investment. They are as follows:

Problem/Solution Needs Identification: The Funnel

The Funnel state is the “capture” state. In this state all new “big ideas” are welcome. They can come from any source. They need no business case or estimates. Tooling is trivial – a document , spreadsheet or simple list on the wall will typically suffice. Since investment is minor, this state is not WIP limited; all ideas are captured for consideration. Funnel epics are discussed periodically. Based on decision criteria, epics in this state can be a) promoted to the next state, b) left in this state, or c) deleted.

Analysis: Backlog

The Analysis:Backlog state is more meaningful; epics that reach this state warrant further investment of time. These epics are roughly sized (we’ve indicated t-shirt sizes to avoid overinvesting in estimates). Investment is controlled to discussion level, and perhaps some very preliminary and lightweight investigation. The epic may be elaborated to a paragraph or two.

Since the investment is increasing, this state is WIP limited, primarily by the capacity of the architecture team.

Analysis:Backlog epics are discussed periodically. Based on decision criteria, epics in this state can be a) promoted to the next state b) demoted back to Funnel state, or c) deleted.

Analysis: Evaluation

Epics which are promoted to this next state deserve a more rigorous analysis, and require further investment. An architect is typically assigned as the epic owner. An active collaboration with development is initiated. Design alternatives are explored. Options for internal development and/or outsourcing, including 3rd party solutions acquisition, are considered. The affected development teams may estimate the epic. A lightweight business case, with a go or no-go recommendation, is developed. If 3rd party solutions are evaluated, the recommendation is given in priority order.

This state uses scarce resources, so it is WIP limited based on capacity of the architecture and development teams.

Epics that meet the go criteria are promoted to Implementation; otherwise, they are deleted from the queue.

Implementation

In this state, the epic is moved to implementation and the primary responsibility for the epic is passed to the development teams. This hand-off is effected when the epic appears in release planning. Architect resources remain available on a “pull” basis. (i.e. the responsibility for implementation rests with the development teams, but the architect assists the teams and shares responsibility until the team has developed an understanding of the work required.

This state is WIP limited by the capacity of the development teams and architect’s.

Promotion from State 3 to state 4 is an important economic decision for the enterprise that can be made only by the appropriate authority, based on the developed business case.

Summary

In this and the prior post, we’ve introduced the motivation and the model for an architectural epic kanban system. In follow on posts,  we’ll develop the model further.

7 thoughts on “An Agile Architectural Epic Kanban System: Part 2 – The Model

  1. Ver y interesting adaptation of agile! I really like the idea of limiting the size of architectural and visionary work in order to avoid waste. Waste on every level is just it, waste.

  2. Great post you constantly inspire me. I will be sharing this with our chief architect and hope to build a similar model.

  3. I am adding the same comment I added in the previous post – should have read both before responding! Will add additional thoughts in a second comment as well…

    Great series! Looking forward to future posts on this subject. Some thoughts to consider

    Innovation – the concept is addressed indirectly but it’s something you may want to consider as one of your existing four our a fifth business driver. Creating new ideas / products is a form of innovation but more often than not, it’s incremental vs. revolutionary. A seasoned architect team can really enable / help shift a company’s direction. For example, what if… you had a strong business development team that also was allowed to feed in new ideas into the architect’s pipeline? You now have enabled an increased ability to pursue new markets, etc. with the right technical focus. This even includes acquisitions, etc.

    Visibility – such a critical concept and one that fits well in this approach. All to often, “architectural” work is a hidden / misunderstood concept that gets carved out of the capacity of a development team. This on the surface makes it easier for the development organization but what it’s really doing is causing outside stakeholders to question why things take so long and in turn, it’s devaluing the role / importance of architecture.

    Collaboration with Development – this is perhaps one of the most critical items. I love how this model prevents the “architecture from on high” issue that so often criples motivation and buy-in at the development levels. In addition, it keeps architects sufficiently hands on with the new technologies to enable them to continue to grow. This in turn prevents a second key issue “design without constraints”

    Nice job guys!

    Mo

  4. Another great post – here are some quick additional thoughts:
    * Big ideas – I still believe they should always be framed with business context. While you may not need a formal / traditional business case, you do need to be able to sell your idea. Ultimately one big idea w/ strong business context will win over another “just” big idea
    * Backlog concept – what a great way to help external stakeholders understand the value of architecture and the potential need for additional architects!
    * Throughput – one key construct to never lose sight of is the concept of a quick feedback loop. It would be worthwhile to measure the begining to end progression of an architecture epic. We must continue to be agile in principle and while I think this is a great framework, the wrong type of thinking could easily turn this into a more analysis/paralysis waterfall pipeline. For folks that understand what is being proposed, it’s not an issue. For folks just embrassing agile, it’s potentially dangerous because I could only imagine trying to explain this concept and why it’s different to a traditional waterfall architect

    Nice job again!

    Mo

  5. Hi Dean,

    I’m a big fan of kanban development and lean. Thanks for your contribution to helping us scale it for larger systems and organizations.

    However, I’m also a fan of context-driven school, and believe that all models are wrong, and only some can be useful. All models are incomplete and compromise. I think your models needs some clarification before I could discuss it internally in my enterprise.

    First, I assume that this model tries to depict the desired state, not the current state. Thus I assume the model is not “tested in production” yet, and needs more clarifications about the underlying circumstances and assumptions, in order to trying to apply the model in different contexts.

    However, while I think this model makes the work in progress (AWIP) visible, and possibly enables to limit the wip to capacity, it does not look optimal in terms of the flow. At least to me it seems that there are a lot of possibly unnecessary queues and hand-offs in the model, which prevents the ultimate flow: 1. funnel-queue, 2. architectural-approval-queue, 3. backlog-queue, 4. evaluation-queue (including potentially many internal queues), 5. product-council-approval-queue, 6. implementation-queue (including potentially many internal parallel queues due to breaking epics). Are all these queues and hand-offs between e.g. architects and devs really necessary? What are the assumptions and conditions behind all these queues?

    Second, I have a problem with the concept of “architectural epic”. Could you please elaborate on that a bit? Again, the architecture in software is a model, and all models are wrong… and ultimately the source code is the design in software. Also, all development should have the goal of adding value to the customer. Having architecture-centric items in the product backlog makes it very difficult for the business to make trade-offs against customer-centric items.

    BR,
    Anna

    • Anna,
      Thanks for your perceptive comments. However, I think you overstate the case. (probably because I haven’t elaborated the model yet).

      1) there are only three logical backlogs, funnel, backlog and analysis. We actually debated deleting the funnel stage, but the folks at F-Secure thought it was helpful to separate it and make it really easy to get something in the funnel. Of course, then you have to work to reduce the list, which is why we have the Backlog state.
      However, its just a model so I suspect some will delete this state; others will add more states. Moreover, these states could easily be physically implemented as a single backlog with a state attribute, if tooling is a concern. (BTW The implementation backlog is not a new backlog, the teams already have that.)

      2) You are correct in that this is a model of a future state and is unproven. However, it is being applied at F-Secure now and we’ll for sure learn some new things. But the kanban method for software development is well proven, and these are just architectural epics, not user stories. So I suspect the kanban lessons learned will apply well here.

      3) there is only one hand off, that is to the development team for implementation, and that occurs after the collaboration in the analysis state.

      4) an architectural epic is just a big cross-cutting thing that requires many teams to redesign, re-implement or refactor their product, feature, or component. Examples include:
      – build a framework for porting 30 existing mobile applications to a new user interface paradigm
      – use a common installer for all products
      – double byte enable a suite of products to enable future localization
      – adapt a trading transaction back office server to support 64 bit servers
      – support Windows 7
      – replace a search engines underlying data base with My SQL to support rapidly increasing user loads
      – implement the new GUI style standard, corporate logo and copyright language across all user facing web and PC apps

      Of course we wouldn’t do it if it doesn’t deliver user value, but the effect is less immediate and there is no user involved in the immediate work.

      And yes, it is harder to prioritize a backlog that has these things in it than one that doesn’t, but that’s just the nature of the beast. If we don’t have the debate at backlog prioritization, then only immediate user value items will get implemented. We know how that works out.

      -Dean

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s