A Standard Iteration and Release Pattern for the Enterprise?

I’ve discussed “around this topic” in the book and in various posts as well as the classroom environment, but it occurs to me that I’ve never directly proposed nor published a standard pattern for iterations and releases for the enterprise. While “standardization and agile” are unlike things in many agilists minds, I often fall back on the mantra of make routine that which can be routine in order for the company and teams to be able to focus their energy, creativity and initiative to flex about the things that cannot be made routine. (A daily stand-up is just such an example, meet routinely every day to discuss the events of the prior day and the upcoming day, so as to avoid having to schedule ad hoc meetings throughout the day).

Moreover, I’ve also recommended that agile teams focus on high frequency internal releases, and not worry so much in advance about what does or does not go to the customers or market, as there are a variety of outside factors and other stakeholders who make that determination. (Chapter 20 – Impact on Customer and Operations). Along with that guidance, we have seen or espoused other recommendations that include:

• Standardize on a short iteration length – my primary recommendation – two weeks
• Allow for one hardening iteration at the end of a release cycle. These are iterations devoted almost exclusively to technical debt or additional testing backlog (such as system and performance tests which may require resources not practical in the daily build environment, testing on secondary platforms, etc.), or perhaps finalization of documentation and support materials, etc.
• Push product to the market as frequently as practical – typically 60-120 days maximum

Given these inputs a standard pattern for internal releases can emerge as seen in the figure below. Note: some of these internal releases may morph to become external releases, but the teams do not typically have to worry about that too far in advance, especially if they are just starting down the agile enterprise path.

I’ve used this pattern successfully to kickstart a number of larger enterprises. With some experience, the enterprise will likely modify this pattern it to better suit its purpose over time, but it’s a great starting point.

Role of the Product Manager

I’ve been blogging off and on the “agile product manager vs. agile product owner” roles over the last few months. (These are categorized under Product owner/Product Manager) In one post (Responsibilities of the Agile Product Owner vs. Enterprise Product Manager) we noted that within the enterprise, it’s not always appropriate nor feasible to have product managers fill the role of the agile product owner, and we noted that organizing the roles around the release and iteration boundaries/ownership often make sense. See below.

When this happens, while the activities of the product manager role evolve in an agile enterprise, so long as the product owner assumes the basic iteration responsibilities, many aspects of the enterprise product managers role are largely unchanged.

I just came across an article I wrote in 2002 which was my articulation at the time of the growing influence of product managers and their responsibilities within the software enterprise. I’ve posted the article below and on the resource page as well. I think it is still largely relevant to the discussion at hand.

The Role of the Product Manager in a Software Products Company

Release Planning Day 1 Schedule (AERP4)

Note: this is one of a continuing series of posts, categorized under Release Planning.

In earlier posts, I’ve described the critical role that release planning plays in enterprise agility. I’ve described the motivations and objectives (AERP1 and AERP2) and planning and preparation (AERP3).

Over time, I’ve developed a somewhat standardized, two-day “program” for such an event which has served a number of teams quite well. This program provides a draft, tried and true agenda that your teams may want to start with. Of course, as with all things agile, there is no one size fits all, so you should feel free to try it as is, modify it, or throw it out and plan your own event. In any case, I’m confident it will be an important learning experience for the enterprise.
Here’s what Day 1 looks like:

Perhaps you can see from the end of the program, that one day doesn’t always quite get it done; that’s why we have reserved Day 2.

Register for the London Course

I’ll be teaching the one-day course, Scaling Software Agility: Best Practices for Larger Enterprises, March 27, 2008 at the Southwark Mercure London City Bankside Hotel, 71-79 Southwark Street
London from 9:00-17:00. The hotel is a 10 minute walk from Waterloo or Waterloo east, and a five minute walk from the Southwark tube stop.

The course is filling out and there will be a number of London-based, large software enterprises in attendance, so the shared learning environment will be a real benefit. There are a few slots left, so register here if you’d like to attend.

Hope to see you there.

Dean Leffingwell

Organizing and Planning the Release Planning Event (AERP3)

Note: this post is part of a continuing series on Release Planning. The main posts in the series are organized and numbered sequentially (AERP1-n), and there are additional posts under the Release Planning category as well.

Participants

As we have described, (Release Planning: The Agile Enterprise Main Event (AERP2)) the release planning meeting is the seminal event in the enterprises agile process and representatives from all affected teams should attend. (For larger enterprises, there will be a sequence of such meetings, organized around cooperating components or subsystems.) Other stakeholders (executive sponsors, other departments, customers, etc.) who will be affected by the outcome should also attend. The meeting is typically a two-day face-to-face event with attendees including:

• Agile Masters, team leads, Scrum Masters
• Product Owners/Business Analysts
• Product Managers and Customer Representatives (latter where feasible)
• Tech lead and representative team members: developers, QA/Test, documentation, etc.
• System and team architects (if applicable)
• Release or project managers (where applicable)
• Engineering Development Managers
• VPs and executives responsible for product strategy, delivery and/or other affected operations

Preparation

A significant number of team members have the responsibility to prepare for the release planning meeting. Organized by role, these responsibilities fall roughly as follows:

Product Managers and Product Owners

• Understand status of current release in process
• Prepare draft release plan, dates and themes. Where multiple product managers are involved, meet ahead of time and solidify a single vision and a single set of priorities
• Prepare a product backlog consisting of prioritized features
• Prepare vision presentation for the meeting
• Be open to the negotiations/scope management that will likely occur

Development Teams

• Organize and update the current product backlog (items not in current release)
• Decide on story estimating mechanism
• Understand current team velocity (work capacity) and any upcoming schedule constraints (vacations, holidays etc)
• Understand and prepare to discuss current architectural/infrastructure status and initiatives

Engineering Managers/Release Managers/Project Managers

• Plan schedule (1-2 day event) and logistics (main event room with lots of wall/board space, plus break-out rooms)
• Invite and assure appropriate attendance
• Coordinate executive preparation and presentation
• Determine program and facilitator/facilitation process
• Assure coordination and readiness of product managers, architects etc prior to event

With this preparation under way, I’ll describe a program (sample agenda and sequence of activities) in a post later this week.

Release Planning: the Agile Enterprise Main Event (AERP2)

Note: This post is part of a continuing series where I’ve been discussing the critical role that release planning has in enterprise agility.

These seminal release planning events are one of the key mechanisms the enterprise can apply to use its emerging agile practices to drive a coordinated and directed strategy into the marketplace – a foundation for the Agile Enterprise.

I’ve discussed release planning extensively in the book and have more recently emphasized Release Planning as an answer to a number of critical questions in the following posts:

You Might Need Better Release Planning if….

Yes, Enterprise Agility is Hard

Enterprise Agility is Hard, but Release Planning Helps (AERP1)

In a nutshell, release planning is to the enterprise what iterations are to the team, i.e., the basic cadence, a critical best practice and organizational metaphor that holds all the other practices together. For without iterating (working software every two weeks), the teams are not agile. Without effective release planning, effective agile teams may well drive the enterprise to distraction – lots of pressures for improvement, increased productivity – but the enterprise itself may not yet have the ability to more quickly drive the right solutions to the right customers in the marketplace.

However, as with all things agile, when one considers the need for even doing release planning, which implies a certain amount of cost and overhead, we are reminded of some fundamental agile (manifesto) principles:

1. the most efficient form of communication is face-to-face
2. the best requirements, architecture and designs emerge from self-organizing teams
3. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

And of course, the basic mantra,
“what is the simplest thing that can possibly work?”

These principles drive us to the conclusion that a periodic, one or two day, face-to-face meeting which gathers the right stakeholders together is the enterprises “Main Event”, an event intended to address the following objectives:

• Build and share a common vision
• Communicate market expectations, features and relative priorities for the next release(s)
• Reflect and apply lessons learned from prior releases
• Estimate and plan, in real time, the next release
• Gain a commitment from the team to deliver the next release
• Communicate any governing requirements such as standardization, internationalization, usability, etc.
• Communicate and coordinate any major architectural initiatives, refactors and the like
• Evolve, refine and communicate the product roadmap which the company uses to communicate to its customers and other stakeholders

In addition for those enterprises new to agile, this is an important opportunity to communicate to all stakeholders (particularly the executives) what the current priorities are, how the resources are allocated and reallocated based on priorities, and how the new agile processes are applied to deliver software to the market

Frequency of the Release Planning Event

The frequency of the event is dependent upon the company’s agile iteration and release patterns. We’ve recommended iterations of a maximum length of two weeks and internal releases (internal synchronization points that are potentially shippable, but decoupled from the general availability/customer consumable release) of a maximum length of 60-90 days. (By decoupling the internal releases from the external ones, we provide a degree of freedom that allows internal release planning to be scheduled and periodic, and the engineering teams to have a rapid delivery cadence, independent of any more public commitments (SLAs, public announcement venues, etc) the enterprise must make.)

So for many companies, an internal release cadence consisting of three “development” iterations plus one “hardening iteration” could look as follows:

With this (fairly rapid) cadence, release planning events would be generally be scheduled immediately before or after the internal release milestones. In practice, we’ve found that it doesn’t really matter exactly when, so long as a) the frequency generally matches that of the internal release and b) it happens before the last responsible moment, i.e before the release begins. We’ve also observed that for larger organizations, the 60 day internal release frequency is a big bite, and often we see it at a more leisurely 90-120 days. This may seem long to an agilist, but pretty fast to an enterprise that might have been releasing as infrequently as annually.

With this as context, I’ll be describing the preparation and agenda for the release planning event in posts throughout this week.

Does Development Productivity go DOWN before it goes UP?

One of the challenges I often have when talking to industry executives about a significant enterprise agile transformation is that they are usually pretty smart! After all, to have achieved their current position, they have probably been around for a while and they have likely delivered some hundreds of millions of dollars in software products or services to the market. In so doing, they didn’t get there with blind-eyed enthusiasm for every technology wave that they have seen over the decades. They may be looking for change, they may not, but in any case they are not absolutely certain that they should be looking at this change, and at this time. After all, many of them are well aware of the technology hype>disillusion>enlightenment curve that has been around for as long as many of us can remember. Here’s my memory of such a curve.

As I said, this curve has been around a long time (one technology/sales presenter I saw back when said it had been around since the ancient Greeks, where it was called the “Curve of Pystophenese”, perhaps referring to the trough part of the curve) and it is usually a safe assumption that any new technology follows this disruptive profile.

Is that the case with agile? Must we suffer a decline in productivity before the benefits accrue? If so, that could be a hard sell because in all likelihood, we are not delivering as much as we hoped at the moment, so who can afford to take even a small step backwards before the productivity gains kick in?

Fortunately, in my personal experience, the answer to the big question is an equally big NO. However, based on my decades of experience as one of those experienced (if not jaded) executives above, this causes me to question as to whether I am potentially being blinded by my own dogma. (“let us not be baffled by our own BS”, I often remind myself.)

But indeed, it is not my personal experience that productivity goes down before it goes up. Two reasons/cases in point.

1. For example, one experience that I remember well was mentoring a software project in deep trouble at a time when my agile skills were really new and untested. It was an emergency, so there was no time for intensive or even appropriate training; I just threw the team into a Scrum-like daily inspection and management cycle and insisted on the delivery of working software every two weeks. The intensity of focus parallels the Meeting Deadlines: Tiger Teams Vs Agile Project Management Practices I described in an earlier post and the results were predictable – the intensity and focus went up, multiplexing was lowered, scope was managed, defects were reduced and delivery and quality went up in just a few weeks.
2. From a more abstract, best practices view, if a team instills both the Scrum like management practices above plus some improved technical practices, for example, concurrent testing (or TDD), or more simply just mandating that all stories are tested before they are demo’d and accepted, would productivity go up or down over the near term? I would posit “up” because we know that the delayed defect discovery-reset context-fix-retest cycle is one of the least efficient processes in all of software development.

These are just two reasons why productivity can go UP, not DOWN. And there are more (like reduced team member multiplexing).

Now having said that, I’m not saying that in every agile adoption instance, productivity will first absolutely go UP, rather than DOWN at first, as I suspect there other circumstances that are quite different and perhaps not every implementation applies the basic agile practices with the appropriate fidelity. But I would say that that productivity doesn’t necessarily have to go DOWN, solely because the curve of Pystophenese exists. A careful and experienced-based agile implementation should help assure the former.

Yes, Enterprise Agility is Hard – But Release Planning Helps (AERP-1)

Note: This is the first post (AERP-1) in a series on Agile Enterprise Release Planning which I hope to continue over the next few weeks. They’ll be labeled AERP1-n, so you can eventually read them in sequence, front to back, if you like.

From the last post, Yes, Enterprise Agility is Hard, I noted:

“I’ve also come to the conclusion that effective Release Planning (agile, synchronized, multi-level release planning that is) is one of the critical keys to understand and implement agility effectively at enterprise scale. While I covered it somewhat in Chapter 12 – Smaller, More Frequent Releases and in Chapter 18, Systems of Systems and the Agile Release Train, I didn’t emphasize it as a significant and somewhat stand-alone enterprise practice that I now see it to be. To address this shortcoming, I’m writing an article on release planning from the inside out, i.e. an experiential, day-in-the-life emulation of a team lead experiencing an enterprise release planning session. This article should be published in the March 12 edition of the Agile Journal, Sharing Agile Successes. In addition, I intend to describe it more succinctly in some upcoming posts this week.”

When transitioning an enterprise to agile, I don’t generally start with the larger enterprise practices (Part III of the book), rather we (meaning me and a number of other coaches I know) typically start teams in what I call “heads-down-iterating” mode. Getting started with agile is difficult enough that first things must come first, and first, development teams must master, or at least have experienced, the basic project management and engineering practices of software agility. Forget the long term objectives for now, focus on what you can deliver and how you go about doing it in an agile, incremental, and quality-embedded way. After all, you can’t build an agile enterprise out of non-agile teams. And you can’t build a big, high quality system out of quality deficient little bits.

To start, many coaches start out by just getting the component teams organized and starting them iterating. This requires some basic reading at a minimum, and ideally some coaching and training. It also requires some time – at least 60-90 days – just to get the basic patterns down, and as I noted in my last post, even longer to master.

In order to help the teams succeed, companies often need to ignore many of the existing process and governance practices, or at least provide a short amnesty for those teams while they experiment with the new model. After all, an agile team can’t run very fast while trying to conform to non-agile milestones like “test plan complete” or “Milestone 2’s 50 point checklist”, or “software requirement specification approved”.

This problem/state of the enterprise is also paralleled in larger companies who are starting to see agile growing from the “bottom up”. Teams are doing stand-ups, software seems to be working and available more frequently and something is definitely different afoot. But in all likelihood, the types of documents, reviews, milestones etc. the company has insisted on prior (we respectfully note that all of which were originally designed to help, not hinder teams on their quest for quality and productivity) are not being followed well, or even at all. This causes a lot of consternation in the executive ranks who are responsible for governing these activities. After all, they are ultimately responsible for the outcomes and their jobs, if not the company’s future, hangs in the balance. It’s also possible that the agile teams may have “thrown out the baby with the bathwater” (for example, a corporate governance requirement that “all components must be subjected to a threat assessment review” in order to assure newly developed software meets security requirements, cannot be safely ignored).

At about this time in the evolution of the company, the entire agile initiative is threatened and if this stage of development is not recognized and addressed, it will “die on the vine” right then and there.

Moreover, the example above is just one example of the many concerns that are appropriately raised at that time. The question naturally arises in executive offices: “If this agile thing is to work, how do we answer the types of questions that some of our original practices were designed to address? Questions such as:

1. What is the vision the teams are currently working against?
   (“after all, there is likely no marketing requirements document or software specification, so how do we know what they are doing. Even if we trust them, isn’t it our job to take this company in the right direction?”)
   
2. What will we ship next and when will we ship it?
   (there is not likely to be a pert char around anymore. Even worse, some agile teams might be telling management, “we’re agile now, we’re adaptable, we’re empowered, and we don’t have to tell anyone what we will ship or when”)
3. What longer term architectural initiatives must be addressed across multiple teams?
   (“if architecture emerges, how do we impose common infrastructures, usability guidelines, security solutions, etc”)
4. How do we communicate common requirements across all teams?
   (example: “Can they all decide on their own what programs get internationalized, in what languages and when? That will be chaos.”)
5. How and when do the teams coordinate their interdependencies?
   (“we know they should work together, but they are too big or too distributed to make that natural. Are they talking about the right things? After all, they aren’t sending us plans, documents or status reports anymore”)
6. How are our resources allocated to the important, prioritized initiatives?
   (“How do we know whose working on what and whether the priorities are being addressed in a way to assure ROI. We don’t see documented plans any more, so how do we know who is working on what and when”

Fortunately, these are the types of questions that Agile Enterprise Release Planning (AERP) is designed to address. And we must address them, or our initiative will likely fail. So we’ll look further at this important practice in upcoming posts.

Yes, Enterprise Agility is Hard

In my continued work with some of the larger software enterprises, sometimes I am still surprised at just how difficult mastery of agile practices can be. (Of course this affects only those concerns with the courage to try!) And it’s hard twice:

Once – at the team level, as the individual component teams struggle to master the basic agile practices. In order to be successful, all cooperating teams must adopt and adapt the basic agile project management practices (mastering iterations, component teams, daily stand-ups, individual and team accountability, complete visibility and the like) as well as the software engineering practices (shared code ownership, cross training, concurrent testing, incrementalism, continuous integration and the like). Each of these taken alone can be difficult in itself, but when taken together the challenges mount up. Further these changing practices drive additional changes in team and organizational behavior, team and individual responsibility, personnel and role changes. In my experience it takes many months (4-6-8 or more) for a team to get “in the flow” where they can reliably deliver potentially releasable software in small increments, and it’s not unusual to see teams take up to one year to really be on their game.

Twice – at the enterprise level, when the impact of improved team productivity starts to affect the rest of the organization. That’s when the company comes to realize that there are additional changes that need to be addressed. These include extended enterprise practices (requirements and architectural governance, multilevel release planning, impact on customers and operations, enterprise-scale rollout and the like). Moreover, many of the method and procedural practices we have used in the past to address these larger concerns have been abandoned by the agile teams. There is not likely to be an all encompassing pert chart, critical path analysis, marketing requirements document, project manager, software requirements specification, written test plans, documented architectural models, etc. in existence to guide the company through the challenges. And if they do exist or the teams are forced to create them, they’ll likely and rightly be rejected by the teams as the wrong way to address the remaining problem set anyway.

Fortunately, all is not lost and there is wide and growing body of experience that helps the enterprise move forward.

At the team level, there are dozens of great books on agile development and they are generally lightweight and accessible. In addition, the growth of the Scrum Alliance and its certification process has put hundreds (maybe thousands by now) of agile coaches into the field to help the nascent teams. And as the movement has grown, in all probability there is some number of individuals in the company with substantive agile experience and those people can be the flag bearers and coaches for the new teams. So bottom up, team by team, there is a lot of guidance for the team. That doesn’t make it easy but it does make it practical.

At the enterprise level however, there is much less documented (see reference list of Leffingwell – Scaling Software Agility: Best Practices for Large Enterprises, Eckstein- Agile Software Development in the Large and Schwaber- The Enterprise and Scrum) on how to scale agile to the enterprise level. (Indeed, one sees some agilists who would debate whether that is even the right thing to do.) In addition, there are far less teams/companies etc. who already “get it” from their prior experiences. So it is newer territory. Of course, that’s why I wrote the book

As for the latter, while there can be no substitute for experience and the full set of best practices in Part III of the book, I’ve also come to the conclusion that effective Release Planning (synchronized, multi-level release planning that is) is one of the critical and relatively straightforward keys to understanding agility at enterprise scale. While I covered it somewhat in Chapter 12 – Smaller, More Frequent Releases and in Chapter 18, Systems of Systems and the Agile Release Train, I didn’t emphasize it as a significant and somewhat stand-alone enterprise practice that I now see it to be. To address this shortcoming, I’m writing an article on release planning from the inside out, i.e. an experiential, day-in-the-life emulation of a team lead experiencing an enterprise release planning session. This article should be published in
March 12 edition of the Agile Journal, Sharing Agile Successes. In addition, I intend to describe it more succinctly in some upcoming posts this week.

You Might Need Better Release Planning if…..

The following bit I wrote was just published as the Tip of the Month in Agile University’s newsletter for February.

• IF your agile team is just getting into the flow in a nascent agile enterprise, and if your teams have been head down so long meeting near term iteration objectives that they are starting to ask about the bigger picture – then there is something wrong with your release planning process (inadequate team vision)
  
• IF your company organizes special projects to emphasize new initiatives and project managers spend time meeting about them – then there is something wrong with your release planning process (new epics not factored into release commitments)
  
• IF late discovery of interdependencies amongst teams prevent increased velocities of system- level deliveries – then there is something wrong with your release planning process (interdependencies not considered in release commitments)
  
• IF your team invests too little time in longer term architectural initiatives and big refactors – then there is something wrong with your release planning process (architectural runway given insufficient consideration in release commitments)
  

IF you and your teams find yourselves experiencing any of the above problems, don’t worry so much about the symptoms, focus on improving your release planning process.

IF you’d like to learn more on this topic, Agile Release Planning (synchronized, multiple-level release planning for agile teams of teams building systems of systems) is one of the critical, but little understood, enterprise skills I’ll be covering further in my upcoming Agile University Course, Scaling Software Agility: Best Practices for the Large Enterprises, which will be offered in London (Southwark) on February 27, 2008 and in Boulder, Colorado on March 18, 2008. A free copy of the book will be provided to all attendees who register for the course.

Also, I’ll be blogging about the value and mechanics of this important skill in the weeks ahead.