Agile Release Train: A Whitepaper

Recently, I’ve been working with a  software enterprise helping them tune their agile process to better align the agile teams to the broader enterprise objectives. One particular subgroup has twelve agile teams, each of whom has a specific product mission in the marketplace. (Think “point products”). However, they must now also cooperate on building their application so that users of multiple products have additional utility based on integrations and extensions to the point products. (Think enhanced features for users of the new “business suite”). In passing, one executive commented that the teams reminded him a bit of the “Twelve Tribes of Israel, each wandering through the desert in search of their own piece of the promised land.”

This was reminiscent of my time at Rational Software, where after we had acquired a number of products (ex: RequisitePro, ClearCase) and built some new ones from scratch (ex: ClearQuest), our business problem moved from being point product focused to an emphasis on the suite (Rational Suite). Over a year or two, we were able to successfully address this problem by

  1. building a more effective, aggregate product management function, and
  2. implementing our version of an early Agile Release Train, which synchronized delivery of all products to the market under a common schedule and common installer.

When I started working with agile at scale in 2004-5 or so, I used that experience to further develop the principles of the Agile Release Train, which became the topic of Chapter 18: Systems of Systems and the Agile Release Train. I’ve depended on these principles and the adjunct, Release Planning, in virtually all my work at scale. But it also occurred to me that while the Agile Release Train is assumed in the Big Picture series, I hadn’t really called it, or its benefits, out as first class citizens in this blog. To remedy this, I’ve converted Chapter 18 to a free whitepaper in PDF format, which you can download from the link to your left, or from the resource page.

If you recognize the twelve tribes of Israel metaphor, (+/- your number of tribes of course!), you might find the whitepaper to be useful.

Agile Product Owner and Agile Product Manager in the Enterprise 5: Seeding the Product Owner Role

In prior posts in this blog series and in the Agile Journal, I’ve described the critical role the team-based agile Product Owner plays in helping achieving enterprise agility. Indeed, the role, originally defined by Scrum, is now so comprehensive in agile methods that the role is hard to separate from the practice of agility itself. I’ve also described the larger challenge of scale, noting that it is no trivial task to identify and train 10, 20 or 100+ individuals to effectively implement this role in the larger software enterprise. I also promised to describe some best practices as to how some larger enterprises go about filling this role. This is that post.

Roles, Titles and Responsibilities Vary by Software Business Type

Since the business mission, organization, operating methods, roles, titles and responsibilities differ dramatically across industry segments, it follows that the patterns of agile adoption vary across these segments also. In this post, I’ll provide real world examples of how this challenge was addressed in three primary software business types:

Information Systems/Information Technology (IS/IT) —teams of teams who develop software to operate the business; accounting, CRM, internal networks, sales force automation and the like. Customers are primarily internal to the enterprise.

Embedded Systems (embedded) — teams of teams who develop software that runs on computers embedded in other devices – cell phones, electronic braking systems, industrial controls and the like. Customers may be either internal or external to the enterprise.

Independent Software Vendors (ISV) —teams of teams who develop software for sale, including products like network management, supply chain management, mobile applications, etc. This segment now also includes the rapidly emerging Software as a Service (SaaS) vendors. Customers are external to the enterprise.

When it comes to agile adoption, each of these business types has a unique set of challenges, none more critical than the successful implementation of the agile Product Owner role. In the examples below, we’ll look at some specific case studies that may serve to guide others who head down the agile enterprise adoption path.

ISV Example 1 – TradeStation Technologies

TradeStation is the premier brokerage trading platform for rule-based trading.  They note: “whether you trade stocks, options, futures or forex, TradeStation offers uniquely powerful strategy creation and testing tools, customizable analytics and fully automated trading technology in a single trading platform.”

At TradeStation, Keith Black, VP of Product Development, and John Bartleman, VP of Product Management, have been driving a comprehensive, all-in agile transformation that affects 100+ practitioners. (Keith has already opined on the dual-roles of agile Project Owner and agile Product Manager in a prior post). John described their approach to filling the Product Owner role as follows:

“Before transitioning to Agile, our Product Management team was made up of ten product managers who reported into Development. In general, we have always had an internal/technical focus as opposed to the traditional external/marketing focus.   When we transitioned to Agile, seven of the ten Product Managers became full time Product Owners; the other three now focus on the market-facing Product Manager role.  This separation of labor and concerns has helped us bring additional focus to both the market and technical aspects of our solution.”

John then comments on the staffing challenge:

“When staffing the Product Owner role, I would have preferred to use a few lead developers and/or testers since they have the domain knowledge and technical expertise; however, we are reluctant to do this because of the impact on development resources.  Therefore, I am hiring a few additional Product Owners from outside the company. I am finding that these people need to be technical, but also need to have good industry specific experience, and that is a difficult combination to find.  So far, former developers/tech leads with business sense and good project management skills seem to be the best fit.

For example, one Product Owner we hired was a former developer/development manager with specific industry experience.  Another was a very tech savvy developer/project manager who knew our technology, but was from outside our domain. This demonstrates that, in my view at least, technical skills are mandatory, and domain experience is a plus whenever I can get it.”

ISV Example 2 – CSG Systems

CSG Systems is a customer interaction management company that provides software- and services-based solutions that help clients engage and transact with their customers. Today, CSG’s solutions reach more than half of all U.S. households each month and manage over $36 billion in transactions annually on its clients’ behalf.

In 2007, CSG began transforming their ACPx product development (>100 practitioners) efforts using enterprise agility best practices. One of the leaders of the effort, Mauricio Zamora, Executive Director at CSG, whose team is responsible for defining the ACPx vision, roadmap and agile processes leveraged to deliver the vision, noted that they established product owners through a series of phases:

“We first had to leverage a combination of Product Analysts originally responsible for waterfall requirements, Architects originally responsible for designing our software and a few Product Owners already experienced in agile execution. Over the course of a few releases, we leveraged the really good Product Owners to set the standard for others. In the process, we discovered candidate Product Owners that weren’t a good fit for the role , moved them to other slots, and then replaced them with more appropriate internal resources and a few additional external hires.”

Mauricio (Mo) went on to note that the transformation wasn’t easy and it took time to help people see all that needed to change.

“We first educated everyone on the differences between the traditional Product Management, agile Product Owner and Architect roles. We then had to convince management that the Product Owner role required dedicated focus. The visibility agile provides made the increasingly obvious gaps in Product Ownership easier to see and address. Finally, we had to revisit and revise organizational titles and compensation because the new Product Owner role didn’t map well into our existing organization”

ISV Example 2 – BMC Business Performance Manager Project

BMC Software is a leading provider of IT Infrastructure Management Solutions. In 2006 and 2007, the Infrastructure Management Group (IMG) transformed their development organization (>100 practitioners) using agile development practices to deliver “a major product to the market in less time and with higher quality than previously possible” (see BMC Case study). Israel Gat, then VP of Infrastructure Management at BMC and now a Cutter Consortium consultant, led the transition. Recently, he described how they filled the Product Owner role (which they re-titled as “Requirements Architects”) as follows:

“Between the product managers and the requirements architects we had enough “hands on deck”. We must have needed half a dozen architects to become requirements architects, and we were able to generate them. We might have borrowed a little from the project leads, but the architects were our #1 pool. We did not borrow any from the project office.”

When I pushed Israel on how he would envision a transition that involved 20-50 or more teams, he commented:

“A principle I would suggest generating 50 or 100 product owners must be done through apprenticeship (in addition, of course, to good training/consulting). If you accept this premise, you need to factor in a fair amount of time and grow the numbers gradually.”

Other Thoughts on Larger Scale ISVs

When I was a VP at Rational Software, where we employed 900 or so software development types, we had a specialized role titled something like “Technical Marketing Specialist”. This role was dedicated to working with the teams in defining and positioning the technical aspects of the solution to the marketplace. These individuals were typically collocated with the teams of their business units and they spent about half their time working with the teams in defining the solution and about half their time working with customers, training field engineers, etc. While this was a RUP, rather than agile shop, it occurs to me that the Technical Marketing Specialists were filling a role roughly equivalent to a Product Owner in agile. I mention this because I suspect that the Technical Marketing Specialist role, where it exists in the ISV today, could make a good role model for the Agile Product Owner in today’s larger ISV.

Ryan Martens, founder and CTO of Rally Software and formerly a Product Manager at BEA,had a similar thoughts when he commented on a prior post on this topic:

“At BEA, we had a certain class of product managers called Technical Product Managers. They were exactly Product Owners.”

Ryan went on to note that the title of “Program Manager” also performed a similar role in some larger scale contexts:

“After we acquired CrossGain, there was a tremendous amount of pressure to adopt the Microsoft titles of Program Managers – see http://blogs.computerworld.com/at_microsoft_program_managers_dont_program_or_manage who filled a comparable function. ”

In the post above, Microsoft describes a current operating model for Program Managers which is somewhat equivalent to the Product Owner role as defined in agile today.

Embedded Systems Example – Symbian Software Limited

When it comes to embedded systems and an even much larger enterprise scale, Symbian Software (now a part of Nokia) develops and licenses Symbian OS, the market-leading open operating system for mobile phones. Symbian initiated an agile transformation in 2008 that will ultimately affect many hundreds of practitioners.

Clearly, the development of a mobile phone operating system is a highly technical endeavor and one where the ultimate user (mobile device user) is fairly far removed from the major technologies (OS, device drivers, media players, etc.) which are the primary focus of the solution. As such, the development process does not lend itself quite so easily to the traditional, customer/user facing, agile Product Manager/Product Owner roles. However, the Product Owner role must still be successfully addressed in this highly technical context.

Mathew Balchin VP of Multimedia, PIM, CBS within Symbian Software Engineering was integrally involved in the design of the agile rollout model for Symbian. The scope of the challenge, (i.e. finding lots of product owners) was daunting. Matthew described their approach this way:

“We applied the product owner role pretty much out of the box (per Scrum training) and defined the interaction at the outset with our traditional product management functions. We have had some pretty good debates about selecting them. We have so far applied a skill -based selection process rather than an open recruitment approach. All our POs come from engineering teams and are senior engineers with product or customer experience. We have a one PO to two team mapping typically, rarely 3 teams, sometimes 1. It was flagged as one of the pivotal roles in success by pretty much all senior stakeholders and early on we identified the need for role-based soft skills training in addition to the standard agile training.”

IS/IT Examples –

When it comes to IS/IT, I have observed that the role/title of the “Business Systems Analyst” (someone responsible for analyzing the business needs of clients to help identify business problems and propose solutions) is often a reasonably good fit for the Product Owner role. In a couple of cases, I’ve seen that role directly assigned to fulfill the Product Owner responsibilities, (if not the title) and many such individuals have even be collocated to live with the team as part of the agile transformation.

In other cases, I have seen the role filled by “Project Managers” in the larger IT shop. In many cases, the self-managing and team-based planning lightens the workload for the project manager in the agile enterprise, and they often have the domain knowledge, inclination and insights necessary to fulfill the Product Owner role. Therefore, many have the time, skills and inclination to fill this role.

IS/IT Example—Discount Tire.

Discount Tire is America’s Largest Independent Tire Dealer. Chris Chapman, Application Services Manager and his teammates are driving a phased agile transformation that will ultimately affect the entire IS/IT team. Chris noted how they addressed the Product Owner challenge:

“In our case, our product owners are in IT.  They are the liaison to the business and in many cases speak for the business (it’s not always designed that way, but ends up being that way due to not always having business representation on the teams).   Our Business Systems Analysts in IT are filling the role of Product Owner.  Their previous responsibility of documenting detailed business requirements and rules now falls to the entire team in the form of user stories and acceptance tests (which is still a major “cheese moving” event for us).”

Summary

In summary, an agile implementation will be no more effective than the enterprise is at filling the critical Product Owner role with people who have the right mix of skills and interests to help one or more agile teams reach its productive potential. While, as we have seen, the sources for the pool of candidates vary from company to company and across industry segments, the challenge is being successfully addressed in even the largest software enterprises transitioning to these more productive, and more personally fulfilling, agile enterprise methods.

Looking Ahead

I suspect that this completes the Product Owner portion of this blog series. In the next few posts in this series, I’ll be moving on to describe some of the ways the Product Manager role also must evolve as the enterprise successfully transitions to agile methods.

Agile Portfolio Management Briefing

Tuesday night, I was the guest speaker at a joint meeting of the Denver chapter of the International Institute of Business Analysis (IIBA) and the newly formed Denver chapter of the Agile Project Leadership Network. We had a good crowd of 50-60 or so Business Analysis, Project Managers and the like. While most of the Business Analysts in the group were fairly new to agile, the attendees from APLN, of course, had more agile experience.

My topic was “Moving to Agile Portfolio Management”. The topic was of great interest to many in the group and we had a lively discussion about the issues of evolving the enterprise to take a more agile approach at the portfolio, as opposed to project or program, level. Of course much of this discussion revolves around the legacy (fixed budget-fixed scope-fixed timeline) thinking that is predominant in so many PMO offices, and how that thinking can be a major inhibitor to the achievement of real agility at the enterprise level. More importantly, we also discussed some ideas as to what to do about it.

I promised to post the slides from that presentation, so here they are. agile-porfolio-management-denver-apln-mar-2009

If you are a business analyst or project manager starting to see agile in your present or near future (and who isn’t?) you owe it to yourself to join one of these organizations.

Agile Product Owner and Agile Product Manager in the Enterprise 4: Responsibilities of the Product Owner in the Enterprise

Note: This is the fourth post in a series on the agile Product Owner and agile Product Manager‘s roles within the agile software enterprise. My primary collaborators on this series are Mauricio Zamora of CSG Systems, and Jennifer Fawcett of www.agileproductowner.com. In this post, I’ll describe some of the specific responsibilities of the enterprise Product Owner.

Earlier, I described why, in the enterprise, the responsibilities of the agile Product Owner, (as primarily defined by Scrum), is really a set of responsibilities that are shared between a significant number of agile Product Owners and a smaller number of agile Product Managers, and defined the following matrix of responsibilities.

Agile Product Manager

Agile Product Owner

Market/customer facing Product/technology facing
Collocated & reports into marketing/business. Collocated & reports into development/technology
Focuses on market segments, portfolio, ROI Focuses on product and implementation technology
Owns the Vision Owns the Implementation
Drives the Release Drives the Iteration

Responsibilities of the Agile Product Owner

Generally, the activities of the Product Owner can be divided into three primary responsibilities:

  • Driving the iteration
  • Co-Planning the release
  • Collaborating with Product Management

Driving the Iteration

As the iteration (see Figure) is the heartbeat of agility, we’ll take some time to understand the Product Owners role in this critical process.

An agile iteration

An agile iteration

Iteration Planning

Each iteration starts with a structured planning meeting which is one of the key “ceremonies” in agile. The meeting is a 2-4 hr, time-boxed meeting with all team members in attendance. The objective of the meeting is to agree on content for the upcoming iteration .

Preparation

Given the intensity and objective, the Product Owner should prepare in advance:

  • A short statement of the draft objective of the iteration.
  • Coordination of common objectives and dependencies with other Product Owners/Product Managers.
  • Review and reprioritize the backlog. This includes stories that (a) were already in the backlog; (b) failed acceptance in the current iteration; (c) are generated from defects or bugs.
  • Consider necessary refactors, defects, constraints, and dependencies.
  • Understand the team’s velocity for the upcoming iteration.

The Planning Meeting

The meeting begins with reviewing the objective and then moves to discussion of the prioritized work in the backlog.

  • The team discusses each item until it is well enough understood for the development team to detail and estimate engineering tasks.
  • This process is repeated for each story on the backlog until the team runs out of capacity.

The development team then presents its estimates to the Product Owner. Rarely, however, do the team’s estimates and velocity match the Product Owner’s initial objectives. (After all, what self- respecting Product Owner would have such limited vision that they might under-commit the team?) Therefore, the final, agreed-to scope of the iteration is the result of some negotiation between the Product Owner and the development team.

The Result- A Committed Plan

The end result of an iteration planning meeting is an iteration plan that contains:

  • A committed iteration objective.
  • A prioritized list of stories with estimated tasks and owners.

In any case, however, the Product Owner helps positions the development team for success in the iteration. For if they fail, they fail together.

Executing the Iteration

Thereafter, the primary responsibility for successfully executing (“landing”) the iteration lies with the development team. The team members deliver the stories to the code baseline in priority order:

  • Define – elaborate the story and its acceptance test.
  • Build – build the code and the test.
  • Test – Get the code to pass the test and ready for final acceptance

This cycle repeats within iteration until the end of the time box, with an objective of getting all stories completed and accepted.

While the primary responsibility for landing the iteration rests with the team, the Product Owner has a critical, daily role as well:

  • Work with developers and testers to elaborate each story.
  • Re-scope where necessary.
  • Attend the daily stand-ups.
  • Review stories that are ready for acceptance.
  • Accept those stories that pass the acceptance criteria.

Iteration Review – Accepting the Iteration

At the end of the iteration, a demo of the working, integrated software is held for all interested stakeholders. The format is as follows:

  • Presentation of each story by the responsible party.
  • Discussion and feedback with stakeholders.
  • Product Owners move the story to “accepted state” or leave the story in the backlog if incomplete.

At the end of the review process, the Product Owner reviews the objectives of the iteration and decides whether to accept or not the iteration based on how well the team (inclusive of the Product Owner!) did against the stated objectives. The final activity is the retrospective, where the team takes the time to reflect and assess on the results and then adapt the development process accordingly.

Maintaining the Backlog

In addition to working the current iteration, however, the Product Owner must always be preparing for the next iteration—and the one after that—at the same time. Doing so involves maintaining the backlog— pruning and reprioritizing, adding new items that are discovered, prioritizing defects relative to new development, and accepting interdependent stories from other teams.

Just In Time Elaboration

In practice, countless iteration retrospectives have surfaced the common feedback:

“we failed to deliver these stories that weren’t understood before we accepted it.”

Therefore, one of the most important Product Owner activities is just-in-time elaboration of those stories that are about to reach an iteration boundary.

To this end, the Product Owner always has a backlog stories that need additional elaboration via research, use-case development, prototyping or whatever to assure that they are sufficiently defined just-in-time-prior to the iteration boundary. These well-elaborated stories can be better estimated, accepted into the iteration, and most importantly, delivered.

Iteration Preview Meeting

A structured way to address this problem is via an in iteration
preview meeting, wherein the Product Owner discusses stories that are anticipated for upcoming iterations. This gives developers a a break from the “tyranny of the urgent iteration” and a chance to think about future work. Also, a better understanding of future stories gives the developers the ability to implement existing stories in a more synergistic way.

A Product Owner’s Iteration Calendar

Taken together, the activities and meeting commitments can fill up a Product Owner’s daily diary pretty well, as the figure below indicates.

Iteration meeting calendar

Iteration meeting calendar

Co-Planning the Release

Of course, the iterations serve a larger purpose — frequent, reliable and continuous release of value- added software to the customer or marketplace. Pictorially, this is seen in the Agile Enterprise Big Picture as the Release (or PSI – Potentially Shippable Increment) boundaries in the figure below indicate.

Co-planning the release

Co-planning the release

Release Planning is the seminal enterprise event that regularly aligns the teams to a common vision and release objective and the Product Owner plays an active role. I’ve written about the Release Planning event extensively in my blog and I won’t belabor it here. . Prior to the event, the Product Owner will typically:

  • Update the team’s local backlog.
  • Meet with other Product Owners to understand overall system status.
  • Meet with Product Managers to understand the vision for the upcoming release.
  • Brief the team on the upcoming release objectives.

During the event, the Product Owner will typically:

  • Help identify, prioritize and estimate stories that will be necessary to achieve the release objectives.
  • Help design the release plan by laying the stories into iterations.
  • Participate in the team’s discussion, impediment and risk identification.
  • Identify and coordinate dependencies to ensure a cohesive solution
  • Participate in refining the release objectives and making a commitment to the release.

Collaboration with Product Managers

While the Release Planning event is one highly structured collaboration opportunity, the reality is that enterprise agility is most effective when Product Owners have a far more closely coupled relationship with product management. Indeed, I often recommend that Product Owners “report on a fat dotted line to product management” as the figure shows.

capture-fat-dotted-line

From a line management perspective, Product Owners typically report into development. They:

  • Are collocated with the team.
  • Are rewarded, etc. based on how the team as a whole performs.

But they are also honorary members of the product management organization, where they:

  • Receive overall product direction.
  • Attend most relevant PM meetings, functions, and planning sessions.
  • Receive input with respect to career growth and performance.

Looking Ahead

In prior posts, I’ve described the challenge of enterprise scale, noting that it is no trivial task to identify and train 10, 20 or even 100+ individuals to effectively implement the product owner role in the largest software enterprises. In the next post, I’ll provide some case study “vignettes”, which illustrate how some specific agile enterprises found the right people necessary to fill this role, along with some of the unique challenges they faced and the solutions they applied.

Agile Journal Part II – Responsibilities of the Agile Product Owner in the Enterprise

For those following this blog series, some of this content has been repurposed for a three part series in the Agile Journal. Part I- On Agile Product Managers and Product Owners: A Scalable, Nuanced Approach, was published last month. Part II – Responsibilities of the Agile Product Owner in the Agile Enterprise has just been published. The content mostly follows what is being published here, but it is stitched together for a little easier reading and better digestibility. Together, Part I and Part II simply “tell a better story” than these bite sized blog posts. And it’s more portable, too, as you can print and download it in PDF form.

For those who hadn’t been following the Agile Journal, I’d strongly recommend it as it has diverse and refreshing content in every edition.

My Agile Journal series and this blog series on this topic will continue in parallel, as they serve slightly different purposes. In prior posts, I’ve described the challenge of enterprise scale, noting that it is no trivial task to identify and train 10, 20 or even 100+ individuals to effectively implement the product owner role in the largest software enterprises. In the next post, I’ll provide some case study “vignettes”, which illustrate how some specific agile enterprises found the right people necessary to fill this role, along with some of the unique challenges they faced and the solutions they applied.


New Pricing for Lean & Kanban Conference 2009

The Lean & Kanban Conference 2009 (http://www.leankanbanconference.com/) has just announced a more favorable and modular pricing structure for this unique, first year conference, to be held in Miami, May 6-8.

Learn the latest thinking in applying Lean techniques like Kanban to shorten cycle time and deliver more value from your software projects. Mix with the experts and practitioners using these ideas every day in organizations around the world. Be part of the next wave of software management and leadership.

This will be a great opportunity to see some of the latest thinking on the application of lean software thinking in both the smaller and larger software enterprises. I’ll be giving a keynote talk on the Lean and Scalable Enterprise Requirements information model at this event and you’ll also hear from a number of thought leaders, including David Anderson, Alan Shalloway and Amit Rathore. See the conference web site for additional details.