The Agile “Shine On” (Visibility and Truth in Agile)

Recently the post “Zero to Fear in 60 Seconds” appeared in Agile Juice . This great post highlights some of the challenges with the transparency and fact-based status assessment that agile purports to deliver. Simply put, it’s not unusual (at least initially) that the apparent visibility provided by the daily standups and story status tracking may paint a far rosier picture than the facts warrant. The post points out that these problems may not be discovered until the hardening iteration, by which time it is too late for effective corrective action. The article provides some rationale for this behavior, along with some recommendations.

Perhaps some of the reasons for this initial “agile shine on” (overly optimistic reporting) include:

1) our organizations and software development practices haven’t necessarily evolved in a fact-based way prior to agile adoption (typically reporting on plan until it’s proven that we are not)
2) companies have sometimes used the facts as bludgeons, rendering a fact-filled environment risky for the fact provider
3) people are people and asking them to step up to the bar of personal accountability is not a guaranteed win for all practitioners (the bar may be too high for some)
4) people would rather not deliver bad news

This just reminds us that visibility isn’t free and initially, it may not even be transparent!

So as leaders,  we must constantly foster and encourage the delivery of truthful and factual information at all times. We must never punish the messenger; rather, we adjust the process. Remember, the Facts are Always Friendly in agile.


The New New Product Development Game and the Roots of Scrum

I was recently doing some research on prospective agile governance models for a company in the throes of understanding how to manage the entropy of a large number of newly energized and empowered agile teams. For perspective on the problem, I returned to the following article for guidance: The New New Product Development Game. This article was published in the Harvard Business Review in 1986 and provides a perspective on the “Toyota Way” of concurrent engineering in the automotive business.

I read this article back in the late 1980s and considered it incredibly relevant to the product development work I was doing then. More recently, I was reminded of this article from a talk Jeff Sutherland gave in 2006 or so entitled the “Roots of Scrum” and it provided some excellent background for Scaling Software Agility.

I am always amazed by the seminal, agile principles that are surfaced so well in this article. I consider this to be essential reading for anyone trying to better understand the core philosophies and principles behind Scrum (and agile). Here’s a teaser:

“This new emphasis on speed and flexibility calls for a different approach for managing new product development. The traditional sequential or “relay race” approach to product development – (editors note: i.e. “waterfall” ) … – may conflict with the goals of maximum speed and flexibility. Instead, a holistic or “rugby” approach – where a team tries to go the distance as a unit, passing the ball back and forth – may better serve today’s competitive requirements.”

And since they were building automobiles with this approach, it “scales” too! I bet you’ll agree if you take the time to read it.

On Agile Maturity and Hardening Iterations

I’ve noted a teams need for an occasionally “hardening iteration” in SSA Chapter 13 as well as in a number of posts (Release Planning at Enterprise Scale: an Inside-out Narrative.) The hardening iteration is dedicated to eliminating technical debt, such as minor refactors, bugs, testing across multiple platforms, final QA, docs etc., that is most typically scheduled at the end of a GA release cycle.

Some might argue that needing a hardening iteration is a sign of agile immaturity and it’s just a testing waterfall phase by an agile name. In practice, however, I’ve seen really good agile teams continue this practice for years, so I personally think it’s a practical solution to a real world problem and I typically encourage teams to adopt a hardening iteration pattern in their initial iteration and cadence models.

In a recent post, Paul Beavers notes an email thread from an agilist friend of his, noting that the use of hardening iterations can be both a sign of immaturity (lots of such iterations at the end) or a sign of maturity (occasional hardening iterations in mid-release cadence drawing towards elimination of the need for hardening over time). In any case, it’s a pretty reasoned treatment of the question and it can be found at: Indications of Agile Maturity.

Meeting Deadlines – “Tiger Teams” vs. Agile Project Management Practices

Often times when introducing some of the basic agile project management practices to new teams (for example, the daily standup) some of the more experienced (ok, older) team members comment on the fact that these techniques look a lot like the techniques they used in the past when their projects were in deadline trouble. Indeed, most of us have similar experiences with challenged projects and when a project is struggling, and the deadline approaches, we would always fall back on some tried and true disaster recovery techniques. We’d form “tiger teams” that would typically work as follows:

  • Quickly form a team of key, dedicated individuals, representing all the necessary disciplines
  • Empower the team to cut scope as necessary in order to achieve the deadline
  • Meet daily, first thing every morning (meet twice daily as the deadline approaches)
  • Bring honesty and complete visibility to actual, objective (not planned) project status
  • Commandeer outside resources as necessary
  • Continue doing all the above until deadline achieved

We immediately see the parallels to agile project management, reflecting again that agile is not so much new, as a synthesis of some of most effective and highly tactical project management techniques from prior experience.

Parallels are as follows:

Tiger Team Action Agile Team Principle
Form a team of key, dedicated individuals, representing all the necessary disciplines Get the right people on the team. All necessary disciplines (define-build-test) must be present. Dedicate the team members for the life of the project (iteration or release).
Meet daily, first thing every morning (meet twice daily as the deadline approaches) Daily standup. (Plus we have observed many teams doing twice daily stand-ups as iteration or release deadlines approach.)
Empower the team to cut prioritize and cut scope as necessary in order to achieve the deadline Product Owner who is a team member, empowered to manage priorities and cut scope is systemic in agile
Bring complete visibility to actual, objective (not planned) project status Empirical, demonstrable frequent outputs in hands of product owner and customer
Manage/commandeer outside resources as necessary Commit resources, eliminate outside impediments that inhibit success
Continue doing all the above until deadline achieved Repeat for every iteration throughout the project lifecycle.

In other words, the best way to keep projects out of trouble is to apply the same intensity at the beginning of the project as we used to do at the end. While this may seem intense for the routine day to day existence of the project, the actual result is sustainable and highly productive development process that avoids the enormous deadline pressures we used to see at the end, as the illustration below shows.


Ah Ah, The Daily Build Revisited ….. Revisited

“We place the highest value on actual implementation and taking action.

There are many things one doesn’t understand; therefore, we ask them, why don’t you just go ahead and take action?

You realize how little you know, and you face your own failures and redo it again, and at the second trial you realize another mistake . . . So you can redo it once again.

So by constant improvement one can rise to the higher level of practice and knowledge.

This guidance reminds us that there is no problem too large to be solved if we are only willing to take the first step.”

— Fuijo Sho, President, Toyota


See the post “Ah, the Daily Build revisited”, dated March 23, which describes one agile teams initial reticence to invest in a true, automated, daily build process.
Six weeks later, I am back in the Ukraine and I am happy to report that the team has successfully converted to daily (nightly) builds. It was not a trivial project, nor was it as demanding as the team thought going in. The daily build process they use is as follows:

1) Check source code out of SCM system along with current versions of unit tests and functional tests

2) Run Make to build the application. Check logs. Abort if error.

3) Execute units tests on build server. Check logs. Abort if error.

4) Run configuration scripts to initialize/prepare Deployment Server

5) Deploy built application and unit tests to Deployment Server via SSH

6) Initiate Functional Test Framework (custom in this example) to execute functional test suites on application, builds error logs and reports.

During this process, however, they also discovered that the current BVT (Build Verification Tests) took a full 7 hours to run. Obviously this would kill the Daily build process, so the team was forced to refactor the functional tests to a lighter weight version. However, in doing so, they did not compromise the BVT quality as they simply refactored the test data sets to match only those data configurations that were actually used by the functional test framework and test suites. (in other words, as part of the BVT, they were spending time downloading and initializing data that was superfluous to the actual test cases.)

After about six weeks, the team had mastered the process and it is now a critical ability that helps the team build in quality and avoid the big bang crunch at the end of the iteration.

Kudos to yet another agile team that has now mastered this previously un-masterable process!

Project results:

Nightly build: Fully automated, daily build successful!
Project time: 6 elapsed weeks
Total person-hours invested: <80 hours

Lessons Learned:

1) There is some invention in developing scripts to automate all this stuff

2) If you discover that the BVT is just too big to run in less than an hour, stop and refactor it immediately until you have a sufficient test that can be run in less than an hour

3) Don’t be intimidated. Initially the team believed that the data dependencies and continuous refactoring of the baseline would prevent nightly build success.


§ Team has immediate feedback on any changes that affect the baseline and automated tests.

§ Small debugging of daily builds increases confidence. Confidence in the iteration grows daily, along with the code.

§ Daily builds foster communication. ( new developer: “I checked this in but broke the build, what did I do wrong?”)

Moral: Just Try It!

The Standard Two-Week (well maybe one is just as good or better…) Iteration

Philippe Kruchten, while reviewing the book in preparing for writing the foreword ( and thereby causing me no end of grief as well as the obvious need to spend a weekend or two rewriting some sections that he fairly, and roundly criticized) noted that in Chapter 11, Mastering the Iteration, that I was “ again being dogmatic about iteration length” with my steady drumbeat of a two week iteration.

I defended the recommendation as I have substantial and quite successful experience with this pattern in over a half dozen projects, large (think BMC and others) and small (no names provided because no one thinks their project is small) , and therefore left it as such in the book.

And now you might ask, (link) how do I then find myself directly managing an agile project with one week iteration lengths? The answer, I never even saw it coming. When we first started this project in summer of 2006, the team was brand new and new to each other, all raw recruits, the project was not well defined, the technology choices were all over the map. So naturally we started with shorter iterations, giving us a more frequent cadence and giving us the chance to review the exploratory results as well as individual talent on a frequent weekly basis. As the team learned the domain, we moved from exploration to hard core development and never changed the pattern.

Simply, the team learned how to divide their work into one week size chunks even as the system grew. When faced with a major refactoring or particularly large and seemingly atomic chunk of work, we would simply lay out the objectives in two or maybe three separate iterations, (Not necessarily forcing fully integrated functional outcomes at each boundary, but absolutely forcing fully testable outcomes, protocols, interfaces and the like) but all of one week length. The weekly pattern is so simple and obvious (“if it’s Tuesday I better have all my stories in acceptance testing…”) and so easy to manage, I doubt we’ll ever leave the pattern. We could, and we would likely be successful, but do we need to? Do we think we would benefit? – nope.

Will I update the two week to one week recommendation in the book? I have no idea, but I suspect you will never hear or see the words “three week iteration.”

Ah, the Daily Build Revisited……

I just left the development team in Kyiv that I have been working with for the last six months. This is a new team, formed in only the last six months, building a large scale application (unfortunately, the project is in deep stealth mode, so the nature of the application will be “non-disclosed” for a bit longer). The agile master, (Alexander ,or Sasha ,to the team), was recruited specifically for his experience in agile development. None of the other members of the team had any prior agile experience but with Sasha in the lead and with some steady coaching from me, their agile prowess is pretty good (I’d give them four “bars” on the agile scale). Even though the application is fairly large in scope, the team runs weekly iterations. (Why, you might ask, given the steady recommendation in my book for a standard two-week length? The answer to that question will come in another posting).

Recently, we’ve been working hard on some of the “long poles in the agile tent”, most specifically completing functional testing within the iteration, splitting the team as it got to 14 members, etc, and I was fairly comfortable with their overall progress and we had settled into a fairly routine weekly cadence with Wednesday being demo day.

For entirely different purposes we had a consultant visit to look at some of the technical aspects of the application we were building. The consultant had 30 years of experience in building large scale systems, mostly by traditional process models. He understood a bit about agile and on the way in from the airport (Kyiv Borispol, Ukraine) he asked some basic questions about the team and their process. He immediately asked about the daily build process. Sasha noted that, while the team didn’t build daily, they could build on demand and would build once or twice per iteration and that was adequate. I didn’t say anything at all on the point. After a spirited defense of the ad hoc build by Sasha, the consultant seemed satisfied.  Sasha looked over at me and noted that I had pulled the hood on my parka down over my face. I suspect he understood the point.The next day Sasha I discussed some of the problems we had been having in testing certain modifications to essential algorithms that produced results that required review by the product owner (me, in the interim). Basically, we noted that we often had to put together a custom build at every point where we needed to test the new algorithms, and often that was a scramble and a delay of a few days to see results. (note to agilists of seniority: remember when seeing full results in 2-3 days was lightning fast?!) We also noted that we had spent significant time in reaching the demo state with Wednesday integration and demo preparation being a monumental weekly task, and that more than one iteration had failed to be successfully integrated in time for the weekly demo. After, a few more minutes of discussion, Sasha gave me the knowing look that said –it’s time–. So Sasha built a story in the next iteration to instigate the daily build 

When I think of the daily build, I’m reminded of the high bar that is set by the true agilists (See Chapter 14, Continuous Integration.) For example, Martin Fowler notes that he and other agilists have a pretty aggressive view about what makes a successful build:

l       All the latest source files are checked out of the configuration management system (including all scripts, config files, etc.).

l       Every file is compiled from scratch.

l       The resulting object files are linked and deployed for execution.

l       The development system is started, and a suite of tests (build verification tests) is run against the system.

l       If all these steps execute without error human intervention and every test passes, then we have a successful build. 

Reference: Fowler, Martin. 2006. 

I promise to report back in a few weeks and let you know how it goes.

The Simple, Visual, Iteration Kanban

I remember so vividly one of my first impressions of an effective agile team. It was about 2001, and I walked into a development shop with a team of about 12 people seated at one large table (later abandoned for modest, semi-open cubicles, by the way.) On one side of the room was a large whiteboard with three vertical columns of sticky notes. I don’t remember for sure what the columns were labeled, or even if they were labeled, but they each contained a set of stories the team was working on, with the “state” of the story represented by the column. However, somehow it was clear to me immediately that the states represented were not started, in progress and done. (Over time, I’ve come to think in four basic “states” of a story, “defined”, “in progress”, “complete” and “accepted”.)



An Iteration in Flight
At that time, one of the companies I was working with, Pelion Systems Inc., was developing software to help manufactures “lean” their enterprise. Pioneered by the Japanese, lean flow manufacturing makes extensive use of the kanban card, which loosely translated, means “signal”. In lean manufacturing, each kanban card is a “simple visual signal” that indicates that a manufacturing work cell is ready for more work. On the floor, it pops up as a visual reminder to everyone in eyesight that a particular cell will soon need more raw material to keep it in production.


In the software development shop that day, I was struck by a number of insights, including some of the parallels to agile development and lean flow manufacturing (developed further in SSA Chapter 6 – Lean Software) and the power of the simple visual signal. In this particular development shop, any manager, developer, team member, and even the janitor could see immediately what the state of their project was at the point in time – what work was in process (on the floor), what work was completed (in finished goods) and what work had not been started (raw material). And, as that knowledge wasn’t enough for starters, if any stakeholder in the development effort knew the length of the iteration (easy in this case, two weeks standard) and which day of which week (week one or two) it was, it was possible to predict with some reasonable accuracy whether the project (two week iteration) was likely to be landed (completed on time). A manager’s dream; instant at-a-glance status.

I was hooked. Any software process that provided such a simple way for everyone on a project to know where they were, at a moments glance, was a process that needed much more serious investigation. And so my foray into agile and lean software development practices was further accelerated.

–Dean Leffingwell