Ah Ah, The Daily Build Revisited ….. Revisited

“We place the highest value on actual implementation and taking action.

There are many things one doesn’t understand; therefore, we ask them, why don’t you just go ahead and take action?

You realize how little you know, and you face your own failures and redo it again, and at the second trial you realize another mistake . . . So you can redo it once again.

So by constant improvement one can rise to the higher level of practice and knowledge.

This guidance reminds us that there is no problem too large to be solved if we are only willing to take the first step.”

— Fuijo Sho, President, Toyota


See the post “Ah, the Daily Build revisited”, dated March 23, which describes one agile teams initial reticence to invest in a true, automated, daily build process.
Six weeks later, I am back in the Ukraine and I am happy to report that the team has successfully converted to daily (nightly) builds. It was not a trivial project, nor was it as demanding as the team thought going in. The daily build process they use is as follows:

1) Check source code out of SCM system along with current versions of unit tests and functional tests

2) Run Make to build the application. Check logs. Abort if error.

3) Execute units tests on build server. Check logs. Abort if error.

4) Run configuration scripts to initialize/prepare Deployment Server

5) Deploy built application and unit tests to Deployment Server via SSH

6) Initiate Functional Test Framework (custom in this example) to execute functional test suites on application, builds error logs and reports.

During this process, however, they also discovered that the current BVT (Build Verification Tests) took a full 7 hours to run. Obviously this would kill the Daily build process, so the team was forced to refactor the functional tests to a lighter weight version. However, in doing so, they did not compromise the BVT quality as they simply refactored the test data sets to match only those data configurations that were actually used by the functional test framework and test suites. (in other words, as part of the BVT, they were spending time downloading and initializing data that was superfluous to the actual test cases.)

After about six weeks, the team had mastered the process and it is now a critical ability that helps the team build in quality and avoid the big bang crunch at the end of the iteration.

Kudos to yet another agile team that has now mastered this previously un-masterable process!

Project results:

Nightly build: Fully automated, daily build successful!
Project time: 6 elapsed weeks
Total person-hours invested: <80 hours

Lessons Learned:

1) There is some invention in developing scripts to automate all this stuff

2) If you discover that the BVT is just too big to run in less than an hour, stop and refactor it immediately until you have a sufficient test that can be run in less than an hour

3) Don’t be intimidated. Initially the team believed that the data dependencies and continuous refactoring of the baseline would prevent nightly build success.


§ Team has immediate feedback on any changes that affect the baseline and automated tests.

§ Small debugging of daily builds increases confidence. Confidence in the iteration grows daily, along with the code.

§ Daily builds foster communication. ( new developer: “I checked this in but broke the build, what did I do wrong?”)

Moral: Just Try It!


2 thoughts on “Ah Ah, The Daily Build Revisited ….. Revisited

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s