Updated Agile Requirements Metamodel (Enterprise Backlog Model)

I was walking through some blog posts in the requirements category today looking for a later version of the full “agile requirements metamodel”.  (Some readers commented that “enterprise backlog model” might be a better descriptor, and I tend to agree). The model has evolved and been refined based on peer reviewer feedback (thanks Gabor and others….) of Agile Software Requirements. Evidently I didn’t track that evolution here on the blog, so here is the updated graphic of the final, (at least for this book version) full model as it will appear in the appendix of the book.

Full agile requirements metamodel (Enterprise backlog model)

Note: This metamodel is also integral to the work I’m currently doing with Craig Langenfeld on applying agile development to high assurance systems, so I’m tagging this post to that category as well.

Software Verification and Validation in High Assurance Agile Development: Definitions

Series Background: In this series of posts, I’ve been using Medical Device Development (as Regulated by U.S. FDA via CFR 820.30 and international standard IEC62304) as an exemplar for suggesting ways to develop high quality software in regulated (and other high assurance, high economic cost of failure) environments in an agile manner. This series is sponsored, in part, by Rally Software Development.

====================================================

In the last post, I introduced a suggested process model that teams could use to visualize and reason about how to apply iterative and incremental, and agile methods in the development of such systems. The graphic is reproduced here:

High Assurance Agile Software Development Lifecycle Model

I described the iterative and incremental nature of the model, and noted that we’ll need a better understating of Verification, Validation and Traceability to further understand it.

The use of the terms verification and validation,  (also described as V&V, SV&V) are often used interchangeably in the industry. The most common interpretation of these terms is that they translate to “assured testing” practices. In point of fact, the words have different meaning and lumping them together as V&V can obscure the constructs entirely.

One of the reasons we’ve picked an exemplar for our discussion is to provide a more definitive basis for the method, grounded in at least one known, public example: in our case the development of medical devices containing software. So we’ll return to these roots for definitions of these terms, and build from there. In an earlier post, I described the chain of regulations covering medical devices marketed in the US, and summarized with the graphic below:

Chain of regulations governing medical device software in the US

While CFR21 Part 820.30 is the governing regulatory requirements, it’s surprisingly short and not very explanatory. To address this, FDA produced the document on the bottom right, (General Principles of Software Validation; Final Guidance for Industry and Staff) to provide guidance to those who operate under the 820.30 mandate.

This document provides some meaningful definitions, and we’ll start with the most critical ones that are relevant to our model:

Verification

“Software verification provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase.

Software verification looks for consistency, completeness, and correctness of the software and its supporting documentation, as it is being developed, and provides support for a subsequent conclusion that software is validated. Software testing is one of many verification activities intended to confirm that software development output meets its input requirements. Other verification activities include various static and dynamic analyses, code and document inspections, walkthroughs, and other techniques.”

Clearly this definition takes us beyond testing, and takes a more “white box” look at assuring that each development activity meets the requirements imposed from the prior activity. With our yet-to-be-described agile , high assurance practices, continuous (and where possible, automated) verification activities will play a central role. Fortunately, we’ll discover that many of these practices (hierarchical reqs, small units of functionality, unit testing, pair/peer review, acceptance testing, etc.) are part of standard agile, high quality hygiene, so we’ll be able to “rigorously apply” existing practices, rather than invent new ones.

Validation

“…FDA considers… software validation to be confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”

In practice, software validation activities may occur both during, as well as at the end of the software development life cycle to ensure that all requirements have been fulfilled. Since software is usually part of a larger hardware system, the validation of software typically includes evidence that all software requirements have been implemented correctly and completely and are traceable to system requirements. A conclusion that software is validated is highly dependent upon comprehensive software testing, inspections, analyses, and other verification tasks performed at each stage of the software development life cycle. Testing of device software functionality in a simulated use environment, and user site testing are typically included as components of an overall design validation program for a software automated device.”

In addition to these definitions, this document also provides an introduction and definition to a few other requirements that will be imposed on the process:

Software Requirements Specification

“A documented software requirements specification provides a baseline for both validation and verification. The software validation process cannot be completed without an established software requirements specification (Ref: 21 CFR 820.3(z) and (aa) and 820.30(f) and (g)).”

So while in agile, we don’t often create these in a formal way, instead using the backlog, collection of user stories and acceptance criteria,  test cases and the code itself to document requirements. But in this context,  it is 100% clear that we will need to rigorously develop and maintain a software requirements specification as part of our high assurance, but still largely agile,  practices.

Traceability

This document goes on to describe traceability and traceability analysis as one of the primary mechanisms to assure that verification and validation are complete and consistent. However, it doesn’t define traceability, for this we refer to FDA Glossary of Computer Systems Software Development Terminology, where we find the IEEE definitions:

traceability. (IEEE) (1) The degree to which a relationship can be established between two or more products of the development process, especially products having a predecessor-successor or master-subordinate relationship to one another; e.g., the degree to which the requirements and design of a given software component match. See: consistency. (2) The degree to which each element in a software development product establishes its reason for existing; e.g., the degree to which each element in a bubble chart references the requirement that it satisfies.

Next Steps

With these activity/artifact/process definitions and the lifecycle graphic behind us, we can go on to a more meaningful elaboration of the model. We’ll do that in the next few posts.

An Iterative and Incremental Process Model for Agile Development in Regulated Environments

In this series of posts, I’ve been using Medical Device Development (as Regulated by US FDA via CFR 820.30, and internationally as regulated by IEC 62304) as an exemplar for suggesting ways to develop high quality software in regulated (and other high assurance, high safety, high economic cost of failure) environments in an agile manner.

In an earlier post, based on my reading of the 820.30 regs, I described the chain of regs and explanatory documents that might tend to imply a “waterfall” (sequential, one-pass, stage-gated) development model, which might look something as follows.

An implied, high assurance, waterfall lifecycle model

Whether based on CFR 820.30 or not, this would seem like a reasonable conclusion based on our historical approach to such development. With the addition above of a single pass “verification” and “validation” activities, this closely follows the requirements>design>code>test models of the past. On the surface, at least, this model has some obvious advantages:

–       it is simple and logical (what could be more logical than code following requirements, and tests following that?)

–       you only have to do “it all” (especially verification and validation, which can be both labor intensive and error prone) once.

Of course, readers of this blog know it doesn’t work well that way at all, and that’s why we strive for agility in the high assurance markets, just like we do everywhere else. But for many, the apparently beguiling (but false) simplicity of the model is one reason that it has made its way into the various governing corporate quality standards, etc., if for no other reason than “that’s the way we’ve always done it”. (And admittedly, most of the rest of the software industry is doing it that way too).

It Doesn’t Have to be that Way

Momentum aside, however, we now note that while such a thing could be inferred by the various governing documents and our own quality affairs personnel (and it certainly has been applied by countless FDA reviews and other quality audits), in point of fact the specific regs we are using in the exemplar DO NOT require such a process:

1)   From US General Principles of Software Validation; Final Guidance for Industry and Staff (US FDA CDRH 2002) we note the following:
“This guidance does not recommend any particular lifecycle models…. “

2)   Even more specifically, as Pate and Russell note from IEC62304 (a widely recognized international standard for medical device software which is largely harmonized with FDAs interpretation of CFR 820.30)
“these activities and tasks may overlap or interact and may be performed iteratively or recursively. It is not the intent to imply that a waterfall model should be used.”

3)   And finally, from the 2009 Abbott Labs case study describing the successful application of agile development in a Class III (the most stringent FDA regulatory category) Adopting Agile in an FDA Environment:
“we will describe the adoption of agile practices…..This experience has convinced as that an agile approach (multiple iterations during the course of development) is the approach best suited to development of FDA-regulated medical devices.”

There you have it. If we are using waterfall development in support of high assurance systems (at least in the case of the regs we have described here), it is evidently because “we have always done so” (indeed, as this is still mostly case in much of the rest of the software industry) rather than “the regulators made us do it.”

So if, as an industry, we do want to increase the productivity, quality, and yes, even safety and efficacy of the software we produce, it’s time to move on!

Moving to an iterative and incremental, and … agile model.

With a nod to Abbott Labs whitepaper and the associated presentation, and with due respect to the verification and validation activities that will still be required in high-assurance development (see High Assurance Agile Development in Medical Devices: Prove It post), we offer the following figure as a potential, general model for iterative and incremental (and increasingly, as we will see) agile development in high assurance, regulated markets:

Iterative and Incremental High Assurance Lifecycle Model

It can be seen from the figure that development of software in this model does not follow a waterfall, sequential reqs-code-test time line. Instead we use a series of short iterations, each of which “defines|builds|verifies” some new, and valuable user functionality. [note: I described this as the Define|Build|Test atomic building block of agile development in Scaling Software Agility.)

Periodically (typically after 3-4 iterations) this new increment of software can then be validated and reviewed, prior to being made available for alpha or beta testing, or general availability release. (The shorter the iterations and increments-the faster the feedback;the faster the feedback-the higher the quality. The shorter and faster we go-the more agile we become.)

With respect to additional elaboration of the model, it occurs to me that I’ve used the terms “verify” and “validate” here, as if we knew what they actually meant. Indeed, there is much confusion on the topic, so I’ll try to clarify that in the next post as I go one to explain more about the model above. When we do so, we’ll also hit that nifty high assurance bugaboo called “traceability”. (And doesn’t that one already just sound like some fun?)

High Assurance Agile Software Development in Medical Devices: Prove It

In this series of posts, I’ve been laying the background for describing a set of practices that I believe can help development teams working in high assurance and regulated industries build the highest quality software possible…. using agile methods. It’s not an idle thought. In the last 3-4 years, I’ve had the privilege of working with some extraordinary agile teams, in addition to their obvious market success, one things stand out: they have the highest quality solutions and products that I’ve experienced. It isn’t accidental. Properly applied, agile methods build software quality in, rather than leaving gobs of untested code for a nasty waterfall test/triage/death march at the end. And we know the elimination of a large number of reported defects does not induce quality. It’s too late for that. In my new book Agile Software Requirements, I cover a lot of the basic agile requirements practices and how they produce endemically higher quality. This book will be released in the next 30 days or so, and I hope it provides value to all agile development practitioners.

But in this environment, we must go one step further, we not only have to have quality, we have to prove it, as there are QA governance and regulatory affairs people watching our every step.

The Rules of “Prove It”.

In an earlier post, I introduced a useful briefing presented by SoftwareCPR at the recent GE Agile Conference, Based on their experiences and interpretations with FDA 820.30 and IEC62304, Brian pate and Mike Russell posit that there are a set of immutable rules, the “gotta haves”, if you will, that are necessary for development and deployment of medical device software. I agree with the assessment, and with their permission, I now don’t have to make those up myself! Figure 1 below, provides their view of the “gotta haves’ for medical device development (and by inference at least, a comparable set of gotta haves occurs in most other high reliability, regulated environments):

It’s probably obvious that most of these items would not be required in typical agile environments, even environments where the cost of errors is prohibitively high.  But if we are going to use agile in this environment, we are going to have to make our peace with these requirements, and then use agile, iterative and incremental approaches to building the code we will eventually deploy. That means that many of the above artifacts, (for example: Software Requirements Specifications, traceability, and verification and validation activities) will need to be iterative and incremental, too. We can’t just do them once, whether at the beginning, or we will fall back in the waterfall trap.

I guess we expected that.

In the next post, I’ll describe one view of such an agile model, and then proceed to address these key requirements, without killing the goose that laid this golden agile egg.

 

Scrum of Scrum Ground Rules

In the last post, I described an impressive Scrum of Scrums meeting I attended this week. It’s a pretty big Agile Release Train that has 14 cooperating teams. After just a few months of agile, this program really seems to be “on their game”, and this team clearly is developing some significant “ba”.

I also snagged a photo of their “ground rules for the Scrum of Scrum meeting”. But the photo isn’t clear, so I’ll just list the rules here (with their permission):

• “Like my daily Scrum, this meeting is meant to be fast and short.
• I will be present on time and I will come prepared.
• I will first state my Scrum Team name.
• I will talk about my team, not individuals on the team.
• I understand that problems can and should be raised during this meeting, but I will not discuss solutions until after everyone has had a chance to go thorugh their standard report.
• After all teams have reported, the focus will shift to any issues, problems or challenges that were raised. These will then be discussed and resolved or added to the Scrum of Scrums parking lot for future discussion.”

Thanks guys! (you know who you are).

On “Ba” at a Scrum of Scrums

I was a “chicken” today at one of the more effective Scrum of Scrums for an Agile Release Train that I have witnessed. Big program. 14 teams. The uberScrumMaster (Release Team Manager) started the meeting at 9 am, facilitated the meeting and ran a pretty tight ship.

Fourteen teams (2-3 remote) reported in a standard format (see pic) in about 20 minutes total.

A few “meet afters” ( we need to talk about this more) were raised.

In the standard report, one of the ScrumMasters narrated the following;

–       “yesterday, we reported that we were blocked (something was put in our way) by another team.

–       It wasn’t true. It was we that had blocked ourselves and, yes, we blocked everybody else too!

–       We are really sorry and we fixed it just as soon as we could .”

This conversation happened part in jest and fun, and part in dead seriousness. This team of teams has “ba”. (the energy of an effective, self-organizing team. See below.).

After the standard report, there were 3-4 “meet afters” discussed. The uberScrumMaster took notes and his laptop was projected in the room so everybody can see the work and agreements in process.

The meeting adjourned at 9:25 AM , with all questions asked and answered.

Make no mistake, this team of teams has significant challenges ahead (don’t they all?), but it is just so satisfying when you see it all come together. And they will handle the challenges. I am so proud of these people. It is so rewarding for any of us to be a part of a high performing team.

Again I am reminded, that with just a little leadership (which these folks and their managers, who were not present at this meeting, of course) clearly exhibit, you can trust the teams EVERY TIME.

Quick note on “Ba” below.

========

What is Ba?

Ba- is the Zen of Scrum, a shared “context in motion and the energy that drives a self-organizing team:

–       “Dynamic interaction of individuals and organization creates synthesis in the form of a self-organizing team

–       The fuel of ba is its self-organizing nature-  a shared context in which individuals can interact

–       Team members create new points of view and resolve contradictions through dialogue

–       New knowledge as a stream of meaning emerges

–       This emergent knowledge codifies into working software

–       Ba must be energized with its own intentions, vision, interest, or mission to be directed effectively

–       Leaders provide autonomy, creative chaos, redundancy, requisite variety, love, care, trust, and commitment

–       Creative chaos can be created by implementing demanding performance goals. The team is challenged to question every norm of development

–       Time pressures will drive extreme use of simultaneous engineering

–       Equal access to information at all levels is critical

(source. Hitotsubashi: On Knowledge Management)

Upcoming AAMI Technical Information Report on Agile and Medical Device Software

Mike Russell and Brian Lewis Pate of SoftwareCPR.com just informed me of an important new development with respect to the use of agile in medical device development (we are using medical device development as the exemplar for high assurance development in this blog series).

Specifically the AAMI (Association for the Advancement of Medical Instrumentation) is developing a Technical Information Report (TIR) which is devoted precisely to this topic. The description of the TIR is below. Mike and Brian tell me that it will likely be published sometime next year. (Hopefully, I might become involved as a reviewer). In any case, I’ll proceed apace with this blog series over the next few months, because Rally Software (my sponsor for some of this work) and I have some groundwork that we need to lay well prior to next year. Thanks for the tip Brian and Mike!

The foreword and abstract is include below:

Foreword

Agile software development (hereafter referred to simply as “Agile”) has been evolving for many years. Agile began as a niche concept being used in small pockets of the software industry, and has since grown to be well established in many different software development contexts. As it has grown, it has been adapted to fit the unique needs of a specific context. For Agile to be established in the medical device software industry, guidance is needed to adapt it to fit that unique context. This TIR fulfills that need.

Agile was developed in response to quality and efficiency concerns posed by existing methods of software development. Too often software projects would deliver software with low customer value, poor quality, and were late to market/costly to develop. Agile attempts to solve those problems that are common to any type of software development.

In addition, Agile brings many quality and safety risk management benefits that are good for the Medical Device world. Some of those include:

• Continuous focus on safety and customer value through backlog prioritization/management and customer feedback
• Continuous assessment of quality through continuous integration and testing
• Continuous evolution of the product and the process that produces it
• Continuous focus on getting to “done” to demonstrate the completion of deliverables and activities that satisfy the Quality Management System
• Continuous participation in and collective ownership of risk management

Agile’s principles, practices, and life-cycle model have been viewed as incompatible with regulatory agency requirements and expectations for a medical device software quality management system. For example, the Agile Manifesto has value statements that seem contrary to the values of a quality management system. Fortunately, along with some strong and perhaps controversial values and principles, Agile also brings the concept of adapting the needs of the context, taking into account all of their customer’s and shareholders needs.

Purpose of this TIR

Agile is compatible and can be used in a regulated environment.

This TIR will examine Agile’s goals, values, principles, and practices, and provide guidance on how to apply Agile to medical device software development. It will:

• Provide motivation for the use of Agile.
• Clarify misconceptions about the suitability of Agile.
• Provide direction on the application of Agile so as to meet Quality System Requirements.

This TIR will provide guidance on the proper way to apply Agile.

(NEED TO REVISIT THIS)

This TIR provides guidance on how to achieve the benefits of Agile while being compliant with medical device regulations.

Scope

This TIR provides perspectives on the application of Agile during medical device software development. It relates them to the following existing standards, regulations and guidance:

• EN 62304 – Life Cycle Requirements for Medical Device Software
• Design Controls as required by CFR – Quality System Regulations
• FDA guidance on premarket software guidance and general principles of software validation
• The following groups are the intended audience for this TIR:
• Medical device manufacturers who are planning to use Agile techniques
• Manufacturers who are currently practicing agile and are entering the regulated (medical device) space, for example, Electronic Health Record manufacturers
• Software development teams, including software test and quality groups
• Senior Management, Project Managers, Quality Managers
• Quality systems and Regulatory affairs personnel
• Internal and external auditors
• Regulating bodies, agencies and organization’s responsible for overseeing the safety and effectiveness of medical devices

This TIR is not intended to be used as an educational tool or tutorial for the following:

• Agile Development practice
• Quality System Regulations

This TIR does not provide perspectives for the following standards:

• 80002-1
• ISO 14971

This Technical Information Report (TIR) should be regarded as a reference and as a guidance which is intended to provide recommendations and perspectives for complying with international standards and FDA guidance documents when using agile practices in the development of medical device software. This TIR is not intended to be a prescription for a specific situation or method.

More Agile Momentum in Healthcare: GE Agile Conference

Craig Langenfeld, of Rally Software, (Craig has been providing some thought leadership and has been working directly with me on this blog series) just brought me up to date on another leading indicator of agile’s march across the chasm into high assurance development.  Last month, Craig attended the first ever GE Agile Conference at the GE Advanced Manufacturing & Software Technology Center in Detroit, Michigan.

The conference was hosted by Carl Shaw, GE Healthcare Global IT Agile Champion (kind of a telling title, eh?) .  Craig noted that topics ranged from basic, introductory Agile concepts to advanced examples of using Agile to build medical devices at GE.

Ryan Martens, Rally Founder and CTO, was the Keynote Speaker on the last day of the three-day conference.  He spoke on applying Agile concepts in a highly innovative world to get products in the hands of those who can better social wellbeing faster.

Craig also noted that Brian Pate of SoftwareCPR, gave a presentation on “Agile methods for medical device software.” This presentation is available on their website (

/SCPRed/AgileForMedicalDevices-SoftwareCPRpresentation101010.pdf

It’s a solid presentation with some guidance on IEC62304 (an IEC and ISO international standard for medical device software, one that is harmonized with FDA CFR 820.30. see my post.). It’s a great example of what this series is all about.

Brian notes three common misconceptions that can interfere with the adoption of agile methods in medical devices, including:

1)   lack of obvious rigorous treatment of requirements

2)   no formal verification and validation in normal agile practice

3)   no formal agile mechanisms for dealing with risk assessment hazard, and hazard mitigation.

But he goes on to illustrate how to deal with these misconceptions in a high-assurance development model as well. Refer directly to that presentation for his advice.

I’ll be dealing with these issues as well in the upcoming series. In the next post or two in this series, I’ll suggest an agile lifecycle model that can support rigorously assured software quality and provides recommendations for continuous verification and as-necessary validation practices. Then we’ll get into the meat of specific practices that agile teams can use to develop such software, in an agile, safe and efficacious way.

Note: Special thanks to Craig Langenthal of Rally, Carl Shaw of GE Healthcare and Brian Pate of Software CPR for providing content and permissions for this particular post.

Agile in Regulated Environment: Abbott Labs Experience Report

In earlier posts in this series on Agile Development as Applied in High Assurance and Regulated Environments, I mentioned that applying agile development in the context of highly regulated environments is not a new thought, and at least some others have gone down this road before. And while the analysis of CFR 820.30 we did in the last post might indicate a waterfall mentality, the fact is that it doesn’t have to be that way at all.

One real-world example is from Abbott Labs in the development of a nucleic acid purification platform and companion real-time analyzer (whatever that is!) .  Some of the team members gave a presentation at Agile 2009 in Chicago. (I’ve contacted the authors to see if the PPT is still posted on line somewhere. I can’t seem to find it now. If I hear back, I’ll update this post with a link at some point.)

Here’s a few “grabbers” for context:

– “software was developed in iterations (short time-boxed iterations or sprints) and released in increments (collection of iterations, validated etc). –

– it is a myth that you can predict the requirements up-front

– Results: estimated team size and schedule reduction of 20%-30%.  Estimated cost savings of 35%-50%.

– at time of launch, a number of features, once thought to be essential, were not included. Some were deferred for up to three years …nonetheless the product was highly successful and trading off features for three years of sales is an easy choice.”

This surely speaks to the fact that medical device doesn’t have to be developed in a waterfall model, and indeed when it isn’t, typical agile business benefits may well result. The whitepaper on the same topic, Adopting Agile in an FDA Regulated Environment can be purchased here.

In future posts, I’ll be introducing an agile process model for use in high assurance software development that reflects this key fact.

[Nov 10 Update. A reader, Rob Matheson found the deck on line at http://c-spin.net/2009/cspin200909Agile_In_Regulated_Experience_Report.ppt. It actually has deeper  content than the written report.

Thanks Rob!]