Valuable Unit Tests in a Software Medical Device, Part 7

Regulated Environment Needs Per 21 CFR Part 820 (Subpart C—Design Controls):

(f) Design verification. Each manufacturer shall establish and maintain procedures for verifying the device design. Design verification shall confirm that the design output meets the design input requirements. The results of the design verification, including identification of the design, method(s), the date, and the individual(s) performing the verification, shall be documented in the DHF.

Simply put, our functional unit tests must be a part of our DHF, and we must document each test, each test result (success or failure) and tie tests and outcomes to specific software releases. This is made extremely easy with a continuous integration environment in which builds and build outcomes (including test results) are stored on a server, labeled and linked to from our DHF. Indeed, what is sometimes a tedious task when it comes to manual execution and documentation of test results, becomes quite convenient.

The same is true of Design validation:

(g) Design validation. Each manufacturer shall establish and maintain procedures for validating the device design. Design validation shall be performed under defined operating conditions on initial production units, lots, or batches, or their equivalents. Design validation shall ensure that devices conform to defined user needs and intended uses and shall include testing of production units under actual or simulated use conditions. Design validation shall include software validation and risk analysis, where appropriate. The results of the design validation, including identification of the design, method(s), the date, and the individual(s) performing the validation, shall be documented in the DHF.

Because our CI environment packages build and test conditions at a given point in time, we can successfully satisfy the requirements laid out by 21 CFR 820 Subpart C, Section 820.30 (f) and (g) with very little effort. We simply allow our CI environment to do that which is does best, and that which a human tester may spend many hours attempting to do with accuracy.

Document the Approach

As discussed, all these tests are indeed very helpful to the creation of good software. However, without a wise approach to incorporation of such tests in our FDA regulated environment, they are of little use in any auditable capacity. It is necessary to document our approach to unit test usage and documentation within our Standard Operating Procedures and work instructions, and this is to be documented in much the same way that we would document any manual verification and validation test activities.

To this end, it is necessary to make our unit tests and their outputs an integral part of our Design History File (DHF). Each test must be traceable, and this means that unit tests are given unique identifiers. These unique identifiers are very easily assigned using an approach in which we organize tests in logical units (for example, by functional area) and label tests sequentially.
Label and Trace Tests

An approach that I have taken in the past is to assign some high-level numeric identified and a secondary sub-identifier that is used for the specific test. For example, we may have the following functional areas: user session, audit log, data input, data output and web user interface tests (these are very generic examples of functional areas, granted). Given such functional areas I would label each test, using test naming annotations, with the following high level identifiers:

1000: user session tests
2000: audit log tests
3000: data input tests
4000: data output tests
5000: web user interface tests

Within each test it is then necessary to go a step further, applying some sequential identifier to each test. For example, the user test package may include tests for functional requirements such as user login, user logout, session expiration and a multiple-user-login concurrency test.

In such a scenario, we would label the tests as follows:

1000_010: user login
1000_020: user logout
1000_030: session expiration
1000_040: multiple concurrent user login

Using TestNG syntax, along with proper Javadoc comments, it is very easy to label and describe a test such that inclusion in our DHF is indeed very simple.

* Test basic user login and session creation with a valid user.
* @throws Exception

@Test(dependsOnMethods = {“testActivePatientIntegrationDisabled”},
groups = {“TS0005_AUTO_VE1023”})

public void testActivePatientIntegrationEnabled() throws Exception {
Fixture myApp new Fixture();
UserSession mySession = fixture.login(“test_user”, “test_password”);


Any numbering we choose to use for these tests is fine, as long as we document our approach to test labeling in some project level document, for example a validation plan or master test plan. Such decisions are left to those who design and apply a quality system for the FDA regulated project. As most of us know by now, the FDA doesn’t tell us exactly how we are to do things, rather, we are simply told that we must create a good quality system, trace or requirements through design, incorporate the history in our DHF and be able to recreate build and test conditions.

If I make this all sound a little too easy it is because I believe it is easy. Too often we view cGMP guidance as a terrible hindrance to productivity. But we are in control of making things as efficient as we can.

Device Advice: Regulation and Guidance, Software Validation Guidelines,