The step-by-step guide to FDA’s software validation process

Back in January 2002, the Food and Drug Administration issued a guideline called ‘General Principles of Software Validation; Final Guidance for Industry and FDA staff’. To this day, this guideline is used as a crucial document for overseeing and validating medical software and this is what we’ll focus on in this article. 

It’s important to mention here that on November 4th, 2021, the FDA released a document that may change the face of the medical software validation/verification and any related processes. ‘Content of Premarket Submissions for 2 Device Software Functions’ is now public in its draft, comment-only version with feedback being collected until the end of 2022. 

As it was with previous articles, this blog post was prepared in collaboration with Krzysztof Minicki, Director of Healthcare and Life Sciences at Spyrosoft. 

What is medical software verification according to the FDA? 

The goal of the medical software verification is to prove that our specified software requirements are consistent, complete and have been properly implemented, and our medical applications work as planned. It serves as confirmation that what we planned to do when it comes to the software implementation has been carried out as required.  

To give you an example, if the application I developed was to provide a daily/weekly/monthly report for my patients’ saturation, temperature, blood pressure in a specific format, then the verification will mean that the application is, in fact, able to generate the report as required. 

What is medical software validation according to the FDA? 

Validation must confirm – through a series of various activities resulting in objective, measurable evidence – that the developed medical software fulfils its function as described in Intended Use and is effective and safe to operate. As with verification, validation is also accomplished through a series of activities to check if the specific groups of requirements have been implemented consistently, completely and as a set of functionalities. This process is focused on providing evidence that the implementation effectively and properly fulfils the requirements and the intended use for this specific medical software.  

Going back to the example, we would have to show the report to a doctor that will be using it, and they would have to confirm that this will be – in fact – useful, clear, accessible and aligned with the initial medical expectations. The clue here is delivering an external validation for our medical software. 

Why – in practice – are verification and validation not that different from each other? 

What’s important to mention here is that the ‘General Principles of Software Validation; Final Guidance for Industry and FDA staff’ guides us through both the verification and the validation process.  

While the verification and the validation have both been separated in the medical sector documentation, in practice – the division is arbitrary at times. It makes sense from a business perspective, though – as it clearly divides the responsibilities between our customers and our team. Once we receive the specifications, the requirements and a list of functionalities from a customer, we can then use those to develop medical software and then verify it using manual and automated tests as well as other activities.  

Once the product is ready, we can then hand it over to the customer to validate it, although small-scale validations can be carried out throughout the development process. 

IQ/OQ/PQ – the core aspects of medical software validation 

The FDA specifies 3 focuses that will be crucial for medical software validation. If you have any medical software, you should be able to validate and verify it using these 3 perspectives: 

Installation Qualification (IQ) 

You can install and use the software safely.  

Installation Qualification (IQ) is defined by the FDA as ‘establishing confidence that process equipment and ancillary systems are compliant with appropriate codes and approved design intentions, and that manufacturer’s recommendations are suitably considered’. 

Operational Qualification (OQ) 

The software has full operational functionality, or as FDA states, ‘establishing confidence that process equipment and sub-systems are capable of consistently operating within established limits and tolerances’. 

Performance Qualification (PQ) 

The software is effective, accessible and performs well even with an increased workload. The FDA defines it as ‘establishing confidence through appropriate testing that the finished product produced by a specified process meets all release requirements for functionality and safety’. 

What are the principles of software validation? 

The most important are the requirements – especially those related to the system – as they will be later used as a basis for validation. System requirements are later divided into software requirements, software architecture, software components, then again into more specific software requirements and details designs.  

Software validation should be planned at the software lifecycle level. The plan itself should be adjusted to a specific medical application, its class and its risks. The effort will be higher for software with higher risk and class II-III.  

At the planning level, we also need to establish how we want to approach any bugs and software failures that are inevitably related to the software development process. To do that, we need to answer this question: 

• How do we want to fix these bugs and how will we verify and validate the fixes? 

As for the validation process itself, we need to check whether the requirements, including those specific ones listed above, are what was eventually built and delivered during the software development process. We also need to explain in the documentation not only if each of these requirements were addressed but also how it was done. This is usually resolved by a list of requirements registered within a matrix with a list of ways and specific functionalities that we developed to address these requirements. This register should also include test cases for each requirement/functionality. 

Once the matrix is complete, one of the validation steps can be deemed successful. 

It’s important to remember that validation is not about testing itself, though. On the contrary, any tests should also be approved and validated before using them for a project. 

How to approach any changes to the product? 

Another thing we should plan for is how we will approach any change to our original build and to what extent we will test our medical software after releasing these changes to production. Conducting extensive testing after a minor change may not be necessary, but we should clearly state what our approach will be after a small adjustment. You may need to conduct an analysis to estimate the impact this adjustment has on our software and the risk for the system as a whole. Based on the results of the analysis, you can then plan our next steps, including regressive tests, to ensure that the change is properly validated and verified. 

The extent of our validation and verification strategy will largely depend on the level of risk and the complexity of our software. The less risk it poses to the patient, the simpler our strategy can be. Eventually, we have to be able to prove that all criteria set at the planning stage have been met. 

What’s particularly important is the fact that both the validation and the verification process should be conducted by independent group of specialists from your company or a third-party body. It often happens that the verification is done internally by the software provider, and the validation process is conducted by the company that requested the software to be developed – it is not a rule, though. In practice, both parties cooperate to complete the processes at various stages of development.  

For every step of the processes, we also need to set acceptance criteria – in short: the input should match the output.  

Human Factor Studies – the final step of the validation process 

If you’re a medical software provider, you’re obligated to conduct human factor studies as the crucial step of the validation process. Software is then tested using a simulated environment, yet with a group of its potential end-users. This group of users must be selected carefully taking into consideration factors such as age, race, gender, educational background, language and location. There are two types of HMS depending on when they are conducted: 

• Formative Human Factor Studies held at the beginning of the development process, 

• Summative Human Factor Studies reserved for the final stage of the validation process. 

These studies help to validate whether the final product is useful, accessible and meets the requirements. The sessions are then summarised in a report on the usability of the medical product, including software.  

Over to you 

Do you need support in validating your medical software in line with the FDA requirements? Contact our Director of Healthcare and Life Sciences, Krzysztof Minicki and his team on LinkedIn or via our website.