Transforming the Landscape of Computer System Validation: Harnessing the Potential of Computer Software Assurance

In sectors such as pharmaceuticals, medical devices, and other regulated industries, the significance of Computer System Validation (CSV) cannot be overstated. This crucial process ensures the reliability and adherence to regulations of computer systems employed for data collection, processing, and reporting. Successful CSV not only ensures data integrity and product quality but also enhances overall operational efficiency and regulatory compliance. In this comprehensive blog post, we will explore the realm of CSV, its challenges, and how Computer Software Assurance (CSA) is poised to revolutionize the landscape.

CSV: An Essential Process

Computer System Validation (CSV) is an ongoing process that establishes documented evidence, providing a high level of assurance that a computerized system and its components are suitable for their intended use. This validation requirement extends to computers and automated data processing systems used within production or quality systems, covering various aspects such as software for design, development, manufacturing, and the organization's quality management system.

While CSV is indispensable, it's crucial to note that different validation requirements and distinct processes apply to various software categories. For instance, medical device software has its unique set of validation requirements, and not all IT infrastructure and business operation software fall under the same validation purview.

CSV involves various phases, including planning, risk assessment, design qualification, installation qualification, operational qualification, and performance qualification. The overarching goal is to verify that these systems consistently produce accurate and reliable results, maintain data integrity, and adhere to Good Manufacturing Practices (GMP) and other relevant standards.

However, the conventional CSV methodology has its drawbacks. As recently outlined by the FDA, CSV suffers from impediments such as the complexity of documentation and processes, and insufficient understanding of regulatory expectations leading to excessive evidence collection. Inadequate product and supplier maturity evaluation, coupled with communication issues, results in redundant activities during software implementation. Traditional risk assessments divert focus to unintended mitigations, burdensome testing, and unnecessary controls. Script errors in testing contribute to a high number of deviations, consuming time without adding value. The checklist-driven approach to validation dilutes the focus from essential requirements, leading to redundant documentation and testing, hampering CSV efficiency. The process places significant emphasis on documentation, leading manufacturers to allocate a substantial portion of their time to document generation. Approximately 80% of the time is dedicated to producing documentation, leaving only 20% for actual software testing. This skewed ratio underscores the need for a more efficient and streamlined approach.

In Figure 1 below, the shift in paradigm between CSV and CSA is illustrated. 

The Emergence of CSA: A Shift in Paradigm

Computer Software Assurance (CSA) emerges as a transformative force in software validation. It challenges the traditional CSV methodology and introduces a more flexible and efficient approach based on tailored validation efforts. The essence of CSA lies in prioritizing critical thinking, assurance needs, testing, and documentation, in that sequence. Documentation remains a vital part of the process, but CSA aims to promote patient safety by emphasizing critical thinking in the validation process. The aim is to ensure delivery of a high-quality, safe product that meets patient needs, rather than robust documentation that passes an audit cycle.

CSA operates as a risk-based approach, aiming to establish software suitability for its intended purpose while optimizing the validation process. It follows a least-burdensome approach, ensuring that the validation efforts align with the associated risks.

In its draft guidance on CSA, The U.S. Food and Drug Administration (FDA) provides a general approach for CSA, which involves the following steps:

- Determining whether the software is intended for use as part of the production/quality system.

- Evaluating the level of risk if the software was to fail to perform as intended.

- Identifying assurance activities commensurate with the risk.

- Capturing sufficient evidence to demonstrate that the software was assessed and performs as intended.

This approach focuses on building confidence through the testing of high-risk functions. Since most applications consist of a mix of standard, off-the-shelf, and configurable or customizable elements, CSA encourages software categorization at the function level. This ensures that validation efforts are tailored to the specific needs of the software.

Furthermore, the CSA methodology encourages users to take advantage of technology that wasn't available 20 years ago, like cloud computing and validation management software, as well as rely on the vendor’s existing documentation. 

A Practical Overview of CSA

In practice, Computer Software Assurance (CSA) operates as a risk-based strategy to ensure that software aligns with its intended purpose. It involves assessing potential risks to the safety and quality of a device and determining the level of assurance needed. This approach is adaptable and flexible, adjusting the rigor of validation activities to the assessed level of risk.

The process of establishing computer system assurance comprises four key steps:

1. Identifying the intended use: The level of assurance required for a computer system depends on its intended use, with higher assurance necessary for systems closely linked to production or quality control.

2. Determining the risk-based approach: A risk-based analysis is used to assess the appropriate assurance activities, considering the potential impact of software failures on medical device safety.

3. Determining appropriate assurance activities: The level of rigor for assurance activities aligns with the level of risk, with higher-risk software requiring more comprehensive testing and evidence collection.

4. Establishing an appropriate record: Manufacturers maintain records that encompass the software's intended use, risk level, and documentation of assurance activities performed.

In Figure 2 below, a schematic representation of CSA steps can be seen. 

One notable feature of CSA is its emphasis on unscripted testing. Unlike traditional scripted testing, which prescribes specific actions in a test case, this dynamic testing method liberates testers from rigid instructions, allowing for free-form testing with sufficient but least burdensome documentation. Unscripted testing primarily documents what needs to be tested in advance on a high level, leaving the tester to fill in the details during testing and redirecting the effort toward challenging software functionality to uncover defects.

Streamlining Documentation: Unscripted Does Not Mean Undocumented

With its risk-based strategy, Computer Software Assurance (CSA) focuses on optimizing resource allocation and encourages a streamlined documentation approach. During validation projects, it's imperative to create and collect objective records demonstrating that the software tool has been assessed and is considered validated. Once the software is validated and released for use, it enters the maintenance phase, where its operation is monitored, and any errors are analyzed. The validation needs are reassessed whenever the software is updated.

In conclusion, CSA represents a transformative approach to software validation that follows a least-burdensome philosophy, ensuring that the validation burden is no more than necessary to address the risk and optimizing effort and resources for better quality. While awaiting final guidance, practical changes can be made to embrace CSA concepts, such as utilizing critical thinking, reducing screenshots in favor of log files, digitizing the process with the help of tools, focusing on quality-relevant requirements, and introducing unscripted testing. CSA offers a more agile, risk-based, and efficient approach to computer system validation.

Conclusion

In summary, implementing CSA may or may not reduce validation time but will enhance validation efforts, directing time and resources toward improved quality. There's no need to fear or delay CSA adoption until the final guidance is released, as its concepts align with current regulations. 

Stepwise improvements can be made without major updates, such as employing critical thinking, reducing screenshots in favor of log files, and digitizing processes with ALM and other tools. We recommend focusing on documenting quality-relevant requirements, introducing unscripted testing, and utilizing the existing supplier documentation. More substantial changes like categorizing software at the feature level, reducing risk through assurance activities, and embracing Agile methodologies in software development could be implemented at the next level. 

If you need support with the implementation of CSA and applying it in practice for your medical device software or AI-containing software, please contact us at: info@quaregia.com. 

Last updated: 2024-01-29