Enhanced Intervention Detection in Aseptic Fill Using AI/ML
We will show how continuous, real-time capturing of data with immediate data analysis by an ML algorithm can improve control over a critical quality attribute. The ML-analyzed data provides the evidence for validation of the change by demonstrating more control over the process along with a decrease in process risks.
Implementing and validating a new process, or maintaining the validation status during process changes, is a requirement of the pharmaceutical industry. From a process life cycle perspective, validating and maintaining the state of validation is the “maintenance of the process in a state of control during routine commercial production.”
The Validation 4.0 approach offers a more sound and scientific approach than merely “revalidating the process” and should enable faster adoption of new technologies and innovation in a regulated environment. The Validation 4.0 approach shows how the control strategy mitigates risks to product quality to ensure patient safety.
This case study demonstrates the implementation of an enhanced control strategy for aseptic technique in an isolator or a restricted access barrier system (RABS). The innovative control strategy involves the use of a ML vision system for monitoring potential intrusions during human interventions into the Grade “A” environment (i.e., the aseptic critical zone) on the RABS filling line.
The Current Paradigm
The current control approach for aseptically filled sterile products is the use of media fills. Media fills are a process simulation because sterility testing all product vials is not feasible. Typically, production lines are initially qualified using three media fill runs (aseptic process simulations) followed by a single media fill run performed semiannually to show continued control. Modifications to the process usually require requalification (including in some cases another media fill). For a media fill to be considered successful, there should be no growth detected, and the number of “contaminated” vials must meet the warning limit and action limit criteria established by the most recent regulatory agencies’ and industry standards.
The aseptic process simulations serve to evaluate the overall production environment as established through aseptic controls such as cleaning and sterilization of the room and process equipment, air filtration (monitoring air flow rate and pressure differentials), continuous microbial monitoring (via sedimentation plates), and nonviable particulate (particulate counters) monitoring of the environment.
These media fill process simulations also serve to evaluate the aseptic technique used during operator interventions. Interventions during these simulations must meet certain requirements, as outlined by the US FDA:
- At least three consecutive separate successful runs be performed during initial line qualification
- Routine semiannual qualifications are conducted for each processing line
- Representative activities and interventions of each shift and shift changeover should be incorporated into the design of the semiannual qualification program
- All personnel authorized to enter the aseptic processing room during manufacturing, including technicians and maintenance personnel, should participate in a media fill at least once a year
Interventions are required and inevitable, but human interventions have the greatest potential variability and subjectivity and therefore pose the biggest risk to product sterility. Currently, the effect of these manual interventions cannot be monitored directly. More specifically, there is no objective determination as to whether an operator intruded into the critical zone of an aseptic filling area during the intervention. Operators and secondary monitors may detect some obvious cases of intrusion into a critical zone, but not every instance may be discovered. Also, determining whether an intrusion occurred and documenting that intrusion adds to the challenges of performing aseptic technique properly (and perhaps adds to the subjectivity of the current control strategies).
The current control strategies used to mitigate the inherent risks of manual interventions are:
- Extensive operator training and operator adherence to procedures
- Preventive rejection of potentially contaminated objects (i.e., the culling of containers)
- Frequent verification by performing media fills at certain intervals
As an additional control strategy, some companies use a second person to monitor and document interventions. Using this second person mitigates the risks associated with the additional distraction of manually documenting the interventions. These control strategies, however, still depend on a subjective determination as to whether an operator intruded into a critical zone during the intervention.
Also, consider the detection of a “contaminated” vial from a media fill run from a risk perspective. Media fills may have an inherently low probability of detection because the following sequential events must occur for a media fill vial to exhibit a positive-growth result:
- A microbe is on the operator glove.
- The operator enters the critical zone.
- The microbe is transferred from the operator glove to the critical zone.
- The microbe enters a container.
- The microbe grows in the container during incubation.
The preceding sequence of events and their probability are an aseptic technique “black box” and are generally considered as a single event. A low probability of detection implies that media fills are best for detecting gross microbial issues. However, if an operator entering the critical zone can always be definitively determined, then probability of detection becomes very high, and the control strategy is improved, with a large degree of subjectivity removed.
As an enhanced control strategy, this case study looks at an ML vision system that objectively determines when an operator has intruded into a critical zone. This case study focuses on the process validation aspects and does not explore the validation of the ML algorithm and predictive model software. Currently, no definitive method for ML software validation has been established by either regulators or industry. The ML software validation approach used for this case study is explained elsewhere.
Case Study: An Enhanced Control Strategy
This case study focuses on the critical quality attribute of sterility and is therefore focused on the contamination control strategy. It explains a new approach for controlling the sterile attribute achieved using an ML vision system to monitor interventions into the aseptic critical zone on the RABS filling line. The objective of this case study analysis is to use QbD, KM, and RM principles to demonstrate that this new control strategy is more effective than the existing control strategy for aseptic technique human interventions. This improvement in the control strategy is characterized by better detection of intrusions into the critical zone during human interventions.
As an enhanced control strategy, this case study looks at an ML vision system that objectively determines when an operator has intruded into a critical zone.
To implement this new control strategy for monitoring interventions, one main equipment change was required: multiple cameras had to be retro-fitted into the existing RABS aseptic core. Cameras were positioned to provide different views and perspectives into the same critical zone filling and handling areas. Adequate coverage was then verified using smoke studies. These cameras supply time-stamped video images for analysis. The analysis is then performed using two distinct ML random forest (RF) classification algorithms for two objectives:
- To detect glove insertions into the isolator or RABS.
- Detection capabilities localize and classify the intervention into different actions.
- This ML detection extends beyond current technology (for example, using light barriers) for a simple detection of “glove port insertion.”
- To differentiate between critical and noncritical interventions.
The ML RF algorithms establish a mathematical model (that is, a correlation) between the video images and whether an operator has intruded into the critical zone. To establish that correlation, the ML algorithm was trained with labeled images, including borderline or edge cases. To properly identify and label the borderline/edge images, smoke studies were used to precisely define the coordinates of the critical zone. Line speeds were established and qualified prior to the smoke studies, and media fills were performed after the smoke studies to capture a significant volume of video images for training the ML RF algorithms. After extensive training, testing, and validation, the ML RF algorithms were verified to provide an accurate analysis of when an operator intruded into the critical zone during an intervention.
These ML RF analyses provide a fully automated, objective verification of operator adherence to aseptic technique (that is, avoiding the critical zone during interventions) while minimizing risk by releasing operators from distracting activities (like documentation). Vials are rejected or discarded when critical zone intrusion is detected during operator interventions. The relevant parts of the time-stamped video are retained as part of the batch record and replayed for operator training/retraining purposes.
This case study shows that the Validation 4.0 approach, applied to a process change, ultimately demonstrates that risk is lowered and that the change provides increased control over patient safety as compared to the current control strategy.
Table 1 lists the key benefits accomplished by the change from human monitoring to monitoring by ML analysis. Also in the table are data integrity upgrades in terms of some ALCOA+ improvements (ALCOA = Attributable, Legible, Contemporaneous, Original, and Accurate; + = Complete, Consistent, Enduring and Available).
Because the ML RF analysis provides 100% real-time, automated monitoring of operator interventions, it may remove the need for media fill assessments of the operator interventions. The aseptic environment control strategies combined with the ML RF analysis of operator interventions may be sufficient for isolator or RABS production suites. Aseptic environment control strategies include:
- Cleaning and sterilization of the room and process equipment
- Air filtration (monitoring air flow rate and pressure differentials)
- Continuous microbial monitoring (via sedimentation plates)
- Nonviable particulate counters monitoring
A failure modes effects analysis (FMEA), or other RM approach, of the aseptic filling operation may be sufficient if it can be clearly demonstrated that the control strategy provides sufficient continuous monitoring, and that all sterility failure risks have been adequately mitigated. Fundamentally, ML-enhanced vision systems—where the images are evaluated for “correct” operator activities—provide an inline/online process analytical technology for achieving continued or continuous process validation. The authors believe that this approach is a significant improvement over the current media fill process simulation control strategy, and may someday remove the need for media fills.
Applying Validation 4.0 to the Case Study
As stated previously, the Validation 4.0 approach aims to validate a process by demonstrating the control strategy is in place and effective using QbD, KM, and RM principles. These principles may be used in a three-step approach to show the importance of what process aspects need to be controlled. Step 1 takes a QbD approach to provide a high-level process flow showing the relationship of sterile product filling and media fills. Then, the data is identified at each process step, whether they are inputs, outputs, or transacted data. Step 2 uses RM to assess the risk of each data point within the context of the process step and identify the risk control measures for the more critical process steps/data points. For purposes of this case study, these steps are demonstrated by comparing the current paradigm against this change to the ML visions system. Step 3 builds the evidence that the control strategy is in place and effective.
We will show, by comparison, how the change discussed previously will provide equal or better control around mitigating risk to product quality and ultimately patient safety.
Step 1: Process and Data Flow
We start with a QbD approach rooted in product and process understanding. Then we map out the process flow and identify the data created, transacted, and processed at each step. Figure 1 provides process flow steps (QbD) applicable for this case study. It is not a detailed view of all steps necessary to manufacture a sterile filled product. Figure 2 provides the process data elements steps (KM).
Step 2: Risk Assessment (Risk Management)
RM typically looks at three factors to assess risk: severity, frequency of occurrence, and probability of detection.
Regardless, most likely every batch has some interventions that intrude into a critical zone, and therefore the frequency of occurrence may be considered moderate to moderate-high. The risk factor most affected by the change is probability of detection. Visual, human monitoring may be categorized as moderate to moderate-low because the borderline/edge cases may not be consistently detected by the visual observer, whereas video imaging coupled with ML RF analysis provides a high probability of detection.
Also, as discussed previously, a media fill may not detect a poor performance of an aseptic technique because a series of sequential events must occur to detect a poor performance with a positive-growth test result from the media fill.
As illustrated in the risk assessment diagram in Figure 3 (before change) and Figure 4 (after change), the criticality of the data elements from the process does not change. Based on QbD, the criticality of the data remains, as it directly relates to controlling the critical quality attribute: a sterile product. What does change is the detectability of failure of these data elements upon implementing the artificial intelligence (AI; video analysis) and automated recording of critical data elements.
| Current Control Strategy – Human Monitoring |
Enhanced Control Strategy – Monitoring by ML Analysis |
Data Integrity Improvements |
|---|---|---|
| Subjective human observation to classify interventions, which can result in human visual variability in real time coupled with real-time decision-making. |
ML is trained by a human but not under real-time constraints of filling operations. Objective ML analysis then identifies and classifies the interventions limiting real-time fluctuations. |
Accurate |
| Delayed operator recording of activity after completion. | Real-time capture of activity (no distractions/no forgetting). | Contemporaneous Accurate |
| Media fill assessment of operator interventions. | Continuous 100% monitoring for the assessment of operator interventions. |
Consistent |
Step 3: Control Strategy
The process flow step shows that the effectiveness of aseptic technique is only evaluated during the media fills and subjectively monitored during the production runs. The data flow step shows that continuous video-capture and ML analysis provides data that is more accurate, contemporaneous, consistent, and complete compared to human, visual observation during media fills and production runs.
The risk assessment step shows that the risk of not detecting an intrusion into the critical zone is eliminated (or at least greatly reduced) compared to the human, visual observations of operator interventions. As well, potentially the occurrence of intrusions goes down because the operators are focused on doing the work rather than documenting the work.
Thus, from the QbD, KM, and RM perspective, the change enables a much more rigorous control of the process and achieves an enhanced control strategy. The control strategy consists of operational and technical controls that are in place to mitigate high-risk data elements as defined by QbD concepts; the data elements that are closer to managing or characterizing our product are most important to control.
Summary of Control Strategy Changes
With the preceding evaluation of the critical data elements associated with the aseptic processing of sterile products, we have demonstrated that the ML-enhanced, automated, continuous monitoring provides a better level of control than the traditional media fill approach for documenting sterility assurance.
The evidence provided by the control strategy, the risk assessment, and the rigorous validation of the ML model demonstrate that the proposed change, the ML-enhanced vision system observation of operator interventions, is an improvement over current control methodologies. The evidence shows better control (lower risk) associated with the sterile product than before the change, as summarized in Table 1. Changes from control strategy, quality RM, and validation perspectives are discussed next.
Control Strategy Perspective
Operator interventions and the possible/probable intrusions into critical aseptic zones is the current “weak link” in control strategy for aseptic processing and media fill validation. ML enhances the monitoring of operator interventions by automating the detection of operator intrusions into the critical aseptic zone(s) and monitoring the process continuously. The ML-enhanced, automated, continuous monitoring of operator interventions is objective rather than subjective and is therefore an improved process control over the subjective human observation of operator interventions.
Quality RM Perspective
Media fills represent a high risk. The unknown release of aseptically produced product heavily relies on statistics and decisions that are derived based on the unknown probability of the occurrence of a microbial contamination during a media fill (versus a production run) and the unknown likelihood of detection (via growth in the limited number of media fill runs). Therefore, outcomes of media fills/aseptic process simulations can only indicate an approximation of actual sterile filling.
| What’s Changing | Supporting Evidence |
|---|---|
Intended changes
|
ML data
|
Unintended positive changes
|
In principle, in other training circumstances where the feedback is real time and ongoing, operator improvements are typically seen.
|
The ML-enhanced, automated continuous monitoring of operator interventions represents a lower risk because of the high likelihood of detections (of operator intrusions into a critical zone). The probability of occurrence of any intrusion into a critical zone is assumed to be at least equivalent to what might occur during a media fill. Additionally, any intrusion has the potential to cause a microbial contamination.
Therefore, compared to the unknown likelihood of detection of the media fill approach, the ML-enhanced, automated continuous monitoring approach provides an extremely high likelihood of detection. From a quality RM perspective, the ML-enhanced, automated continuous monitoring, with its high likelihood of detection, results in a much lower risk overall.
Validation Perspective
The ML algorithms have been designed, extensively trained, rigorously tested, and verified by performance metrics to provide an accurate analysis of operator intrusion into the critical aseptic zone during interventions.
Any changes made to the processes would require normal change control, including change impact assessment and verification that the control strategies remain effective. This would include evaluation of the ML algorithm and its ability to detect any new/changed critical zones due to changed operation. Retraining of the ML algorithm may be necessary to detect any new or changed operation.
Conclusion
The use of AI/ML as discussed in this article provides a rapid and automated personnel monitoring method in critical zones for an aseptic filling process within a closed environment. For the acceptance of AI/ML as a rapid and automated personnel monitoring method, the principles and approach used for the acceptance of rapid and automated microbial monitoring methods should be considered. Volume 4, Annex 1 of EU Guidelines for GMPs for Medicinal Products for Human and Veterinary Use, Section 9.28 states: “rapid and automated microbial monitoring methods may be adopted after validation has demonstrated their equivalency or superiority to the established methods.”
This case study shows that the Validation 4.0 approach, applied to a process change, ultimately demonstrates that risk is lowered and that the change provides increased control over patient safety as compared to the current control strategy. We identified that the monitoring of operator interventions is a process parameter critical for patient safety. Through quality RM techniques, the current controls of media fills and observed interventions during routing production were shown to be of a higher risk than the control achieved by the ML-enhanced monitoring of operator interventions.
Furthermore, we are potentially able to certify operators to a higher level of confidence because they can focus on the filling operation while the ML-enhanced monitoring with video recording can continuously document interventions in real time. The objective evidence demonstrates that a ML-enhanced vision system provides a better level of control for sterile fill processing than with the current aseptic validation methodology. This demonstration of a higher level of control essentially demonstrates that the process is validated after change.
