ISPE’s APQ Program & Guides Advance Pharmaceutical Quality

The CAPA guide provides guidance, recommended tools, and suggested key performance indicators (KPIs) to assess, aspire, act on, and advance a CAPA system. To provide quantitative business context, ISPE has partnered with the University of St.Gallen in Switzerland to include in the APQ Program an optional operational excellence (OPEX) benchmarking exercise, which offers objective evidence of performance improvement to support ongoing in-vestment of time and resources. The CAPA guide is the first of a series aligned with ICH Q10, Pharmaceutical Quality System (PQS), elements and principles,1 with other APQ guides likely to publish in the near future.

Building ISPE’s Advancing Pharmaceutical Quality Program

For more than a decade, ISPE has actively supported industry efforts to understand, implement, and comment on several high-profile, often global, regulatory initiatives. Since 2018, these initiatives have been focused under the Regulatory Steering Council (RSC);2 examples of RSC efforts include:

• Product Quality Lifecycle Implementation (PQLI)®: Through PQLI, ISPE assists industry and regulators in advancing manufacturing sciences across the product life cycle to achieve excellence in drug development and pharmaceutical production.
• The Drug Shortages Initiative: This initiative is facilitating communication and creating tools to help improve industry’s capability to mitigate and prevent drug shortages.
• The APQ Initiative: This initiative is building industry-for-industry tools and programs to help companies assess and improve their quality operations.

In 2018, the RSC provided strong support and guidance when the Quality Metrics Core Team proposed that the Quality Metrics Initiative evolve into the APQ Program, with the concept being beta tested by developing assessment criteria, development tools, and KPIs for evaluating the maturity of a CAPA system. This proposal was based on ICH Q10 and described with supporting background and justifications in an article published in Pharmaceutical Engineering® in September/October 2018.3 Figure 1 summarizes the initial proposed APQ concept.

A preliminary quality assessment (PQA) would allow determination of the potential value of and need for a “deep dive” (i.e., thorough) examination of an assess-and-aspire series of activities. The assess-and-aspire component would be used to assess a company’s own quality maturity and decide, based on this assessment, how much the organization aspires to improve. Should the organization decide that they wish to improve, the program would point to tools and KPIs that would be the architects of improvement. Tools and KPIs to conduct these activities, along with those to assess, benchmark, and improve quality maturity, would be identified from those that are already available to the company. Where tools and KPIs do not exist, ISPE teams would propose new or alternative options.

• 1International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. “ICH Guideline Q10 on Pharmaceutical Quality System.” European Medicines Agency website. Published 2008. https://www.ema.europa.eu/en/documents/scientific-guideline/international-conference-harmonisation-technical-requirements-registration-pharmaceuticals-human_en.pdf
• 2Nosal, R., and S. P. Miksinski. “Update from the ISPE Regulatory Steering Council.” Pharmaceutical Engineering 40, no. 3 (2020): 22–23.
• 3Potter, C. J. “ISPE Proposes an Advancing Pharmaceutical Quality Program.” Pharmaceutical Engineering 38, no. 5 (2018): 24–28.

The goals, benefits, and principles set out in 2018 have remained unchanged, with the overarching goals of the APQ Program being to:

• Integrate quality management maturity, cultural and operational excellence principles, tools, and approaches.
• Foster industry ownership of quality beyond compliance.
• Promote effective and efficient use of resources.
• Support and incentivize continual improvement.
• Encourage self-improvement and supplier improvement.
• Enable structured benchmarking, knowledge sharing, and learning among companies.
• Increase the reliability of supply for quality products.
• Offer routes for delivering a sustainable competitive advantage.

Benefits were identified for industry, patients/consumers, health agencies, and ISPE (Table 1), and the following guiding principles for the program were established:

• Maintain simplicity.
• Be applicable across all sectors of the pharmaceutical industry.
• Deliver value and benefits for industry.
• Use “as-is” company data and site procedures as much as possible.
• Minimize additional work.
• Be “by industry, for industry.”
• Leverage existing benchmarking and performance management/OPEX methodologies and principles where relevant.
• Build on the ICH Q10 framework, with enhancements to include operational excellence and quality culture (Figure 2).
• Be complementary, where possible, to current regulatory initiatives promoting quality excellence, such as PIC/S data integrity guidance, the US FDA’s New Inspection Protocol Project (NIPP), and the MHRA data integrity guideline.

initial APQ proposal has been refined and enhanced as a result of the following:

• A pilot exercise run by an ISPE subteam using the ICH Q10 element CAPA as an indicator of company health. An effective CAPA system demonstrates whether issues are acknowledged, tracked, and, ultimately, remedied in an effective and permanent manner. Feedback led to refinement of maturity assessment descriptors, how KPIs are presented, and how improvement tools are aligned to maturity level.
• Continued collaboration with the University of St.Gallen leading to a formal memorandum of understanding whereby ISPE can use in the APQ Program, as an option, the OPEX benchmarking and quality maturity assessment tool, both of which were developed by St.Gallen.
• Increased FDA interest in quality management maturity.

Overall, these enhancements and refinements led to the program being renamed the Advancing Pharmaceutical Quality Program. Furthermore, the goals are being realized with publication of the CAPA guide as the first guide in ISPE’s APQ Guide series.

AAAA Framework and Benchmarking

The APQ Program has been designed with an assess, aspire, act, and advance (AAAA) framework as a core to provide formal continual improvement (CI) opportunities: Assessment allows a baseline to be established and opportunities for potential CI to be identified; aspire involves selecting and prioritizing the improvements to act upon; act requires a detailed, resourced action plan to be developed with targeted improvement outcomes; and, advance evaluates and confirms the required outcomes have been achieved.

The APQ AAAA framework is:

• A self-assessment process
• Composed of four distinct but interconnected stages
• Based on a five-step maturity model
• Intended as an iterative CI process
• A detailed quantitative and qualitative exercise with criteria to evaluate the current state of quality, diagnose gaps, and identify improvement opportunities

To provide a quantitative baseline, the University of St.Gallen OPEX benchmarking and quality maturity assessment tool is included as a pre-and post-benchmarking exercise for the APQ AAAA framework, as shown in Figure 3. This benchmarking could be performed optionally by St.Gallen or by self-application using tools provided by St.Gallen in the APQ Guide.

To assist practitioners performing the ISPE APQ AAAA process, each APQ guide will address:

• Background, overview, and structure of the APQ Program
• How to conduct the quantitative pre- and postassessments either in the St.Gallen benchmarking program or through internal use of methodology provided by St.Gallen
• How to conduct and score a deep-dive assess-and-aspire exercise for each ICH Q10 element
• How to set up an act-and-advance improvement program
• A case example to assist practitioners

The APQ Guide series is based on ICH Q10 and hence will cover the following elements: the CAPA system; management responsibilities and review; the change management system; and process performance and product quality monitoring system. Because management responsibilities and management review are strongly linked, one guide is being created for those two elements.

Notably and as shown in Figure 2, the concepts, principles, and tools given in ISPE’s cultural excellence work—for example, The Cultural Excellence Report4 and the ISPE/Parenteral Drug Association article on root cause analysis5—are embedded into the APQ AAAA framework.

In addition, there are ISPE publications and resources that support the guide series and specific guides, such as:

• The Knowledge Management Good Practice Guide, which is in development
• ISPE resources such as training programs on quality risk management
• PQLI® Guide, Part 3: Change Management System as a Key Element of a Pharmaceutical Quality System6
• PQLI® Guide, Part 4: Process Performance and Product Quality Monitoring System7

The APQ Program has been successfully developed, tested for practicalities and value, and refined and enhanced.

APQ CAPA Guide Stages

Stage 1: Preassessment Benchmarking (Optional)

At the outset of the CAPA process, it is useful to establish a baseline of current performance by formally documenting selected KPIs and organizational enablers. This preassessment benchmark may be completed using the companion tool provided by the St.Gallen OPEX team, designed specifically for use with the APQ program as a low-resource, simple-to-complete exercise. This step is optional but recommended. In total, the benchmarking involves providing values for 13 KPIs in four dimensions, answering 18 maturity questions, and providing information for some contextual factors (e.g., site type and size).

The St.Gallen APQ benchmarking tool can be used for self-evaluation conducted internally by a company, or in a benchmarking process evaluated by the St.Gallen OPEX team with results provided in a formal analysis report highlighting potential areas for improvement. (Note: This APQ preassessment performance analysis report can also be provided by participation in a full St.Gallen OPEX benchmarking study.)

The preassessment may also inform the prioritization of where a company should focus its resources to apply the deeper diagnostic APQ quality management maturity self-assessment in stage 2. Ultimately, the results can act as an important comparator or baseline against which future advancement results can be evaluated using the postaction benchmark.

The core of the ISPE APQ Program, the APQ AAAA framework is outlined in stages 2 and 3.

Stage 2: Assess and Aspire

In this stage, using the APQ self-assessment tools detailed in the guide, a cross-functional assessment team will perform a deep evaluation of the organization's quality management maturity. The APQ assessment process is a guided process providing objective quantitative and qualitative criteria to assess business process capabilities and performance, leadership and workforce competencies, and associated behaviors of a selected quality system element. The self-assessment process enables a proactive and honest review of current practices and outcomes to determine the current level of quality management maturity. When the APQ self-assessment is conducted, any underlying gaps or issues and opportunities for improvement are formally identified and documented.

A matrix has been created with five levels of maturity common to all elements/APQ guides and an appropriate number of subelements/areas relevant to each ICH Q10 element. For each subelement assessed using the APQ assess tool, the assessment team will observe, review, and provide demonstrated evidence of current policy and practice compared with specific criteria set out across the five-level maturity model. From this detailed review and as explained in the guide, a maturity level is assigned.

The next step is to complete the APQ aspire process, where an analysis of the results of the self-assessment process is undertaken to review the improvement opportunities identified and to determine where and by how much the company aspires to improve. This analysis will confirm the overall maturity score for the element under consideration and prioritize the specific improvement opportunities based on current performance or business needs. The output of the APQ aspire process forms the basis for the improvement action plan.

Stage 3: Act and Advance

The next step is the APQ act process. Its purpose is to develop an improvement action plan that is appropriately resourced and defines the necessary actions to enhance maturity to the next level. The APQ guide contains a catalog of improvement tools and KPIs and has been developed to provide further recommendations on available supporting resources and useful KPIs worthy of consideration by the team responsible for the improvement action plan.

The APQ advance step involves the careful design and evaluation of the effectiveness criteria necessary to demonstrate achievement of the improvement goals.

Stage 4: Postaction Benchmarking (Optional)

At an appropriate duration after completion of the APQ AAAA process, it is recommended that a postaction benchmarking exercise be conducted using the same St.Gallen APQ benchmarking tool used in stage 1. This postaction benchmarking is intended to evaluate the impact of the improvements on the overall company performance.

It is hoped that in time, as the APQ knowledge-sharing forum develops, case studies quantifying the benefits gained will become available to share with others. These cases will serve to provide incentives for broader adoption within the industry by demonstrating the value to the business of adopting such a formal program.

Influences on the Initial APQ Proposal

As summarized earlier, major influences on development of the APQ AAAA framework were the ISPE CAPA pilot; St.Gallen research (which included research funded by the FDA); and evolving FDA publications on assessing the state of pharmaceutical manufacturing quality.

ISPE CAPA Pilot

The CAPA maturity pilot was described in a Pharmaceutical Engineering special report (September-October 2018)8 and a presentation at the 2018 ISPE Annual Meeting & Expo in Philadelphia.9 The objectives of the pilot, which involved nine companies, were validating the concept and understanding its value to the industry. Feedback particularly focused on questions related to the following:

• Maturity level descriptors and how participants conducted assessment
• How and from where KPI data were obtained
• The value and appropriateness of KPIs
• Whether the catalog of tools is helpful

Key insights from the pilot (Table 2) were then incorporated into the final design of the CAPA framework and guide.

St.Gallen Research

Figure 4 presents the timeline and major milestones of the Quality Metrics Initiative. This graphic illustrates the origins of the St.Gallen research that informed the creation of the ISPE APQ framework, and which may also have influenced the evolution of the FDA’s thinking on evaluation of the state of quality in the pharmaceutical industry.

ISPE has conducted two pilot research studies into quality metrics to provide data-driven responses and comments about FDA quality metrics guidances. ISPE has also organized several workshops involving participants in the pilot studies as well as representatives of the FDA and the University of St.Gallen. These workshops were designed to provide input for ISPE positions and responses to quality metrics guidances and to broaden thinking as ISPE focus evolved from quality metrics to the more expansive and challenging task of assessing the state of quality in the pharmaceutical industry.

In response to a request in 2015 for academic research into the application of quality metrics in the pharmaceutical industry, the University of St.Gallen was awarded a research grant, which was subsequently extended. This research by St.Gallen has been published in three reports10^, 11^, 12 and influenced the FDA in their efforts to assess the state of quality in the pharmaceutical industry.

An outcome of this St.Gallen research and the original St.Gallen OPEX benchmarking program was a concise, user-friendly benchmarking module. In this module, all selected performance indicators and enablers are meaningful for understanding overall plant stability and performance. Developing this set of measures required extensive research, primarily using St.Gallen’s databases containing operational performance data from more than 380 manufacturing sites and around 100 quality control labs. These databases were built over the last 15 years and contain the outcomes of the full St.Gallen OPEX benchmarking programs. Based on the available data sets, statistical exploration—such as correlation and regression analyses or t-tests—led to the selection of a subset of 13 metrics surrogating overall performance. Additional validation comprising the direct comparison of the new abbreviated overall performance score calculated based on the chosen measures only, and comparison with the full performance score used in the legacy benchmarking, provides confidence from a system perspective.

The ISPE collaboration with St.Gallen provides a platform for the university to advance their OPEX benchmarking database work and research within the industry.

FDA Publications on Quality

In 2018 and 2019, the FDA drew from St.Gallen research as the agency started to discuss the concept of quality maturity in public presentations,13 leading to an extensive discussion on quality management maturity in the FDA report “Drug Shortages: Root Causes and Potential Solutions.”14 The report concludes that “economic forces are the root causes of drug shortages” and further identifies one of three major root causes to be that the “market does not recognize and reward manufacturers for mature quality management systems.”

The report also contains a substantial discussion of quality management maturity in Appendix B and has a section on the challenges in assessing quality management maturity. It states:

A quality management system is a collection of business processes focused on consistently meeting expectations, expressed as the organizational goals and aspirations, policies, processes, documented information and resources needed to implement and maintain quality. Quality management maturity is a measure of the consistency and reliability of business processes related to an organization’s goals.

ISPE asserts that the APQ AAAA framework should assist with the challenge of assessing quality management maturity.

• 4International Society for Pharmaceutical Engineering. The Cultural Excellence Report—Six Key Dimensions. North Bethesda, MD: International Society for Pharmaceutical Engineering, 2017.
• 5International Society for Pharmaceutical Engineering and Parenteral Drug Association. “ISPE–PDA Guide to Improving Quality Culture in Pharmaceutical Manufacturing Facilities. Module 1: Root Cause Analysis Guide.” 2019. https://ispe.org/sites/default/files/regulatory/ispe-pda-guide-to-improving-quality-culture.pdf
• 6International Society for Pharmaceutical Engineering. PQLI® Guide, Part 3: Change Management System as a Key Element of a Pharmaceutical Quality System. North Bethesda, MD: International Society for Pharmaceutical Engineering, 2012.
• 7International Society for Pharmaceutical Engineering. PQLI® Guide, Part 4: Process Performance and Product Quality Monitoring System. North Bethesda, MD: International Society for Pharmaceutical Engineering, 2013.
• 8Chelmedos, L. R., and K. L. Ford. “CAPA Maturity.” Pharmaceutical Engineering 38, no. 5 (2018): 32.
• 9Ford, K. “ISPE’s Approach to Advancing Pharmaceutical Quality—The CAPA Example.” Presented at the ISPE Annual Meeting, Philadelphia, November 2018.
• 10Friedli, T., P. Basu, and N. Calnan. FDA Quality Metrics Research Final Report, Version 2. St. Gallen, Switzerland: University of St.Gallen, 2017.
• 11Friedli, T., P. Basu, and N. Calnan. FDA Quality Metrics Research 2nd Year Report. St. Gallen, Switzerland: University of St.Gallen, 2018.
• 12Friedli, T., P. Basu, and N. Calnan. FDA Quality Metrics Research 3rd Year Report. St. Gallen, Switzerland: University of St.Gallen, 2019.
• 13Mullin, T. “Center Update: CDER.” Presented at the 24th Annual GMP by the Sea Conference, Cambridge, Maryland, 12 August 2019.
• 14US Food and Drug Administration. “Drug Shortages: Root Causes and Potential Solutions: A Report by the Drug Shortages Task Force.” 2019. https://www.fda.gov/drugs/drug-shortages/report-drug-shortages-root-causes-and-potential-solutions

In March 2020, the FDA funded a global quality benchmarking 2020 study (also referred to as Pharmastudy) as a collaboration between the University of St.Gallen and Dun & Bradstreet.15 The study will produce a globally representative baseline data set that characterizes quality management maturity among human drug manufacturers. It strives to analyze quality management maturity and operational data from approximately 2,000 manufacturing establishments in 52 countries. This benchmarking study has used the same ICH Q10 quality maturity benchmarking module as that used in the APQ Program.

In parallel, the FDA has continued its investigations into the potential for a quality metrics program. In June 2018, the FDA published two Federal Register notices (FRNs) announcing new voluntary efforts to gather stakeholder feedback on the use of quality metrics.

The first FRN described a quality metrics feedback program with efforts that include Type C formal meeting requests and pre-abbreviated new drug application (pre-ANDA) meeting requests, as well as a pilot study to gain feedback from establishments.16 The second FRN announced a 2018 CDER and CBER staff experiential learning site-visit program to provide learning opportunities for FDA staff involved in the agency’s quality metrics program and to give stakeholders an opportunity to explain the advantages and challenges associated with a robust quality metrics program.17 Some companies with members on the Advancing Pharmaceutical Quality Core Team have participated in these programs, and ISPE has facilitated interactions between company members relating to preparation for and feedback from participation in these feedback efforts.

Conclusion and Next Steps

The APQ Program has been successfully developed, tested for practicalities and value, and refined and enhanced based on feedback and collaboration with the University of St.Gallen. The first APQ guide, on the ICH Q10 CAPA element, is available, and other guides in the series are planned: The management responsibilities/review guide criteria, tools, and KPIs are developed and being tested. Criteria, tools, and KPIs for the change management guide have also been developed. Work to develop the matrices for the process performance and product quality monitoring guide has commenced.

ISPE considers the APQ Program as applied using the APQ guide series to be a major tool that the industry can use to assess and advance the state of quality management maturity.

Learn More & Purchase

• 15Dun and Bradstreet. “2020 Quality Benchmarking Study.” Accessed 12 October 2020. https://www.dnb.com/marketing/pharmaceutical-manufacturing-quality-study.html
• 16US Food and Drug Administration. “Modernizing Pharmaceutical Quality Systems; Studying Quality Metrics and Quality Culture; Quality Metrics Feedback Program.” Docket No. FDA-2018-N-1903. Federal Register 83, no. 126 (29 June 2018): 30748–30751. https://federalregister.gov/d/2018-14005
• 17US Food and Drug Administration. “Quality Metrics Site Visit Program for Center for Drug Evaluation and Research and Center for Biologics Evaluation and Research Staff; Information Available to Industry. Docket No. FDA-2018-N-1896. Federal Register 83, no. 126 (29 June 2018): 30751-30752. https://federalregister.gov/d/2018-14006

Acknowledgments

Thank you to the APQ CAPA subteam members: Lori Chelemedos, Pac-Side LLC; Andrew Denny, Bristol-Myers Squibb; Ketan Gujarathi, Bristol-Myers Squibb; Kira L. Ford, Eli Lilly and Company; Tami Frederick, Perrigo; and Blain Spangler, Baxter.