Abby Statler, MPH, MA, CCRP
Research Regulatory Quality Assurance Coordinator
Cleveland Clinic Cancer Institute
Abstract: In 2008, The Cleveland
Clinic Cancer Institute established a quality assurance (QA) program for investigator-initiated trials
(IITs). Over the past nine years, the program has become an integral part of
the Institute’s research department, supporting the growth of IITs while
improving the proficiency of regulatory operations. This article describes the
program’s objectives and discusses the operational strategies employed to
achieve these goals. Other clinical research sites are encouraged to consider
how components of the Cleveland Clinic Cancer Institute’s QA program may be
adapted to meet the needs of their organizations.
The research sponsored by Cleveland Clinic investigators supports the Foundation’s innovation initiatives, making it a top priority for the Institute’s leadership team. Thus, the development and implementation of the Cleveland Clinic Taussig Cancer Institute’s QA program was motivated by the organization’s commitment to effectively oversee the conduct of IITs. Launched in 2008, the Cancer Institute’s QA program was the first of its kind, specifically focusing on providing operational support for investigator-initiated trials.
Before the department was implemented, investigator-initiated trials had few operational resources, protocols required multiple revisions, Investigational New Drug Applications were submitted inconsistently to the U.S. Food and Drug Administration (FDA), and processes for effective essential regulatory documentation management had not been established. To date, the QA program has grown from one to five members, which has enabled the department to cohesively support the IIT profile while also improve the operations of industry-sponsored and cooperative group trials. In this article, I will outline the essential components of a QA program, discuss the challenges that our department faced during the program’s development, and highlight how the Institute’s commitment to quality has enhanced the Cancer Center’s conduct and initiation of clinical trials.
The Profile of Investigator-initiated Research
The first step in developing a QA program for investigator-initiated trials is to inventory and categorize all protocols sponsored by the institution. The profile of this research must be well understood before the QA program’s objectives can be established. This was the first challenge our group faced due to the fact that investigators at the Taussig Cancer Institute were obtaining their own funding and writing their own protocols. A central repository for these studies had not been created. Therefore, the QA group initially focused on determining the overall scope of the Institute’s IITs. To accomplish this, the program manager reviewed Institutional Review Board records, data mined the Institute’s internal database, and regularly collaborated with research coordinators, investigators, and program managers. This was an epic task – it took several months to complete the comprehensive inventory of the Institute’s IITs.
After the profile of investigator-initiated research has been established, the next step is to identify the high-risk protocols. Protecting the safety and welfare of the subjects enrolled in investigator-initiated trials is paramount; therefore, conducting a risk assessment to prioritize oversight is essential. To perform this risk assessment, our QA team developed an algorithm to identify those studies that pose the highest risk. The algorithm assigns different weights to the study characteristics that are essential to the risk evaluation: Investigational New Drug (IND) / Investigational Device Exemption (IDE) status, number of sites, protocol complexity, study phase, and study team experience. The overall risk score assigned to each study determines its prioritization.
To be expected, the trials that pose the most risk involve investigational new drugs and/or devices (Table 1). However, IND/IDE exempt trials also top our institute’s prioritization list. While these studies do not include an investigational product, they still must follow the Common Rule as well as the International Conference on Harmonisation Good Clinical Practice (ICH GCP) Guideline. Of the IND/IDE exempt protocols, the multi-site trials are our highest priority. These studies are difficult to manage, specifically because, as the sponsor, Cleveland Clinic is responsible for overseeing the other institutions. Currently, multi-site IND/IDE-exempt studies receive explicit support from the QA program. We are focusing our efforts on providing the study teams with the resources required to effectively manage external institutions.
The lowest risk studies at the Taussig Cancer Institute are non-interventional; these include correlative studies, chart reviews, and studies involving databases and repositories. Because many of these studies do not require informed consent, the risk to potential participants is minimized. However, this does not mean that the risk is eliminated. We have discovered that inexperienced researchers fail to renew their protocols with the IRB, and in some cases, they violate privacy laws.
QA Review Scope and Priorities
The next steps in developing a QA program are determining the scope of the QA reviews and prioritizing the studies to be reviewed. Reviews should be driven by the organization’s risk assessment, focusing the majority of resources towards the high-risk studies. As discussed above, at the Taussig Cancer Institute, the highest priority studies are the INDs/IDEs, followed by multi-center IND/IDE-exempt studies (Table 2). These studies are generally monitored every 6-12 weeks.
The scope of review depends upon the characteristics of the study. Expectations are documented in the monitoring plan, which outlines the expectation of the initiation, interim, and close out visits. These reviews typically cover three primary components: subject records, pharmacy operations / drug accountability, and essential regulatory documents. Quality assessments, however, will not have the scope defined until the study is elected for review. This allows the QA team to tailor these assessments, ensuring that the highest-risk areas for a specific study are evaluated for compliance.
Goals and Objectives
Once the IIT profile has been established and the protocols have been prioritized according to risk, the QA team should determine the program’s goals and objectives. At the Taussig Cancer Institute, the primary goal is to support investigator-initiated trials (Table 2). Investigators have innovative ideas and are experts in their fields; however, they may not fully understand the regulatory expectations. For this reason, developing operational support for the research teams is a top priority. Educating sponsor-investigators and study coordinators regarding sponsor-specific responsibilities has been a premier aspect of this operational support. Study coordinators, especially those managing multi-site investigator-initiated trials, have indicated that the QA department’s educational initiatives have helped them feel more confident in managing these complex studies. Furthermore, the work of the QA group has enabled study coordinators to better manage their trials, improving the overall efficiency of the department.
The program’s second goal is to mitigate risk. This is accomplished by actively monitoring studies and consistently reviewing deviations. When an issue is identified, the QA group works diligently to emphasize the importance of root cause analysis and corrective action plans – both of which have been embraced by the research teams. Sometimes the same issue occurs across protocols. When trends are identified, the QA group discusses them with the study team(s), offering support and guidance regarding the corrective / preventative action plan. This cohesive process has been instrumental in reducing the number of protocol deviations.
Another goal of the QA program is to facilitate innovation within the institute’s research program. Administrators at the Taussig Cancer Institute recognize the importance of IITs; therefore, providing investigators with the resources to effectively conduct their studies is a top priority. This draws innovative investigators to the Taussig Cancer Institute and supports the ever-growing portfolio of IITs. The operational support that the QA program offers has been pivotal to the growth of the program; providing the regulatory support required to manage these studies allows investigators to focus their efforts on scientific innovation.
After the program’s goals have been defined, the team should develop performance metrics. Quantifying the group’s progress is essential for the growth of the program; this ongoing evaluation allows the team to determine if the goals and objectives of a QA program are being met. Performance metrics have been an essential aspect of the Taussig Cancer Institute’s QA program: Cleveland Clinic is data-driven. Objectives must be quantifiable, meaningful, and effectively communicated by leadership.
In order to measure the effectiveness of the QA program, the group quantifies:
- Number of reviews
- Monitoring visit frequency compliance
- Number of observations.
Tracking the number of reviews is important; this data has been instrumental in driving the growth of our team. If the QA team had not tracked the number of reviews and the amount of time spent on them, it is unlikely that the group would have been able to grow at an appropriate pace.
Compliance with the monitoring plan is also an essential metric. This allows the group to determine its efficiency. If a visit is missed, the monitor must provide an explanation. This accountability supports the group’s transparency, a key feature of our QA program. Furthermore, by tracking compliance with monitoring frequency, the QA group is able to provide the administration with the data required to determine whether the group’s objectives are being met.
The quantification and categorization of queries is another useful metric. The QA group has a spreadsheet that lists the different regulatory documents, organized according to Section 8.0 (Essential Regulatory Documentation) of ICH GCP. Each QA Coordinator populates this spreadsheet with the data from the respective reviews. This process has helped the group identify areas for improvement, drive process improvement initiatives, and guide future educational offerings. For example, after identifying informed consent documentation issues, the QA group conducted a number of training sessions for the nurses and study coordinators. This lead to a decrease in these observations, indicating that the targeted training initiative was effective.
The QA Program Pilot and Evaluation
Over the past nine years, the Taussig Cancer Institute has piloted a variety of program approaches, testing, and evaluations of specific review processes. After about two years of working with the administration, investigators, and study coordinators, the group has developed practices that are both efficient and effective.
The QA team’s standardization of the process for developing a monitoring plan is just one example of the practices that emerged from the initial piloted efforts (Table 3). The process begins with reviewing the protocol and providing feedback to the primary investigator (PI). This initial collaboration has been key to the program’s success – it allows QA to develop a relationship with the PI early on, which is crucial to a successful monitoring program. QA is able to provide feedback to the PI from a compliance perspective, ultimately improving quality of the protocol.
The next step involves the review of the database. QA compares the case report forms (CRFs) to the protocol to ensure that the appropriate data is being collected and that the structure of the calendar reflects the protocol. Additionally, the CRF instructions are reviewed; it is essential for both the monitor and the study coordinator to have the same expectations regarding data entry. Without effective instructions, the consistency / accuracy of the data collection may be compromised.
Next, QA works with the study coordinator to develop the manual of operating procedures (MOP). The research coordinator creates the initial version and sends the document to QA for review. During the review of the MOP, QA specifically ensures that subject registration, drug accountability, and sponsor communication are explicitly documented. These components of the MOP are particularly important for multi-site studies.
Finally, after the protocol, database, and MOP have been reviewed by QA, the development of the monitoring plan can begin. This involves editing the Taussig Cancer Institute’s monitoring plan template to meet the needs of the protocol – the standardized monitoring components remain consistent across studies while the source data verification section will be unique to each protocol. The draft plan is then sent to the study team for feedback; at this point, QA ensures that they have connected with the appropriate team members. This communication is essential, without it, the quality of the monitoring plan could be compromised. The last step in the process involves the final approval by the sponsor-investigator. He or she will sign the finalized plan, signaling the completion of the process.
Another example of the QA program’s evolution has to do with the program’s expectations regarding query responses. During the first few years of the program, the study coordinators were required to provide written responses to all regulatory queries. A few years after this requirement became effective, the coordinators began to argue that they were spending too much time responding to queries; this laborious requirement was not productive, taking time away from their ability to efficiently address the violations. In response, QA agreed to no longer require responses to the regulatory queries. This demonstrated QA’s willingness to work with the research teams, ultimately strengthening the department’s relationship with the institute’s disease groups. After the response requirement was eliminated, the QA team recognized that the responses were not needed to effectively address violations. In fact, after this requirement had been eliminated, study coordinators had more time to focus on addressing the queries themselves (i.e., filing documents, creating notes to file, preparing IRB submissions, etc.). This improved the operational efficiency of the department, which was a win for the QA program and the study teams.
Ultimately, a high-functioning IIT QA program should have the following components:
- a risk assessment algorithm
- a review prioritization strategy
- goals and objectives
- performance metrics
- a well-defined process for piloting quality assurance practices.
These are the cornerstones of a robust QA program, providing a solid foundation from which the department can grow. Furthermore, the standardized QA practices must be adaptable; as the research landscape changes, the QA program will need to evolve, modernizing the way in which compliance is assessed.
Building a successful QA Program takes time, trust, and stakeholder support. The cultivation of a high-functioning department is a dynamic process; it requires cohesive and coordinated efforts from the front line employees to the Chair of the Institute. The success of this collaboration truly depends upon the strength of the stakeholders’ relationships. These relationships, above all else, should not be underestimated.
The development and implementation of the Cleveland Clinic Cancer Institute’s QA program has tested relationships and presented unforeseen challenges. Blazing the trail for this program was not easy; however, each challenge presented an opportunity for growth, allowing the program to mature into a valuable resource for the institute. The QA department is now an integral part of the institute’s research operation, providing essential support for the growing profile of investigator-initiated clinical trials.
Risk Assessment for Investigator-initiated Trials at the Taussig Cancer Institute
- IND/IDE studies
- IND/IDE exempt studies (specific focus on multi-site trials)
- Non-interventional research
The Taussig Cancer Institute QA Program’s Objectives
- Support investigators conducting investigator-initiated trials
- Mitigate risk
- Reduce deviations
- Improve trial management
- Facilitate innovation
- Expand the investigator-initiated trials program
Process for Developing a Monitoring Plan at the Taussig Cancer Institute
- Review the protocol
- Review the database
- Review the manual of operating procedures
- Develop the monitoring plan
- Connect with the study team
- Solicit feedback and fine-tune the monitoring plan