The Value of Centralized Monitoring

Joanne Malia
Associate Director, Clinical Documentation Management
 Regeneron Pharmaceuticals

Abstract: Regulatory agencies are advocating for sponsors to take risk-based approaches in various clinical trial-related activities, especially in the area of monitoring. Sponsors are looking at and beginning to use centralized monitoring. This article describes centralized monitoring, regulatory and industry expectations and initiatives related to centralized monitoring, the value of centralized monitoring in enhancing data quality in clinical trials, and use and documentation of centralized monitoring,

Disclaimer: The views and opinions expressed in this article are those of the author and should not be attributed to the Society of Clinical Research Associates or Regeneron Pharmaceuticals.

Regulatory Expectations

In 2012, the U.S. Food and Drug Administration (FDA) released an excellent guidance document entitled Guidance for Industry: Oversight of Clinical Investigations – A Risk-based Approach to Monitoring. This is a clear guidance document that lays out the FDA’s thoughts behind the risk-based approach and the agency’s expectations for monitoring plans. The guidance document defines risk-based monitoring and encourages sponsors to prioritize and focus monitoring on the critical data and processes.

Around the same time, the European Medicines Agency (EMA) released Reflection Paper on Risk Based Quality Management in Clinical Trials. The reflection paper lays out methodology for risk assessment, control, review, management, and communication throughout the course of the study. Along with identification of risks, the reflection paper covers mitigating, or if possible, eliminating risks, and evaluating risks throughout the study to ensure that the risks have not changed drastically or that new risks have not arisen, and communicating risks to appropriate stakeholders. Appropriate stakeholders can be upper management, other team members, or even clinical research sites and contract research organizations (CROs).

Clinical Trials Transformation Initiative and Transcelerate

The Clinical Trials Transformation Initiative (CTTI) is a public-private partnership to develop and drive adoption of practices that will increase the quality and efficiency of clinical trials. CTTI encouraged “quality by design” in the development of protocols. This includes quality control steps in the protocol, and ensuring that staff members are trained and understand what they need to do. CTTI advocated for quality data and adherence to the protocol. A number of regulators participated in the CTTI initiative.

TransCelerate is an industry initiative. This non-profit organization works across the biopharmaceutical research and development community to improve the health of people around the world. The large pharmaceutical and biotechnology companies that formed TransCelerate realized that the current monitoring process did not work well. Monitoring involved many resources, and evidence showed that it was not effective. TransCelerate made improving monitoring one of its initial initiatives and developed a framework for risk-based monitoring. Much of TransCelerate’s information, including training documentation and white papers, is publicly available on its website.

Integrated Quality and Risk Management Plan

TransCelerate’s framework for risk-based monitoring includes the integrated quality and risk management plan, which defines the critical data and processes for a study and recommends that teams focus on what matters most to the specific study (Table 1). The framework encourages that “quality by design” be implemented in these areas. The integrated quality and risk management plan would be the key plan to which all other plans would adhere and align.   

The integrated quality and risk management plan is the first document generated after the protocol. It defines the critical data and processes for the study. The monitoring plan, data management plan, statistical plan, and other plans would be in alignment with the integrated quality and risk management plan.

Critical data and critical processes are defined in the integrated quality and risk management plan. The critical data include important safety and efficacy data. The critical processes are the standard operating procedures or processes that support quality data. This includes the informed consent process, ensuring that monitoring is done appropriately, as well as safety reporting.

The integrated quality and risk management plan can also be developed at the program level for all of the studies for a particular therapeutic. The plan identifies clinical research risks and includes actions to proactively identify, assess, and manage risk. Since the plan also defines the critical data and processes, it should focus on supporting the primary efficacy endpoints and critical safety parameters. Instead of, for example, dealing with the twenty questionnaires that subjects complete, the plan focuses on the questionnaires that support the primary efficacy objectives or a safety evaluation. Less monitoring may be done on the more experimental endpoints and parameters.

From Integrated Quality and Risk Management Plan to Monitoring Plan

The monitoring plan defines all of the monitoring activities that might be included in the study to ensure that the study provides quality data and that the subjects are safe (Table 2). Using the framework of theintegrated quality and risk management plan, the monitoring plan expands the “traditional” monitoring plan to include on-site, off-site, and central monitoring. All monitoring, including pharmacy monitoring and medical monitoring, should be defined in the plan.

Risk indicators should be identified in the monitoring plan. If, for example, the threshold for serious adverse events in the study is two per subject and one clinical research site suddenly started having five serious adverse events per subject, this would require action to determine what is happening and what might be causing the difference at this site. The monitoring plan must include actions, and as necessary, an escalation path that includes the investigator.

It is also necessary to consider the difference between source data verification and source data review. TransCelerate advocates a source data review process, which is more holistic than source data verification. According to TransCelerate:

• “Source data verification is the process by which data within the case report form or other data collection systems are compared to the original source or information (and vice versa) to confirm that data were transcribed accurately.”
• “Source data review involves review of source documentation to check the quality of the source, review protocol compliance, ensure that the critical processes and source documentation are adequate to ascertain investigator involvement and appropriate delegation, and to assess compliance to other areas.” 

TransCelerate advocates including all of this information into the evaluation of what is happening across the study and using risk-based monitoring metrics. Metrics include developing threshold values for activities so that metrics or reports that indicate critical levels will prompt the necessary actions.

TransCelerate has proposed many metrics for risk-based monitoring. Typically, metrics focus on quality, timeliness (visits, entering information in the case report form, etc.), and efficiency of activities. Using those pre-defined thresholds can help sponsors determine whether action is necessary.

Centralized Monitoring

Centralized monitoring is a more statistical approach to handling this information and looking across sites or across patients within a clinical research site (Table 3). The FDA definition of centralized monitoring states that it:

“Consists of a remote evaluation carried out by sponsor personnel or representatives (e.g., clinical monitors, data management personnel, or statisticians) at a location other than the sites at which the clinical investigation is being conducted.”

Review of the submitted data for quality is part of centralized monitoring, which is performed in order to identify and address missing and inconsistent data, data outliers, and potential protocol deviations that might not be identified without this approach. Centralized monitoring also enables sponsors to conduct statistical analyses to uncover data trends such as range, consistency, completeness of data, or unusual distribution of data within and between clinical research sites.

For example, one clinical research site may be having difficulty because staff members are interpreting the protocol differently than the other sites. Sometimes these trends indicate that something is happening that requires additional monitoring at a clinical research site or requires a conversation with the site or the monitor (CRA).

Centralized monitoring can be used to analyze performance metrics, verify certain critical source data remotely, and complete some administrative and regulatory tasks. Clinical research site performance metrics such as high screen failures or withdrawals, a high number of subject eligibility violations, or study delays can be analyzed to uncover poor performance or noncompliance.  

Benefits of Centralized Monitoring

Before the term “centralized monitoring” was being used, one of the statisticians with whom the author worked wanted to analyze data between various clinical research sites. He looked at data on temperature and identified a site that had an unqualified and uncalibrated thermometer. He worked with the study team to have the site fix this problem. The statistician showed that centralized monitoring could be used to identify information that could potentially affect study results.

Centralized monitoring has many benefits (Table 4). It enables sponsors to look at the protocol and look at critical data to see what might affect efficacy or safety variables. This information can then be used to fix issues or lessen their affect. This methodology has been used in the past to identify data fabrication. Often, people who fabricate data are not creative, and the data similarities are fairly large and obvious.

On-site monitoring can be targeted to higher-risk clinical research sites. Centralized monitoring enables the sponsor to identify sites that are having more difficulty than other sites. Perhaps one site does not understand the protocol or has new staff members who require training.

Many regulators hope that centralized monitoring will increase communication with clinical research site staff. With the sponsor, CRO, and site staff looking at the data, issues can be addressed before they become large problems. Centralized monitoring will also help monitors prioritize on-site monitoring to the processes that are critical to the success of the study.  

Focusing on what matters is a key benefit of centralized monitoring. Monitoring should focus on the primary efficacy and safety variables, not on spending limited resources dealing with data that will not impact the study results.

When implementing centralized monitoring, sponsors often expect to save money because monitors are not traveling to the clinical research sites as many times. Centralized monitoring does not necessarily save money. This methodology requires software and, if centralized monitoring is done internally, programmers are required to program and analyze much of this information. It may be a trade-off, with less money spent on visiting sites and more money spent on programming and analysis. Some CROs already have centralized monitoring capability.

The FDA’s Guidance for Industry: Oversight of Clinical Investigations – A Risk-based Approach to Monitoring defines what really matters in monitoring, including verifying that informed consent has been properly obtained and that the right patients are enrolled in the study (adherence to protocol eligibility criteria). Risk-based monitoring should also focus on appropriate accountability, administration of the investigational product, and verification of procedures and assessments involving study endpoints, protocol-required safety assessments, reporting of serious adverse events, subject deaths, and withdrawals, on time and to the proper authorities. Any procedures that are essential to the integrity of the clinical trial, such as maintaining the blind specified events that must be adjudicated and allocation concealment, must be verified. Study-specific issues related to the type of study, therapeutic area, and type of medication or device under study must also be verified.

TransCelerate has a list of suggested metrics for risk-based monitoring. Metrics can also be modified for use on specific studies or by specific sponsors. Under protocol compliance, for example, metrics include:

• Number of important protocol violations
• Major protocol deviation rate
• Minor protocol deviation rate.

Statistical Analyses

Statistical analyses are a major component of centralized monitoring (Table 5). It is not necessary to have a statistical background to understand the statistical analyses in centralized monitoring; however, it is helpful to have a statistician assist with the programming of the analyses.

One type of statistical analysis based on centralized monitoring is a comparison of data within and between patients and between sites. Since these tools often look at the similarity of data, it is important to be careful about which data are being compared. In the author’s past, she  compared all of the case report form data (CFR) including patient diaries across patients and sites. Since much of the information on the patient diary was expected to be similar, this comparison resulted in a great deal of false positive issues that all needed investigation and resolution. It would have been better to have only analyzed the CRF data and not the diary data.

Software packages often identify trends, outliers, and missing data. At times, it is easy to determine whether the outlier requires investigation. The software is very helpful in detecting fraudulent behavior and data. Sponsors must have a process in place to deal with signals of problems identified through the statistical analyses, such as investigating the problem first, then bringing in quality assurance to discuss the problem. It is also helpful to document the issues and their resolution to prevent reworking the same issues throughout the study. 

The statistical analyses in centralized monitoring allow for identification of training or retraining needs as well as training needs regarding protocol amendments or clarification. The analysis may show that many clinical research sites did not think that the training was clear. A large clarification often requires an amendment.

Equipment issues can be identified through the statistical analyses in centralized monitoring. Many studies involve device diagnostic testing, where sponsors send equipment to the clinical research sites. The people at the sites who are using the equipment usually have some training in the use of the equipment. The statistical analyses can identify differences in the use of equipment between different sites and may indicate that the equipment needs some attention.

The statistical analyses in centralized monitoring can indicate a need to change the monitoring approach or realign staff at clinical research sites. A change in monitoring could be visiting some clinical research sites more often or visiting all sites less often. Some sites may need more resources. The sponsor and the CRO can work with sites that need more help to achieve a better way to develop quality data.

When using centralized monitoring, it is important to ensure that the right data are being evaluated. Do not evaluate all of the data, which would require a great deal of issue management and investigation of false-positives. Focus on the critical data and avoid data that should be similar.

Performance metrics and thresholds should be used. Start small, with 10 to 20 metrics. Add more metrics after staff members have experience using metrics. Over time, the statistical analyses will indicate the most meaningful metrics for the organization. Thresholds for taking action should be set, and the actions to be taken if these thresholds are reached should be pre-defined.

Issue management is important. Sponsors should not have to revisit issues identified in past statistical analyses. Issues should be resolved after they are identified.

Documentation of Centralized Monitoring

Documentation of centralized monitoring is required (Table 6). Centralized monitoring requires a risk management plan, such as TransCelerate’s integrated quality and risk management plan. When starting centralized monitoring, it is best to develop a simple risk management plan that will work for the company. 

Documentation also includes evidence of adherence to the risk management plan and issue management. Evidence of adherence to the plan includes metrics, centralized monitoring reports, and changes in thresholds. Issue management documentation includes interpretation of results, investigation of issues, and results of the investigation. Sponsors often develop reports to document centralized monitoring.

Conclusion

Centralized monitoring does add value to clinical trials. Sponsors that have not yet used centralized monitoring should investigate it. Centralized monitoring allows review of data in a way that cannot necessarily be done through on-site monitoring. It focuses on critical data and process and helps sponsors prioritize what should be monitored.

The metrics and statistical analytics used in centralized monitoring are very helpful. Statisticians should be included in discussions about the appropriate metrics and statistical analytics for the study. It is important to choose the right data for analysis and to not overanalyze other data. Centralized monitoring should focus on critical study data.

The risk management plan must be followed, and appropriate documentation must be maintained. Documenting issue management is especially important, and it will save time in the future.

---

TABLE 1

Integrated Quality and Risk Management Plan

• Define the critical data and processes (what matters)
  • Critical data:
    • Data that support primary and key secondary objectives
  • Critical processes:
    • Processes that support safety or quality
• Implement “quality by design”
• Develop an  integrated quality and risk management plan:
  • Drives all other plans, especially monitoring plans

TABLE 2

The Monitoring Plan

• Expand “traditional” plan to include:
  • On-site monitoring
  • Off-site monitoring
  • Central monitoring
• Define risk indicators
• Ensure that the escalation path is understood and includes the clinical investigator
• Consider differentiating between:
  • Source data verification and source data review

TABLE 3

Centralized Monitoring

• A remote evaluation of the study
• Activities:
  • Monitor data quality through review of submitted data to address:
    • Missing and inconsistent data
    • Data outliers
    • Potential protocol deviations
  • Conduct statistical analyses to uncover:
    • Data trends such as range, consistency, and completeness of data
    • Unusual distribution of data within and between study sites
  • Analyze site performance metrics to uncover:
    • Poor performance
    • Noncompliance
  • Verify critical source data remotely, if possible
  • Complete administrative and regulatory tasks

TABLE 4

Benefits of Centralized Monitoring

• Identifying areas in the protocol for potential clarification
• Identifying indications of data fabrication
• Targeting on-site monitoring to higher risk clinical sites
• More communication with site staff
• More timely verification of informed consent documentation
• Focusing on what matters

TABLE 5

Statistical Analyses

• Compares data among patients at clinical research sites and between sites
• Looks for similarity of data, trends, outliers, and missing data
• Helpful in detecting fraudulent behavior and data
• Allows for identification of:
  • Training or retraining needs
  • Protocol amendment/clarification needs
  • Equipment issues
  • Change in monitoring approach
  • Staffing re-alignment
• Considerations:
  • Typically statistically based:
  • Evaluate critical data
  • Avoid data that should be similar

• Use performance metrics and thresholds
• Manage issues

TABLE 6

Documentation in Centralized Monitoring

• Risk management plan
• Evidence of plan adherence:
  • Metrics
  • Centralized monitoring reports
  • Change in thresholds         
• Issue management:
  • Interpretation of results
  • Investigation of issues
  • Results of investigation
• Report