Why You Shouldn’t Test for Consent Comprehension

Mark Hochhauser, Ph.D.

Readability Consultant

Golden Valley, Minnesota

While federal regulations do not require prospective subjects to be tested for how well they comprehend the informed consent process, researchers and ethicists are concerned that prospective subjects may be signing consent forms they do not understand. To address this dilemma, a few tests have been developed as a formal way to measure consent understanding. But publications on consent comprehension tests have not addressed the practical aspects of using such tests in the subject recruiting process.

Are comprehension tests compliant with federal regulations?

Although comprehension test score results might be helpful and even more ethical, such testing is not required under federal regulations, and from compliance perspective such testing could be interpreted as being noncompliant! On the one hand, comprehension tests could be viewed as one way to ensure that consent forms are written “…in language understandable to the subject or the representative.” On the other, formal testing is not the only strategy for ensuring “understandable language,” since federal regulatory agencies (FDA, OHRP, NCI) typically recommend that consent forms be more understandable by writing them at a 6th-8th grade reading level.

What test would you use?

The DICCT (Deaconess Informed Consent Comprehension Test) is a 14 item generic comprehension test written at an eighth grade reading level and requiring 12 minutes to administer and score. This test (Miller, CK, et al. Pharmacotherapy, 1996, 19(5), 872-878) was based on subjects who were mostly female, 36 years old with about two years of college--may not be valid for subjects with vastly different demographics.

The QuIC (Quality of Informed Consent) test is a 34 item oncology comprehension test (written at about a 12th grade reading level according to my analysis) that takes seven minutes to administer. But the QuIC requires researchers write their own scoring algorithm for the statistical program they use to analyze the data. Because the QuIC (Joffe, S., et al. J. Nat. Cancer Inst., 2001, 93(2), 139-147) was based on primarily white (91%), older (age 55) and well educated (53% with a college degree) subjects, this test may not be valid for studies other than oncology and for subjects with different demographics.

Neither test has been standardized for use in the clinical trial setting; there are no test packages (i.e., a manual, copies of the test, scoring and interpretation instructions) that would ensure consistent testing procedures by all who use it. Researchers must read the original journal article and construct a testing procedure based on their own interpretations, an unsatisfactory strategy for using any comprehension test in the subject recruiting process.

What would you do with the test results?

Comprehension test developers and consent comprehension researchers have not quantitatively defined “comprehension,” so there is no “passing” or “failing” standard. But how much understanding is enough when you have to deal not only with comprehension but potential research liability issues? Since the lack of a “grading” system means that test users have to construct their own grading system, would most return their school experiences and arbitrarily set 60% (a “D”) or 70% (a “C”) as passing scores? Will it matter what 60% prospective subjects do understand and what 40% they do not? Is understanding some information (such as risks) more important than understanding other information (such as confidentiality)?

What will you do if prospective subjects score less than 60% or 70% on the comprehension test? Will you use their lower score as an exclusion criterion and tell them that they cannot be in the clinical trial because their comprehension score was too low, even though they met all the medical criteria? If so, subject recruitment will be slower and even more challenging, as some percentage of qualified subjects will be excluded because of low comprehension scores.

Will you go through the consent process again and test them again until they receive a passing score? How much extra time will that take; how many times can they take the comprehension test? Some researchers may be tempted to speed up the recruiting process by changing comprehension test scores from “failing” to “passing,” a strategy that could create serious legal problems when detected.

Comprehension testing puts you in a no-win situation. If subjects fail the test and you enroll them anyway, their low score could create liability problems if they’re injured in the trial. After all, with a failing score they showed they didn’t understand the clinical trial but you enrolled them anyway.

If subjects pass the test and are later injured, you might have to justify the validity and reliability of the test, as well as how you administered the test, scored it and interpreted the results. Injured subjects’ lawyers will argue that even though subjects “passed,” the test was not a scientifically sound way to measure comprehension, especially if the test you used didn’t have adequate psychometric standards for validity and reliability, guidelines for interpreting scores, etc. Such problems mean that test results might not be acceptable in court if the test is portrayed as being “junk science.” Under no circumstances should you even try to develop your own comprehension test.

How will sponsors react?

Comprehension testing will undoubtedly slow down the subject recruiting process. Even if testing adds only 15 minutes per subject and the average consent time is one hour, 15 minutes will increase the total time to consent subjects by 25%. Repeated testing might be necessary for some prospective subjects to get a passing score, thus could add another 15 minutes for testing plus whatever extra time is needed to go over the consent form again (another 30 minutes, perhaps)—and maybe a third time in an effort to have “no subject left behind.” Repeating the testing and the consent process could easily double the time needed to consent some subjects.

Or will researchers simply “teach” the test by overemphasizing those topics that are covered on the consent comprehension test and underemphasizing those that are not? Plus, there’s no way to know if failing scores are due to low comprehension or to factors such as test anxiety, desperation to be in the trial, low literacy skills (people can’t pass a test they can’t read), people for whom English is a second language, etc.

Sponsors will not be happy with adding even more time to recruit each subject. And that doesn’t begin to address how much more time will be needed if prospective subjects are excluded because of their low comprehension scores. The pool of available subjects will shrink from the larger number of patients who have specific health problems to the smaller number of patients who have specific health problems and can pass a comprehension test.

How will prospective subjects react?

Will prospective subjects react positively or negatively if they’re required to pass a test in order to be in a clinical trial? Will some prospective subjects withdraw because they’re embarrassed to have “failed” the test, or because it takes too much time and mental effort to get a passing grade, or because they haven’t taken a test in decades? Prospective subjects who were not good students in school, who may have been told that they were “dumb” or “stupid” might leave before taking the test because they don’t want to experience failure again; tests come with all kinds of psychological and emotional meanings for the people who have to take it.

How will IRBs react?

Since consent comprehension tests will become part of the consent process, testing procedures will require IRB approval. This means that the testing procedure must be explained in the consent form (making for even longer consent forms) and that researchers would have to explain test selection, administration, scoring, interpretation, etc. to the IRB. Depending on the IRB members, this could slow down the approval process as IRB members might ask for more documentation about the testing procedure since the IRB’s liability will be at stake based on some of the issues raised above.

Conclusions

In theory, comprehension tests offer a way to ensure that prospective subjects understand informed consent before they sign the consent form. Not only are there problems with how the tests have been developed, but the lack of standardized testing procedures adds too much variability to the testing process. When combined with the practical problems of administering, scoring, and interpreting tests in the subject recruiting process, the routine use of consent comprehension tests is not yet justified or practical.

Print