Readability Recommendations: Easier Said Than Done
Mark Hochhauser, Ph.D.
Golden Valley, Minnesota
Because consent forms are long, complicated and hard to understand, federal agencies (FDA, OHRP, NCI and NIH) typically recommend that consent forms be written at a 6th-8th grade reading level. For example, the National Cancer Institute’s “Simplification of Informed Consent Documents” (http://www.cancer.gov/ clinicaltrials/understanding/ simplification-of-informed-consentdocs) readability recommendation is:
“Informed consent documents should be understandable to the patient population at the local facility. Documents should be written at an eighth grade or lower reading level. Investigators are also encouraged to use computer software applications or other techniques that assess reading level. Technical and legal jargon should be avoided.”
From NIH’s Office of Human Subjects Research (http:// ohsr.od.nih.gov/info/sheet6.html): “SUGGESTIONS FOR WRITING INFORMED CONSENT DOCUMENTS
When an investigator writes or reviews a research consent document, she/he should ask the following questions:
Question 1: Is it written at a reading level understandable to research subjects?
A general rule of thumb is that consent documents should be written so that they are understandable to people who have not graduated from high school. The reading level of a document is more difficult if it contains long sentences, words with more than two syllables, and continuous run-on text. Therefore, if possible use words with fewer than three syllables; use non-scientific/non-medical words; use short sentences, and break the text up into short sections.”
Readability of federal clinical trial materials
But consent forms aren’t the only information available to prospective subjects; agencies provide online clinical trial information that can help them understand both clinical trials as well as the consent process. Consent form readability research finds that they’re never written at the recommended 6th-8th grade reading level. So how well written are the clinical trial documents on federal agency websites? Can federal agencies follow their own recommendations and write clinical trial information at a 6th-8th grade reading level? They cannot.
Table #1 summarizes the readability of online clinical trial documents from three federal websites, including the FDA, National Cancer Institute (NCI) and clinical trials.gov. I used Prose: Readability Analyst software which reports grade level based on the average of eight readability formulas, including the Flesch Reading Ease formula. The Flesch scores from 0 (very difficult) to 100 (very easy), with 60-80 corresponding to a 6th-8th grade reading level.
Of these 30 documents, only one was written at a 6th-8th grade reading level—NCI’s “Participating in a Trial: Questions to Ask Your Doctor.” The most difficult to read documents were clinical trial descriptions at www.clinicaltrials.gov; four trials had very low Flesch Reading Ease Scores (11, 9, 4 and 1). Although most readability formulas do not report scores above grade 17, WStyle Writing-Style Analyzer software extrapolates grade levels above 17 using the FOG formula. These four clinical trial descriptions averaged grade 26--a grade level so high that it’s academically meaningless--except to point out how impossible it will be for readers to understand those clinical trials.
Oncology consent forms coming to our IRB often include a recommendation: “For more information on clinical trials and insurance coverage, you can visit the National Cancer Institute’s web site at http://cancer.gov/clinicaltrials/ understanding/insurance-coverage. You can print a copy of the “Clinical Trials and Insurance Coverage” information from this Web site.” Unfortunately, NCI’s insurance coverage information is written at a grade 14 reading level.
NCI’s insurance coverage information tells readers to:
“Understand your health plan. Be sure you know what’s in your policy; request and carefully review the actual contract language. If there’s a specific exclusion for experimental treatment, look closely at the policy to see how the plan defines such treatment and under what conditions it might be covered. If it is not clearly defined, call the plan’s customer service line, consult their Web site, and/or write to them. Ask for specific information about clinical trials coverage.”
Readability of health plan materials
While such common sense recommendations make perfect sense, health plan documents (Summary Plan Descriptions, or SPDs) are not very easy to understand either. My 2004 readability analysis of three Minnesota Health Plan SPDs scored them at grade 16, equal to 31 (“Difficult”) on the Flesch Reading Ease Scale.
Of course it’s reasonable for patients to call their health plan’s customer service line--except that customer service representatives may not be able to answer questions about health plan coverage for clinical trial injuries. In Minnesota, if you call or write your health plan’s customer service department, their response will include wording directly from your SPD--which if you understood when you first read it you wouldn’t be calling your health plan. That’s because in Minnesota only licensed insurance brokers can interpret a health plan; amazingly, customer service representatives are not legally permitted to interpret your health coverage other than restating information directly from your SPD!
Unreadability of clinical trial materials
Individually these may not seem like serious problems to overcome, but each one is another roadblock in the way of patients trying to get important information about clinical trials and their insurance coverage. Some prospective subjects may just give up when they find that they can’t easily get straight answers about what clinical trial injury expenses their health plan will or will not pay for, and what consents forms mean with statements such as “You will get medical treatment if your are injured as a result of taking part in this study. You and/or your health plan will be charged for this treatment.” Given possible copayments and deductibles for health coverage, the ambiguous financial difference between “and/ or” can be substantial.
Readability recommendations for consent forms to be written at a 6th-8th grade reading level are commendable, but have not yet been achieved. Even so, the informed consent form isn’t the only document prospective subjects may read. Motivated patients who seek out information about clinical trials from government websites and their own health plans will find such information difficult or impossible to understand. NCI’s 1998 “Simplification of Informed Consent Documents” identified “The Problem. Many informed consent documents have become too long and complex, and do not provide for informed decision-making,” a situation than hasn’t improved in the past 9 years.