U.S. Food and Drug Administration • Center for Drug Evaluation
and Research |
U.S. Food and Drug Administration • Center for Drug Evaluation
and Research |
 |
Drug Information |
 |
|
|
Evaluation of Written Prescription Information Provided in Community Pharmacies, 2001
Final Report to the Bonnie L. Svarstad, Ph.D.a Jeanine K. Mount, Ph.D., R.Ph.b University of Wisconsin - Madison December 21, 2001 aWilliam S. Apple Professor of Pharmacy and Professor of Sociology, University of Wisconsin-Madison, School of Pharmacy, 777 Highland Ave, Madison, WI 53705; Tel: 608-265-2128; FAX: 608-262-5262. bAssociate Professor of Social and Administrative Pharmacy, University of Wisconsin-Madison, School of Pharmacy, Madison, WI 53705; Tel: 608-262-8678. Acknowledgments: We gratefully acknowledge the assistance of Dr. Nathan Kanous, the clinical consultant who helped develop and refine the expert evaluation forms for this project. We also thank Mr. Pil Ho Kim for assisting with expert and consumer evaluations, data processing, and statistical analysis. We appreciate the contributions of expert and consumer panelists, consumer facilitators, the University of Wisconsin Survey Laboratory, the National Association of Boards of Pharmacy, and Dr. Ellen Tabak and colleagues at the U.S. Food and Drug Administration.
This report describes the methods and findings of a national study to assess the quality of written prescription information provided voluntarily to persons receiving new prescription medicines in community pharmacies. The study was contracted by the U.S. Department of Health and Human Services and the Food and Drug Administration (FDA) and conducted in collaboration with the National Association of Boards of Pharmacy (NABP) and a national panel of experts in pharmacy practice, pharmacotherapy, drug information, and health communications (Appendix A). Below is a brief overview of the study and questions to be addressed in this report. A more detailed description of study methodology, results, and discussion of findings follows. The present evaluation had several advantages over the 1999 interim evaluation of patient information materials provided by community pharmacies in eight states. First, information materials were obtained from community pharmacies that were selected randomly from a national electronic list of licensed pharmacies in the United States. Data based on a national sample allow decision-makers to make generalizations having national implications. Second, patient information materials were collected by trained shoppers who were identified and supervised by a professional shopper research firm (Second-to-None, Inc.). The shoppers acted as patients using a standard protocol that involved four new prescriptions for atenolol, atorvastatin, glyburide, and nitroglycerin. Shoppers (referred to as "patient-observers") followed a standard scenario and script, accepted any information that was offered, and paid cash for the prescriptions. Using trained shoppers and a standard scenario to collect information minimizes biases that can occur when using other data collection techniques such as observation by state inspectors and pharmacy students in different states and/or other methodologies such as patient or pharmacist report of information practices. A third advantage of the study was that patient information materials were evaluated by both expert and consumer raters. Expert raters focused on the extent to which the information was sufficiently specific, comprehensive, scientifically accurate, unbiased, up-to-date, and consistent with principles known to facilitate legibility and comprehension of written materials. Consumer raters focused on the extent to which the information was legible, comprehensible, and useful from a consumer's perspective. Patient information materials also were evaluated using an objective method of assessing readability or the approximate level of education needed to read various materials. Finally, the expert evaluation forms incorporated criteria recommended in the 1996 Long-Range Action Plan for the Provision of Useful Prescription Medicine Information1 and included more explicit check lists for evaluating the adequacy of drug information provided to patients. These explicit check lists allow more precise evaluation and description of patient information and may provide more useful information to health professionals, information providers, and the public.
The specific research questions addressed in this report were the following: · What percentage of patient-observers were given any written patient information with the four study drugs (atenolol, atorvastatin, glyburide, and nitrogylcerin)? · What percentage of patient-observers were given written patient information that adhered to various criteria used by expert panelists? · What percentage of patient-observers were given written information that adhered to various criteria used by consumer panelists? B.1. Selection of Pharmacies A simple random sample of 384 community pharmacies was selected by the contractor (FDA) from a national electronic list of pharmacies certified by the National Council for Prescription Drug Programs (NCPDP). A sample size of 384 was needed to obtain an estimate that would fall within + 5% of the true population value 95% of the time. Pharmacies were eligible if they were identified as an independent, chain, or franchise pharmacy within a retail, grocery, or department store setting. Pharmacies were ineligible if they were identified as an hospital, clinic, long-term care, mail order, IV infusion, dispensing physician, Indian Health Service, Veterans Administration hospital, or other government/federal setting. Pharmacies also were ineligible if they were located in Alaska, Hawaii, Puerto Rico, the U.S. possessions, or a state that prohibited the filling of prescriptions for research purposes. A total of 57,157 pharmacies were eligible using these criteria. A computer then assigned a sequence number to each eligible pharmacy, matched a list of randomly generated sequence numbers to eligible pharmacies with the same sequence numbers, and generated a final list of 384 pharmacies (with replacements). The list included the pharmacy's name, address, and telephone number. The contractor provided the final list of 384 pharmacies with replacements to Second-to-None, Inc., a professional shopper research firm responsible for visiting the pharmacies. Before visiting a pharmacy, the assigned shopper (patient-observer) called the pharmacy to verify location and hours. A pharmacy was replaced if the original pharmacy had no telephone listing or had closed (n=27), had disconnected the telephone or failed to answer it (n=11), no longer filled prescriptions or only filled prescriptions for certain populations or members (n=10), or only carried a limited number of medications (n=3). Several pharmacies also were replaced due to shopper error, scheduling problems, or no available shopper within an 100 mile radius (n=7). A total of 384 pharmacies were visited in 44 states. According to information from NCPDP, 36% of the visited pharmacies were independent pharmacies, 60% were chain pharmacies, and 4% were franchise pharmacies. B.2. Protocol for Collecting Written Patient Information Each study pharmacy was visited once by a shopper acting as a patient (called "patient-observers") with four new prescriptions for atenolol, atorvastatin, glyburide, and nitroglycerin. The patient-observers were hired and supervised by the professional shopper research firm (Second-to-None, Inc), and prescriptions were written by medical consultants in the pharmacy's region. Of 384 pharmacy visits, 72% were made by female patient-observers and 66% were made by patient-observers who were 45 years or older (mean=50 years old). Patient-observers were instructed to use a standard scenario as the script when acting as patient-observers. The standard scenario was developed by University of Wisconsin researchers and their clinical consultant (Nathan Kanous, Pharm.D). The scenario defined the patient-observer's name, address, reason for being in the area, health and medication history and other information that might be needed while acting as a patient-observer. Instructions were written out and provided to the shopper research firm for distribution to patient-observers. The standard patient - observer scenario follows: You are to assume the following scenario. You are from (city and state identified by project manager). You are visiting or driving through (pharmacy site area). You received four prescriptions from your doctor but forgot to fill them before leaving. You have some heart disease and diabetes, but no other medical conditions. You take no other medications and have no drug allergies. You have not taken these medications before. You don't have insurance for medication and therefore pay with cash. The reason you now have four new prescriptions is that you were "short of breath" and had "chest pain" so had to go to the hospital for some tests. They told you that you had "some heart disease" and that your cholesterol and blood sugar were too high. The doctor prescribed medications for the heart (atenolol), chest pain (nitroglycerin), diabetes (glyburide), and cholesterol (Lipitor). The doctor was busy and did not give any other information. He/she said the directions would be on the prescription bottles. You have a follow-up appointment with the doctor in two weeks. Patient-observers were instructed to make their approach and presentation in each pharmacy as uniform as possible. They were asked to be polite, to act interested, to answer questions briefly and to accept any oral counseling or written information offered by pharmacy staff. In addition, patient-observers were asked to avoid volunteering any information unless asked, to not ask questions, to not initiate "small talk" and to politely leave the pharmacy if the legitimacy of the prescriptions was questioned. If patient-observers recognized or were recognized by any pharmacy personnel, they were instructed to exit the pharmacy prior to presenting the prescriptions. Patient-observers were encouraged to role-play or practice the script and learn how to answer commonly asked questions prior to data collection. See Appendix B for commonly asked questions and suggested patient-observer responses and other things to do or avoid in the pharmacies. Patient-observers were provided with study prescriptions and the name and address of their assigned pharmacy. They traveled to the pharmacy, presented the prescriptions to be filled, picked up and paid for the prescriptions with cash, answered any questions according to the standard scenario, accepted any information materials that were offered, and exited the pharmacy. All pharmacy visits were conducted during the year 2001. After receiving the materials, the shopper research firm removed pharmacy and pharmacist names from all materials and forwarded them to the University of Wisconsin for evaluation. B.3. Expert Evaluation Procedures B.3.1 Expert Panel. A national panel of 16 experts was nominated by NABP, investigators, and six professional pharmacy organizations (Academy of Managed Care Pharmacy, American College of Clinical Pharmacy, American Pharmaceutical Association, American Society of Health-System Pharmacists, National Association of Chain Drug Stores, National Community Pharmacists Association). The 16 expert panelists include experienced pharmacy practitioners and pharmacy faculty members specializing in pharmacotherapy, drug information, and health communications. The panelists' role included nominating drugs to be included in the study, reviewing and approving the expert evaluation forms, and rating all patient information sheets collected by patient-observers. Before beginning their work, panelists reviewed sections of the 1996 Action Plan for the Provision of Useful Prescription Medicine Information, including Chapter 3 ("Guidelines for Useful Prescription Medicine Information") and Appendix G ("Specific Language and Format Guidelines, with Samples").1 Panelists also reviewed FDA approved labeling information for each study drug and the first drafts of four Patient Information Evaluation Forms (PIEF) prepared by the clinical consultant and investigators. Panelists were asked to review the evaluation forms for consistency with criteria outlined in the 1996 Action Plan and to offer other suggestions for improving the technical accuracy, clarity, and reliability of the forms as well as the format and scoring procedures. Their comments were incorporated into revised drafts of the evaluation forms. This review and feedback process was repeated until all expert panelists approved the final Patient Information Evaluation Forms. B.3.2. Expert Evaluation Forms and Criteria. The final expert evaluation forms incorporated eight general criteria suggested in the 1996 Action Plan (Table 1). Criterion 1-6 indicated that patient information must be sufficiently specific and comprehensive; criterion 7 indicated that patient information must be scientifically accurate, unbiased, and up-to-date; and criterion 8 indicated that patient information must be readily comprehensible and legible to consumers. To insure good reliability among raters, the forms also listed 5-10 sub-criteria or operationalizations under each general criterion. The sub-criteria defined or illustrated what kinds of information had to be present to be considered acceptable. The total number of sub-criteria was 61 for atenolol, 62 for atorvastatin, 62 for glyburide, and 63 for nitroglycerin. Copies of the final evaluation forms listing each criterion and sub-criterion are provided in Appendices C1 to C4. B.3.3. Scoring Procedures. For each sub-criterion, raters indicated whether there was full adherence (2 points), partial adherence (1 point), or no adherence (0 points). A computer later calculated the percentage of points obtained for each general criterion and for all criteria combined. Six levels of adherence were defined for each criterion: · Level 0: no information · Level 1: information with 0-19% of relevant points · Level 2: information with 20-39% of relevant points · Level 3: information with 40-59% of relevant points · Level 4: information with 60-79% of relevant points · Level 5: information with 80-100% of relevant points B.3.4. Assessment of Readability. Expert panelists rated all sub-criteria except sub-criteria 8.7 to 8.10 under Criterion 8. Panelists suggested that project staff evaluate these particular sub-criteria as they involved relatively objective assessments of font size, line spacing, and level of reading difficulty. Level of reading difficulty was assessed using an objective formula called the Gunning Fog test2. The Gunning Fog test was selected because it is considered one of the simplest methods of estimating the grade level needed to read informational materials3. The method involves selecting a sample of 100 consecutive words (W), counting the number of sentences (S) in the sample, and counting the number of words having three or more syllables (T) that are not combination words, capitalized words, or verbs ending in "ed" or "es". The final step involves calculating the grade level needed to read the material by using the Gunning Fog formula: (W/S + T) x 0.4. In the present study, staff selected a 100-word sample starting with the paragraph that describes how to use or take the medication. Since long drug names may result in overestimation of reading skill required3, drug names were not included when counting the number of words having three or more syllables. Appendix D describes the procedures used to assess readability using a modified Gunning Fog formula. B.3.5. Inter-rater Reliability. After the evaluation forms were approved by all panelists, the 16 panelists were divided into four sub-groups based on their expertise and experience with the four study drugs. Each sub-group was given responsibility for evaluating the information sheets for their assigned drug. Before rating the information sheets, inter-rater reliability was tested using 48 patient information sheets. Researchers assigned and mailed a sub-set of information sheets to various panelist pairs in each sub-group. Panelists independently evaluated their assigned materials and mailed rating forms back to researchers. A computer was used to calculate the percentage of points obtained for each general criterion and the percentage of points obtained overall. Inter-rater reliability was assessed using Pearson correlation coefficients. Pearson r for overall adherence was 0.93 for atenolol raters, 0.90 for atorvastatin raters, 0.97 for glyburide raters, 0.87 for nitrogylcerin raters, and 0.90 for all raters combined. Pearson r was 0.42 for Criterion 7 and ranged from 0.62 to 0.96 for remaining criteria. Further analysis of Criterion 7 showed exact agreement among panelists in 73% of the cases; however, there was some confusion about whether this section should be completed for cases where no benefit-risk information was provided. This problem was clarified through further discussion with panelists. After establishing acceptable inter-rater reliability, researchers sorted all information sheets by drug and mailed one-fourth of the sheets to each panelist in the sub-group responsible for reviewing that drug. Expert panelists rated a total of 1,367 pharmacy-generated patient information leaflets and 31 "patient information booklets" published and included in nitroglycerin boxes by drug manufacturers. This means that each expert panelist rated an average of 87 informational items. After reviewing their assigned materials, panelists returned the rating forms to researchers who submitted all forms to the University of Wisconsin Survey Laboratory for data entry. B.4 Consumer Evaluation Procedures B.4.1. Consumer Evaluation Form and Scoring Procedures. Researchers developed and pre-tested a 1-page Consumer Evaluation Form (CEF) for measuring the perceived comprehensibility, legibility, and usefulness of patient information sheets from a consumer's perspective. (A copy is provided in Appendix E). The Consumer Evaluation Form incorporated criteria suggested by the 1996 Action1 and several items that were validated in a previous study by Krass, Svarstad, and Bultman4. The first section included nine items believed to facilitate legibility and comprehension by consumers. For each item, consumers were asked to: "Please circle one number that best describes how YOU would feel if you were taking this medicine for the first time and received this information sheet from the pharmacy". Each item was given with a semantic differential scale scored from "1" (poor) to "5" (good). The relevant items included: poor - good print size, poor - good print quality, poor - good spacing between lines, poorly organized - well organized, poor - good length, unclear - clear, unhelpful - helpful, incomplete - complete, and hard - easy to find important information. The second section of the Consumer Evaluation Form asked for an overall assessment of three variables: readability, comprehensibility, and usefulness: "Overall, what is your opinion about this information sheet. Please circle one number that best describes how you would feel if you received this information sheet". Each item was given with a semantic differential scale scored from "1" (poor) to "5" (good): hard - easy to read, hard - easy to understand, and not useful - useful. Item scores were summed for the first section (range=9-45), second section (range=3-15), and combined sections (range=12-60). Five levels of adherence were defined: · Level 1: information with 0-19% of relevant points · Level 2: information with 20-39% of relevant points · Level 3: information with 40-59% of relevant points · Level 4: information with 60-79% of relevant points · Level 5: information with 80-100% of relevant points B.4.2. Test - retest Reliability. Researchers analyzed the test-retest reliability of the Consumer Evaluation Form using 18 patient information sheets and nine consumer raters who were asked to independently evaluate the same information sheets at two group sessions held at least three days apart. Pearson r was 0.81 for the first section, 0.78 for the second section, and 0.82 for the combined sections. These results suggest that the Consumer Evaluation Form had good test - retest reliability. B.4.3. Recruitment of Consumer Panelists. After establishing reliability, researchers identified 14 consumer facilitators with the help of pharmacy faculty in different geographic areas. Each consumer facilitator was asked to recruit 8-15 consumer panelists willing to attend a 2-hour group session where each panelist would be asked to read and evaluate a sample of patient information sheets using the 1-page Consumer Evaluation Form. Facilitators were provided a packet of materials that included a sample announcement and sign-up sheet, guidelines for selecting consumer panelists, guidelines for leading the group sessions, and one packet of materials for each consumer panelist. Facilitators used a variety of techniques for recruiting consumers, including: contacting employees or members of various organizations and social groups, posting announcements and sign-up sheets in clinics, senior centers, or apartment buildings, and other "snowball" techniques. Facilitators were encouraged to recruit a diverse group of male and female consumer panelists who: were able to read and understand English; had no training as a doctor, nurse, or pharmacist; did not reside in a nursing home or other institution; and had diverse race or ethnic backgrounds. Since the study drugs generally are prescribed for older adults, recruitment guidelines suggested that a majority of consumer panelists should be 50 years or older. Panelists were limited to English speakers, because it was not feasible to gather information sheets printed in other languages. Facilitators recruited 154 consumer panelists from 11 states. Panelist age ranged from 20 to 89 years old (mean=61.4, sd=16.7), with three-fourths of all panelists being 52 years old or older. Two-thirds (68%) of the panelists were female, 89% were white, and 97% reported English as their native language. Approximately 8% had not completed high school, 38% had completed high school, and 54% had more than a high school education. Three-fourths of the panelists (77%) typically used one or more prescribed medications each day (range=0-16 drugs, mean=3.0, sd=3.3). If a consumer panelist currently used a study drug, he/she did not evaluate patient information sheets for that drug. B.4.4. Consumer Group Sessions. Each consumer panelist attended one small group session led by a consumer facilitator. The purpose of the small group session was to evaluate a sample of patient information sheets using Consumer Evaluation Forms. The facilitator began the session by providing a brief background of the study, gathering demographic information from each panelist, distributing one packet of materials to each panelist, and answering any questions. Each panelist was asked to read and evaluate independently 10-12 patient information sheets during the session. They were asked to rate one information sheet at a time, to place each information sheet and evaluation form into their envelope after finishing it, and to avoid talking with other panelists until everyone had completed their evaluations. Each consumer panelist was given an honorarium of $50 for his/her efforts. B. 5. Data Entry and Processing All expert and consumer evaluation forms were submitted to the University of Wisconsin Survey Laboratory for data entry and merging into an electronic data file. Researchers checked the data file for missing or inaccurate entries and created frequency tables using SPSS. C.1. Frequency of written information transmission Patient-observers presented new prescriptions in 384 community pharmacies located in 44 states. The overwhelming majority of these prescriptions were dispensed with computer-generated "patient information leaflets" beyond container labels. The percent of patient-observers who were given a patient information leaflet was 89.6% for atenolol, 88.8% for glyburide, 89.3% for atorvastatin, and 88.3% for nitrogylcerin (Table 2). A total of 1,367 patient information leaflets were rated by expert panelists. Of these, 62.1% were at least 5.6 inches long and 37.9% were 5.5 inches long or less. Nearly 53.6% of the leaflets were based on information from a single publisher, 45.6% were based on information from unknown publishers, and less than 1% were based on information from other publishers. In addition to these pharmacy-generated patient information leaflets, 31 patients received a pre-printed "patient information booklet" included in the package by the nitroglycerin manufacturer. These manufacturers' booklets will be evaluated and described later. C.2. Overall quality of patient information leaflets according to expert panelists Table 3 presents the distribution of expert panelists' ratings of patient information leaflets by drug, including cases where no written information was given. In the first row, we see the distribution of ratings for atenolol information. Only 20.1% of the patient-observers received a leaflet that met the Level 4 threshold and none received leaflets that met the Level 5 threshold. Approximately 56.5% of the patient-observers received a leaflet that met Level 3 threshold (40-59% of points). The average percentage of points for all atenolol prescriptions was 51.4. Expert panelists' ratings of leaflets for glyburide and atorvastatin were similar, with only 24.5% and 16.9% of patient-observers receiving a leaflet that met the Level 4 threshold for glyburide and atorvastatin, respectively. None received a glyburide or atorvastatin leaflet that met the Level 5 threshold. For both drugs, the average percentage of points received was approximately 51%. The ratings of nitroglycerin leaflets were somewhat higher, with 41.7% receiving information that met Level 4 threshold. Again, none of the patient-observers received a nitroglycerin leaflet that met Level 5 threshold. The average percentage of points received was 55.3%. C.3. Expert panelists' ratings of patient information leaflets by criterion Table 4 shows the distribution of expert panelists' ratings of patient information leaflets by criterion (excluding cases in which no leaflet was given). Level of adherence to criteria varied across the eight criteria, with the highest means obtained on Criterion 7 (scientific accuracy, unbiased, up-to-date). The mean percentage of points on Criterion 7 ranged from 90-99% for the four study drugs, as seen in the final column for Criterion 7. In contrast, the mean percentage of points ranged from 38-66% for Criterion 1 (drug name and indications for use), 33-51% for Criterion 2 (contraindications and what to do if applicable), 54-75% for Criterion 3 (specific directions about how to use, monitor, and get maximum benefits), 33-45% for Criterion 4 (specific precautions and how to avoid harm while using the medication), 40-53% for Criterion 5 (symptoms of serious or frequent adverse reactions and what to do), 36-38% for Criterion 6 (general information and encouragement to ask questions), and 44-56% for Criterion 8 (legibility and comprehensibility). The distribution of ratings showed that the expert ratings generally fell below the Level 4 threshold on five of eight general criteria, including Criterion 2 (contraindications), Criterion 4 (precautions), Criterion 5 (adverse reactions), Criterion 6 (general information), and Criterion 8 (legibility and comprehensibility). While there were some variation among study drugs for Criteria 1-5, there were few drug differences for Criteria 6-8. The most striking pattern was the low level of adherence overall. C.4. Expert panelists' ratings of patient information leaflets by sub-criterion Tables 5-8 show the percentage of pharmacy-generated patient information leaflets with full or partial adherence on each sub-criterion. The percentage of leaflets with "no adherence" is omitted from these tables to simplify data presentation. The tables indicate more clearly why patient information leaflets did not fully meet various criteria used by expert panelists. Criterion 1 requires inclusion of generic and brand names, phonetic spelling of generic names, and other information about the drug and its indications for use. Tables 5-6 show that most atenolol and glyburide leaflets included the generic name and a common indication for use. However, nearly one-half of the leaflets omitted the phonetic spelling of generic names and most did not mention any brand names. Leaflets for atorvastatin and nitroglycerin provided more information about brand names, but also lacked phonetic spellings of generic names (Tables 7-8). Most glyburide and atorvastatin leaflets failed to mention that these medications should be used in persons whose conditions cannot be controlled by other means such as proper diet. Criterion 2 requires specific information about contraindications and what to do if applicable. Results show that 75% or more of all leaflets fully adhered to sub-criteria regarding pregnancy and breast-feeding. However, a majority of leaflets failed to adhere fully to other sub-criteria in this area. For example, many atorvastatin leaflets did not encourage patients to tell their provider if they drink large amounts of alcohol or have a history of liver disease (Table 7: 2.4, 2.5). Criterion 3 requires specific directions about how to use, monitor, and get the most benefit from the medication. Over 80% of the nitroglycerin leaflets fully adhered to seven of the nine sub-criteria in this area (Table 8). Atenolol, glyburide, and atorvastatin leaflets fully adhered to only 2-3 sub-criteria related to missed doses and use with regard to food. Lower ratings were obtained for other sub-criteria because these leaflets did not always emphasize the importance of regular use, taking the medicine at the same time(s) each day, and regular testing of blood glucose or cholesterol levels (Tables 5-7). Criterion 4 requires specific precautions and information about how to avoid harm while using the medication. A majority of the patient information leaflets encouraged patients to tell their providers if they take any other medications; however, leaflets often lacked specific information in this area. For example, only one-fourth of leaflets mentioned the interaction between atentolol and calcium channel blockers such as verapamil and diltiazem and only one-fourth of atentolol leaflets mentioned the potential interaction with immunosuppressants such cyclosporine (Sandimmune). While 80% of glyburide leaflets mentioned sensitivity to sun, many glyburide leaflets did not mention potential interactions with MAO inhibitors, aspirin products, or anticoagulants. One in three nitroglycerin leaflets failed to mention any interaction with sildenafil or Viagra. Nearly all atenolol leaflets included a warning about stopping atenolol; however, only 36% included a warning that was sufficiently specific and clear according to panelists (Table 5: 4.5). Over 80% of the leaflets advised patients to tell their provider or dentist about their use of atenolol before surgery, but patients rarely were informed that atenolol may cause serious reaction to allergy shots or worsen allergic reactions to foods, medicines, or stings. Only one in four atorvastatin leaflets included any statement about the need for liver function tests (Table 7: 4.8). Criterion 5 requires information about the symptoms of serious or frequent adverse reactions and what to do about them. Panelists rated the atenolol leaflets favorably with regard to certain adverse reactions (Table 5: 5.1, 5.5); however, they noted insufficient information regarding other adverse reactions such as feeling depressed and trouble having sex or sleeping (Table 5: 5.7-5.9). The quality of information regarding adverse reactions also varied considerably for the glyburide, atorvastatin, and nitroglycerin leaflets, as shown in Tables 5-8. Criterion 6 requires several items of general information and encouragement to ask questions. Table 5 illustrates why pharmacy-generated patient information leaflets often obtained low ratings in this area. Only 48-57% of atenolol leaflets included a statement about keeping medicines away from children and not giving this medicine to others (6-1-6.2), a disclaimer statement that the leaflet does not include all uses or effects (6.3), publisher name and date of publication (6.4-6.5), and encouragement to ask questions (6.6). Atenolol leaflets rarely informed patients about the availability of longer leaflets written for professionals (6.7). Similar problems were found with glyburide, atorvastatin, and nitroglycerin leaflets. Criterion 7 requires that information be scientifically accurate, unbiased, and up-to-date. The overwhelming majority of pharmacy-generated leaflets adhered fully to all sub-criteria in this area. The exception was that some atenolol leaflets did not adhere fully to sub-criterion 7.2 (no unapproved uses are listed). Criterion 8 requires that information be readily comprehensible and legible. Results indicated several reasons for low ratings in this area. For example, only 19% of atenolol leaflets adhered fully to sub-criterion 8.1 (black box warning information printed in bold-face type or box). Most leaflets received favorable ratings due to minimal use of italics or ornate typefaces (8.2) and use of upper and lower case lettering (8.3). However, they did not adhere fully to other principles for enhancing legibility and comprehensibility. Headings generally were not placed on separate lines (8.4). Bullets were not used to enhance readability (8.5). Information was not always well organized and easy to find (8.6). Spacing between lines was inadequate (8.7). Print size often was smaller than 10-point type (8.8). Some leaflets also lacked good ink-paper contrast (8.9). Finally, atenolol, glyburide, and atorvastatin leaflets often exceeded the recommended 8th grade level of reading difficulty. According to the Gunning Fog test, mean grade levels were 9.6 for atenolol, 8.7 for glyburide, 9.4 for atorvastatin, and 7.6 for nitroglycerin leaflets. C.5. Consumer panelists' ratings of patient information leaflets Table 9 summarizes the ratings of patient information leaflets with all criteria rated by a consumer panelist. Of 318 atenolol leaflets rated by consumer panelists, 24.2% met the Level 5 threshold and 30.2% met the Level 4 threshold. Similar trends were noted for consumer ratings of other study drugs. The percentage of leaflets meeting Level 5 threshold was 20.4%, 27.9%, and 28.6% for glyburide, atorvastatin, and nitroglycerin leaflets, respectively. The percentage of glyburide, atorvastatin, and nitroglycerin leaflets meeting Level 4 threshold was an additional 27.5%, 31.8%, and 34.1%, respectively. The mean percentage of points obtained ranged from 57.0% (glyburide) to 64.6% (nitroglycerin). Table 10 summarizes consumer panelists' ratings of written information using 12 relevant sub-criteria and 5-point semantic differential scales (1=poor, 5=good). Several results are noteworthy. First, consumer panelists were most critical of the leaflets' legibility. One of every three leaflets received a low score of "1" or "2" on print size, print quality, line spacing, and ease of reading. Consumer raters were especially critical of print size, spacing, and ease of reading. For example, print size received a score of "1" (poor) for 23% of atenolol leaflets, 24% of glyburide leaflets, 20% of atorvastatin leaflets, and 21% of nitroglycerin leaflets and line spacing received a score of "1" for 24% of atenolol leaflets, 21% of glyburide leaflets, 19% of atorvastatin leaflets, and 17% of nitroglycerin leaflets. The mean rating scores ranged from 2.9 to 3.1 on print size, 2.8 to 3.1 on line spacing, 3.1 to 3.4 on print quality, and 2.9 to 3.2 on the ease of reading overall. Consumer raters were more positive about the leaflets' comprehensibility and usefulness; however, none of the mean rating scores exceeded 3.9 in these areas. For example, the mean rating scores ranged from 3.5 to 3.8 on clarity, 3.6 to 3.8 on completeness, 3.4 to 3.7 on organization, 3.3 to 3.6 on length, 3.4 to 3.6 on ease of finding important information, and 3.3 to 3.7 on the ease of understanding overall. Results show similar trends on helpfulness and usefulness: mean rating scores ranged from 3.6 to 3.9 on helpfulness and 3.5 to 3.8 on usefulness overall. Finally, it is important to note the similarity in consumer panelists' ratings from one study drug to another. While nitroglycerin ratings were slightly higher than those for other study drugs, all mean rating scores ranged between 2.8 and 3.9. On the other hand, consumer panelists appeared to be quite discriminating. Like expert panelists, the consumer panelists rated some leaflets more favorably than others and appeared to have little or no difficulty evaluating leaflets on these criteria. Further analysis is required to determine whether and how expert and consumer panelist ratings are correlated with each other and how these ratings should be combined when determining the overall usefulness of written prescription information. C.6. Expert and consumer ratings of patient information booklets published by nitroglycerin manufacturers In addition to rating pharmacy-generated patient information leaflets, 31 patient-observers received a pre-printed "patient information booklet" that was published and included in the nitroglycerin box by the manufacturer. Table 11 shows the distribution of expert panelists' ratings of manufacturers' booklets. Overall, manufacturer's booklets only reached the Level 3 threshold of adherence. Booklet quality varied by criterion, with relatively high ratings given on Criterion 3 (directions) and relatively low ratings given on Criteria 4, 5, and 6 (precautions, adverse reactions, and general information). Concerns also were raised about booklet accuracy, bias, and timeliness: 48% of the manufacturers' booklets received only 20-39% of the points on Criterion 7. Table 12 shows the percent of booklets with partial or full adherence on each sub-criterion. These data help explain why these booklets did not receive higher ratings on certain criteria. While excellent ratings were given on directions for use (Criterion 3), the majority of nitroglycerin booklets did not adhere fully to most sub-criteria under Criterion 4 (precautions). In fact, none of the booklets mentioned a potential interaction with sildenafil (Viagra). One possible reason is that many booklets were published 3-6 years before they were given to the patient-observers in spring 2001. Expert panelists also commented that the booklets were not as neutral as they could have been, largely because they appeared to minimize or down-play adverse reactions and precautions (as reflected in panelists' ratings on Criterion 7). Expert panelists also were gave low ratings on booklet legibility (Criterion 8). None of the booklets adhered partially or fully to sub-criteria 8.7 and 8.8 (adequate line spacing and font size). Consumer panelists' ratings were consistent with expert panelists in several respects. First, they gave relatively low ratings on print size, print quality, line spacing, and overall ease of reading. Higher ratings were given on booklet completeness, possibly because the directions for use were perceived to be clear, complete, well organized, and easy to understand. D. STUDY LIMITATIONS AND CONCLUSIONS It is important to note several study limitations when drawing conclusions from this evaluation. First, data were collected from a simple random sample of community pharmacies in 44 states. While the national sample is a major strength of this study, we cannot generalize to mail order pharmacies and other excluded settings. Second, pharmacies were visited by shoppers who followed a standard protocol and scenario involving four new prescriptions. We cannot generalize to patients who pick up refill prescriptions. Third, the study evaluated the quantity and quality of information published in English. Patient access to prescription information in Spanish and other languages was not addressed in this study. Fourth, we obtained good test-retest reliability using the Consumer Evaluation Form and obtained useful information from a diverse group of 154 consumer panelists. It is not known whether similar results would be obtained using a random sample of consumers and other methodologies for assessing the consumer's perspective. Fifth, this study was focused on the provision of written prescription information. Further analysis is needed to determine whether the written information was provided alone or in combination with oral information and how these different methods of presentation affect consumers' evaluation of and behavioral response to written prescription information. This is an important issue, because consumers may be more likely to read and use written prescription information if they also receive oral counseling and encouragement to read this information. Despite these limitations, this study yielded several important findings. First, it is clear that most community pharmacies now provide a computer-generated patient information leaflet with every new prescription. However, leaflet length varies considerably. Approximately 38% of the leaflets were relatively short or abbreviated (less than 5.6 inches in length) and 62% were standard length (5.6 inches long or longer). Why some pharmacies are more likely than others to provide relatively short or abbreviated leaflets is not known at this time. Second, the findings show that leaflet quality varies widely. Expert panelists evaluated 1,367 pharmacy-generated patient information leaflets and found relatively high adherence on Criterion 7 (accuracy, unbiased, up-to-date). In contrast, expert ratings generally fell below the Level 4 threshold on five of eight remaining criteria. Ratings were especially low on criteria dealing with the risks of drug treatment and general information (Criteria 2, 4, 6). Consumer panelists also noted variability in leaflet quality and were especially critical of print size, print quality, line spacing, and overall ease of reading. Third, panelists found a number of problems with the patient information booklets published by nitroglycerin manufacturers. These booklets received high ratings on directions for use. However, expert raters expressed concern about the lack of sufficient, accurate, and up-to-date information on adverse reactions and precautions. They noted that the majority of manufacturers' booklets were not fully neutral or unbiased in content and tone. In sharp contrast, most pharmacy-generated leaflets were considered neutral in content and tone. 1. Steering Committee for the Collaborative Development of a Long-Range Action Plan for the Provision of Useful Prescription Medicine Information, Action Plan for the Provision of Useful Prescription Medicine Information, Unpublished report submitted to The Honorable Donne E. Shalala, Secretary of the U. S. Department of Health and Human Services, December 1996. 2. Gunning R. The Technique of Clear Writing (Rev Ed). New York: McGraw-Hill, 1968. 3. Buck, ML. Proving patients with written medication information. The Annals of Pharmacotherapy. 1998; 32:962-969. 4. Krass I, Svarstad BL, Bultman D. Using alternative methodologies for evaluating patient medication leaflets. Patient Education and Counseling (in press).
FDA/Center for Drug Evaluation and Research |
