|
CFSAN/Office of Nutritional Products, Labeling and Dietary Supplements
November 2004
This guidance document is being distributed for comment purposes only.
Comments and suggestions regarding this draft document should be submitted within 60 days of publication in the Federal Register of the notice announcing the availability of the draft guidance. Submit comments to the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852. All comments should be identified with the docket number listed in the notice of availability that appeared in the Federal Register.
For questions regarding this draft document, contact:
Vickey Lutwak
Office of Nutritional Products, Labeling, and Dietary Supplements
Center for Food Safety and Applied Nutrition
Food and Drug Administration
301-436-2375
Additional copies are available from:
Office of Nutritional Products, Labeling, and Dietary Supplements
Center for Food Safety and Applied Nutrition
Food and Drug Administration, 5100 Paint Branch Parkway
College Park, MD 20740
http://www.cfsan.fda.gov/guidance.html
U.S. Department of Health and Human Services
Food and Drug Administration
Center for Food Safety and Applied Nutrition
November 2004
| This draft guidance, when finalized, will represent the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance. |
Section 403(r)(6) of the Federal Food, Drug, and Cosmetic Act (the Act) (21 U.S.C. 343(r)(6)) requires that a manufacturer of a dietary supplement making a nutritional deficiency, structure/function, or general well-being claim(2) have substantiation that the claim is truthful and not misleading.
This guidance document is intended to describe the amount, type, and quality of evidence FDA recommends a manufacturer have to substantiate a claim under section 403(r) (6) of the Act. This guidance document is limited to issues pertaining to substantiation under section 403(r)(6) of the Act; it does not extend to substantiation issues that may exist in other sections of the Act. (3)
FDA's guidance documents, including this guidance, do not establish legally enforceable responsibilities. Instead, guidances describe our current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word "should" in our guidances means that something is suggested or recommended, but not required.
The Act, as amended by the Dietary Supplement Health and Education Act of 1994 (DSHEA) and the legislative history accompanying DSHEA do not define "substantiation." For this guidance, we drew upon our own expertise with respect to the regulations and case law regarding substantiation of various statements that may be made in the labeling of dietary supplements, conventional foods, and drug products (recognizing that conventional foods and drugs are regulated differently from dietary supplements), the Federal Trade Commission's (FTC) experience with its policy on substantiating claims made for dietary supplements in advertising, and recommendations from the Commission on Dietary Supplement Labels.
The Commission on Dietary Supplement Labels (the Commission), a seven-member body that was established under DSHEA to "provide recommendations for...the regulation of label claims and statements for dietary supplements, including the use of literature in connection with the sale of dietary supplements and procedures for the evaluation of such claims," held public meetings around the United States from 1996 through 1997. During these meetings, several manufacturers asked the Commission to provide guidance regarding the type of information that manufacturers should have in hand to substantiate a statement of nutritional support.(4)
Under the Act, FDA has exclusive jurisdiction over the safety, and primary jurisdiction over the labeling, of dietary supplements. The FTC has primary jurisdiction over advertisements for dietary supplements. Given these jurisdictional assignments, we and the FTC share an interest in providing guidance on what "substantiation" means. In April 2001, FTC issued a guidance document entitled, "Dietary Supplements: An Advertising Guide for Industry."(5) Our guidance document is modeled on, and complements, the FTC guidance document.
Dietary supplement manufacturers should be familiar with the requirements under both DSHEA and the Federal Trade Commission Act that they have substantiation that labeling and advertising claims are truthful and not misleading. Our approach provides manufacturers flexibility in the precise amount and type of evidence that constitutes adequate substantiation. Providing a standard for substantiation may also help to preserve consumer confidence in these products. To ensure compliance with the Act, we recommend that dietary supplement manufacturers carefully draft their labeling claims and carefully review the support for each claim to make sure that the support relates to the specific product and claim, is scientifically sound, and is adequate in the context of the surrounding body of evidence.
The FTC has typically applied a substantiation standard of "competent and reliable scientific evidence" to claims about the benefits and safety of dietary supplements and other health-related products. FDA intends to apply a standard for the substantiation of dietary supplement claims that is consistent with the FTC approach. This guidance document, using examples of claims that might be made for a dietary supplement, describes criteria to be considered in evaluating the nature of the claim and the amount, type, and quality of evidence in support of the claim.
The FTC standard of competent and reliable scientific evidence has been defined in FTC case law as "tests, analyses, research, studies, or other evidence based on the expertise of professionals in the relevant area, that has been conducted and evaluated in an objective manner by persons qualified to do so, using procedures generally accepted in the profession to yield accurate and reliable results."(6)
Although there is no pre-established formula as to how many or what type of studies are needed to substantiate a claim, we, like the FTC, will consider what the accepted norms are in the relevant research fields and consult experts from various disciplines. If there is an existing standard for substantiation developed by a government agency or other authoritative body, we may accord some deference to that standard.
In determining whether the substantiation standard has been met with competent and reliable scientific evidence, we recommend that firms consider the following issues in their assessment:
Each of these issues is discussed further in this draft guidance.
The first step in determining what information is needed to substantiate a claim for a dietary supplement is to understand the meaning of the claim and to clearly identify each implied and express claim. When a claim may have more than one reasonable interpretation, we recommend that a firm have substantiation for each interpretation. Consumer testing may be useful to determine consumer understanding of each claim, in context. We recommend that firms not only focus on individual statements or phrases, but also on what expected effect or benefit is being promoted when all of the statements being made for the product are considered together. Although it is important that individual statements be substantiated, it is equally important to substantiate the overall "message" contained when the claims are considered together.
Example 1: The label of a dietary supplement containing "X" uses the following claims: "The amino acid 'X' is the chemical precursor to nitric oxide. Blood vessel cells contain enzymes that produce nitric oxide. Nitric oxide is important in maintaining blood vessel tone." Assuming this statement were supported by sound science so that each individual statement was substantiated, the "message" conveyed by the claims, when considered together, is that taking oral "X" will affect nitric oxide production and blood vessel tone. Therefore, we recommend in this case that the dietary supplement manufacturer have substantiation that taking the amount of "X" provided by the product affect nitric oxide production and blood vessel tone under the product's recommended conditions of use.
The firm's clear understanding of the meaning of the claim is useful in ensuring that the evidentiary basis for substantiation is appropriate for the claim. Understanding the claim's meaning will help identify the appropriate study hypotheses and measureable endpoints, which can be used to ensure that the firm has appropriate studies to substantiate the claim. For example, a firm making a claim that a dietary supplement "helps maintain blood vessel tone" or "supports healthy immune system" should have a clear understanding of the claim's meaning to develop endpoints that could be measured and replicated in studies used as a basis for substantiation.
Example 2: The labeling of a dietary supplement includes the statement "promotes weight loss." The dietary supplement contains various vitamins and minerals and a botanical extract. The manufacturer relies on a randomized controlled double blind clinical study showing that subjects who took the botanical extract had a small but significant increase in metabolism over subjects taking a placebo over a 24 hour period. The study did not examine the effect of the extract on subjects' weight and there is no research showing that a short term increase in metabolism will translate into any measurable weight loss. The weight loss claim would likely not be adequately substantiated.
Example 3: The labeling for a dietary supplement contains a statement saying, "Recommended by Scientists," in connection with the product's claim. The statement gives consumers the impression that there is a body of scientists, qualified experts, who believe that the claim being made is supported by evidence. Consumers might also reasonably interpret the statement as meaning that there is general scientific agreement or consensus regarding the claim. If the manufacturer does not possess evidence to demonstrate such a consensus, the claim may not be substantiated. The opinion of a single scientist or small group of scientists is probably not adequate substantiation for such a claim.
Example 4: The labeling states, in connection with the product's claim, that the dietary supplement has been "studied for years" in a particular country or region and is the subject of clinical or "university" research. Here, the labeling conveys the impression that the product has been studied and also conveys the impression that there is a substantial body of competently conducted scientific research supporting the claim. We recommend that manufacturers possess evidence to substantiate both the express statements and their implied meaning.
Whether studies or evidence have a relationship to the specific claim being made or to the dietary supplement product itself is an important consideration in determining if a claim is substantiated. The following are some threshold questions in determining this relationship:
Have the studies specified and measured the dietary supplement that is subject of the claim? We recommend that the studies being used as substantiation for dietary supplement claims identify a specific dietary supplement or ingredient and serving size and that the conditions of use in the studies be similar to the labeling conditions of the dietary supplement product. Factors that would tend to indicate a stronger relationship between a substance that is the subject of a study and the substance that is the subject of the dietary supplement claim include similarities in formulation, serving size, route of administration, total length of exposure, and frequency of exposure. Manufacturers should be aware that other substances involved in the study or in the included in the dietary supplement product itself might also affect the dietary supplement's performance or the study results.
Example 5: To illustrate this issue, assume that a firm has high quality studies that are also consistent with the totality of the scientific evidence. The firm would like to use these studies to substantiate a claim that its dietary supplement has a particular effect on the human body, but the studies involved the impact of a specific ingredient in foods on the human body, and did not involve the dietary supplement product itself. In this instance, although the studies might be of high quality, it is not clear whether the results are applicable to the specific dietary supplement product.
Have the studies appropriately specified and measured the nutritional deficiency, structure/function, or general well-being that is the subject of the claim? We recommend that the studies clearly identify the endpoints that are to be used to substantiate the claimed effect.
Were the studies based on a population that is similar to that which will be consuming the dietary supplement product? For example, if the study involved young adults, but the product's claims involve conditions seen only in the elderly, the study might not be applicable to the claims.
Does the claim accurately convey to consumers the extent, nature, or permanence of the effect achieved in the relevant studies and the level of scientific certainty for that effect?
A note on foreign research: Foreign research could be sufficient to substantiate a claim as long as the design and implementation of the foreign research are scientifically sound and the foreign research pertains to the dietary supplement at issue. In evaluating data from studies conducted in a foreign population, care should be taken in extending the results to what might be expected in consumers in the United States who will use the product. Differences between the two populations, such as differences in diets, general health, or patterns of use, could confound the results. Also, it is important to make sure that the study examined the same dietary ingredient about which the claim is being made since there may be instances where, due to provincial or regional differences in custom, language, or dialect, the same name is given to different substances or different names to the same substance.
Example 6: A firm claims that its dietary supplement contains an ingredient shown to promote claim Y. The firm conducts a literature search and finds several references for carefully conducted, well-controlled studies demonstrating that the substance appears to be helpful in persons with claim Y associated with aging when the substance is applied topically to the affected area. However, there is no information provided concerning the effect of the substance when taken orally. Although the evidence may demonstrate that the product is effective when used topically, this information would generally not be useful to substantiate a claim for a dietary supplement (by definition, a product that is intended for ingestion (section 201(ff)(2)(A) of the Act (21 U.S.C. 321(ff)(1)(A))).
Example 7: A dietary supplement firm wants to promote an amino acid product to improve blood circulation and improve sexual performance. The firm conducts a literature search and finds many abstracts and articles about the amino acid's effect on biological mediators of circulation and a few animal and human studies designed to study the effect of the amino acid on blood flow. The firm intends to use this list of studies as substantiation for its claim.
Although the firm appears to have a significant amount of information for its claim, the list is likely not adequate because the firm has not demonstrated that the information is directly related to the claim being made. For example, in this situation we would recommend that the firm provide information to clarify the meaning of "improves blood circulation" and "improves sexual performance." We would also recommend that the firm determine whether the studies examined a dosage of product similar to the firm's product and whether any study measured outcomes (i.e., improved sexual performance) other than blood flow/blood circulation. Until the firm has reviewed the underlying studies, it should not assume that merely finding studies testing the same substance necessarily constitutes adequate substantiation.
Example 8: A firm wishes to market its mineral supplement by using a claim that "studies show that the mineral supplement promotes "Z." The firm has the results of a randomized, double blind, placebo-controlled study conducted in a foreign country showing that a similar product did, in fact, promote "Z," although the study indicates that the foreign study subjects had low blood levels of the mineral at the start of the study. The general U.S. population does not have such a mineral deficiency. Although this study is a high quality study, it may not be adequate to substantiate a claim about the product's use intended for consumers in the United States because it is confounded by the initial abnormal blood levels of the mineral. Since the study is not designed to answer the question of whether the effect would be expected to occur in subjects with normal blood levels of the mineral, the study may not be adequate evidence to substantiate the claim.
Example 9: A firm is marketing a product specifically to reduce nervousness during stressful everyday situations, such as public speaking. The firm has results from several small studies demonstrating that the product will raise blood levels of a chemical that is well known to relax people in stressful situations. The firm also has two small, randomized, placebo-controlled studies showing that its product positively affected measurable indices of anxiety in people placed in stressful situations, including public speaking. These studies may be adequate evidence to support the product claims. Although the studies may be small in terms of the numbers of subjects tested, they are well-designed studies that resulted in statistically significant positive results that are consistent with the larger body of scientific evidence related to stress anxiety in public situations.
Example 10: A firm has developed a product to improve memory and cognitive ability and intends to market the product to parents for their school-aged children. The firm has several high quality clinical studies that examined the ingredient's effect in elderly people with diagnosed, age-related memory problems. These studies alone would likely not be adequate substantiation for a claim about memory improvement in young children because the patient population (elderly people with memory problems) is completely different from the intended population (children) in the claim.
Example 11: A dietary supplement firm is marketing an iron dietary supplement with the claim that the dietary supplement is to correct iron-deficiency anemia in the 10% of menstruating women with menorrhagia. The firm has not studied the product in this population of women directly, but has assembled and carefully reviewed the scientific literature of studies that have investigated the oral dosage and intestinal absorption of the type of iron used in its product, both in the population in general, and in women that match the target consumer of the product. Using this information, the firm has formulated its product to provide the amount of bioavailable iron needed by this population of women. Even though the firm did not test its product directly, it has examined the existing scientific literature and has formulated the product in a manner to meet the standards of products shown effective in well-controlled studies. There is, therefore, a basis to conclude that the existing literature is applicable to the product in the target population in which it is intended. Thus, the firm's claim that the product will be useful in correcting iron-deficiency anemia would likely be adequately substantiated.
Example 12: A firm claims that its multi-vitamin, multi-mineral product "provides the vitamins and minerals needed to promote good health and wellness." In this case, the firm's claim is likely substantiated by the substantial scientific evidence showing that certain vitamins and minerals are essential nutrients that are needed to maintain good health, even though the firm does not have data from specific scientific studies to show that its product results in any measurable outcome. Scientific evidence studying the firm's particular product formulation probably would not be needed for this claim unless the firm were to make claims that its formulation is different or superior to other formulations or confers benefits above and beyond the benefits demonstrated to be associated with adequate intake of vitamins and minerals.
In deciding whether studies substantiate a claim, an important consideration is the scientific quality of studies. Scientific quality is based on several criteria including study population, study design and conduct (e.g., presence of a placebo control), data collection (e.g., dietary assessment method), statistical analysis, and outcome measures. For example, if the scientific study adequately addressed all or most of the above criteria, it would be considered of high quality. Generally accepted scientific and statistical principles should be used to determine the quality of the studies used as evidence to substantiate a claim. The "gold" standard is randomized, double blind, parallel group, placebo-controlled trial design. However, trials of this type may not always be possible, practical, or ethical. There are several systems available to rate scientific information.(7) Firms making claims are encouraged to refer to these systems when developing substantiation for claims or relying on existing information. The following provides some commonly accepted scientific principles in evaluating the quality of scientific evidence.
As a general principle, one should think about the type of evidence that would be sufficient to substantiate a claim in terms of what experts in the relevant area of study would consider to be competent and reliable. Competent and reliable scientific evidence adequate to substantiate a claim would generally consist of information derived primarily from human studies.
Human studies can be divided into two types: intervention studies and observational studies.(8) Of these types of studies, intervention studies provide the most persuasive form of evidence to substantiate the effect of a dietary supplement in humans because they evaluate the product's direct effect in the human body. Observational studies have a more limited ability than intervention studies to distinguish relationships between a substance and the outcomes being evaluated.
Intervention studies
In intervention studies, an investigator controls whether the subjects receive the treatment or intervention of interest in order to test whether the intervention or treatment supports a pre-determined hypothesis. Firms should determine the hypothesis that should be supported or tested prior to identifying supportive documentation or developing a study protocol. Randomized, double blind, parallel group, placebo-controlled trials offer the greatest assessment of a relationship between a dietary supplement and an outcome. Although interventional studies are the most reliable studies for determining a cause-and-effect relationship, generalizing from such evidence on selected populations is cautioned.
Observational studies
In observational studies, the investigator does not have control over the exposure to the treatment or intervention of interest. In prospective observational studies, investigators recruit subjects and observe them before a particular outcome occurs. In retrospective observational studies, investigators review the records of subjects and interview subjects after the outcome has occurred. Retrospective studies are usually considered to be more vulnerable to recall bias (error that occurs when subjects are asked to remember past behaviors) and measurement error, but are less likely to require large sample size, cost, or encounter the ethical problems that may occur in prospective studies. Types of observational studies include:
Case reports, which describe observations of a single subject or a small number of subjects.
Case-series studies, which are a descriptive account of a series of "outcomes" observed over time and reported for a group of subjects. No control group is described.
Case-control studies, which compare subjects with a condition (cases) to subjects who do not have the same condition (controls). Subjects are enrolled based on their outcome rather than based on their exposure.
Cohort studies, which compare the outcome of subjects who have received a substance to the outcome of subjects who have not received that substance.
Cross-sectional (prevalence) studies, which compare, at a single point in time, the number of individuals with a condition who have received a substance to the number of individuals without the condition who did not receive the substance.
Time-series studies, which compare outcomes during different time periods, e.g., whether the rate of occurrence of a particular outcome during one five-year period changed during a subsequent five-year period.
Epidemiological studies, which compare the rate of a condition across different populations.
The following additional types of information would generally be considered background information, but alone may not be adequate to substantiate a claim.
Animal studies - Animal studies may provide useful background on the biological effects of a substance. However, they often have limited or unknown value in predicting the effect of the substance in humans. Care should be exercised in extrapolating results obtained in animal research directly to the human condition. The strongest animal evidence is based on data from studies in appropriate animal models, on data that have been reproduced in different laboratories, and on data that give a statistically significant dose-response relationship. Generally, without any data from human studies, the results of animal studies alone are not sufficient to substantiate a claim.
In vitro studies are studies that are done outside a living body. For example, such studies might examine a product's effect on isolated cells or tissues. These studies are of limited value in predicting the effect of a substance when consumed by humans. The strongest in vitro evidence would be based on data that have been reproduced in different laboratories, but this evidence alone would ordinarily not be adequate to substantiate a claim.
Testimonials and other anecdotal evidence - This type of evidence includes descriptions of experiences of individuals using a dietary supplement product or ingredient. It might also include descriptions of the use of the product or ingredient by others, for example, by other cultures in the past or present. It might consist of an opinion or statement of an expert or someone who endorses the product. Anecdotal evidence generally would not be sufficient to substantiate claims regarding a dietary supplement's effect because each individual's experience might be attributable to factors other than the dietary supplement itself. For example, a person might have experienced a placebo or coincidental effect, rather than an effect attributable to the dietary supplement itself. Additionally, the "honest opinion" of a consumer testimonial or an expert endorsement would not be enough to substantiate a claim; rather, the endorsement should also be supported by competent and reliable scientific evidence.
Meta-analysis is the process of systematically combining and evaluating the results of clinical trials that have been completed or terminated. Meta-analysis may identify relevant reports, which may provide substantiation for the claim.
Review articles summarize the findings of primary reports. Review articles may identify relevant primary reports, which may provide substantiation for the claim. Review articles may also provide background information that is useful to understand the scientific issues about the relationship between the substance and the claimed effect.
Comments and Letters to the Editor usually focus on a particular issue or issues from a study, presentation at a meeting etc. Comments generally do not present the results of a study. Comments and letters to the editor may identify relevant primary reports, which may provide substantiation for the claim. Comments and letters to the editor may also provide background information that is useful to understand the scientific issues about the relationship between the substance and the claimed effect.
Product monographs are prepared by the manufacturer to convey specific information about a product such as its specifications. Product monographs may provide background information that is useful to understand the scientific issues about the relationship between the substance and the claimed effect.
Example 13: A dietary supplement claim states, "Data suggest that including Substance X in the diet may promote brain neuron health in healthy individuals." The firm cites a study in which rats were fed diets containing Substance X and the brains of all rats were examined for ischemia-induced brain damage. The study does not provide a basis that Substance X would have the same effect on brain health in otherwise healthy humans. This study alone likely would not provide adequate substantiation of the claim being made because it relies solely on animal data.
Example 14: A dietary supplement claim states, "Grain Y has been used effectively for centuries to promote gastrointestinal health." The firm has no clinical studies in humans, but has an industry monograph that relies only on historical descriptions of grain Y use by pre-modern civilizations. Although the monograph may be an accurate review of the historical use of grain Y, it would likely not constitute competent and reliable evidence to support the claim because it is not based on objective scientific evidence. Rather, it is largely anecdotal evidence that cannot be objectively evaluated to determine if it applies to the consumers who would use the product.
Example 15: A dietary supplement label claims that, in laboratory tests (i.e., in vitro tests), the enzymes in the supplement can digest up to 20 grams of protein and 15 grams of dietary fat, and the firm is promoting the supplement to assist in breaking down protein and fat that its users eat. The firm has not tested its product or the ingredients in the supplement in humans. Although this evidence may be accurate, it would generally not be adequate substantiation for the claimed effects on dietary components because it is insufficient for reaching a conclusion on whether the enzymes, when consumed, would behave equivalently in the human body. Corroborating evidence from some human studies would likely be needed to determine if the in vitro findings reflect the outcomes of the product when consumed by humans.
Example 16: A botanical product label uses the claim "improves vitality." The substantiation that the firm is relying upon consists of testimonial experience it has collected from consumers and descriptions of the botanical product's traditional use. Although the firm may have testimonial experience to back up the basic claim being made, the claimed benefit would likely not be adequately substantiated because neither source is based on scientific evidence. If the firm wants to make a claim of this type, we recommend that it have scientific evidence that some measurable outcome(s) associated with the general conditions cited in the claim is (are) significantly improved.
Multiple factors should be considered in study design. These include, but are not limited to:
Bias, confounders, and other limitations - Potential sources of bias include lack of appropriate randomization and blinding, the number of subjects called for in the protocol vs. the number of subjects who actually participated in the trial, demographics, adequacy of primary variables, compliance, control agent, drop-outs, statistical procedures, subgroup analysis, safety issues, and reproducibility of results. Confounders are factors that are associated with the outcome in question and the intervention and prevent the measured outcome from being attributed unequivocally to the intervention. Potential confounders include variability in the quantity of the dietary supplement being administered or the presence of other dietary ingredients that may have their own independent effects. These factors can limit the reliability of the study.
Quality assessment criteria - Factors that contribute to higher quality studies include:
Adequacy and clarity of the design
The questions to be answered by the study are clearly described at the outset.
The methodology used in the study is clearly described and appropriate for answering the questions posed by the study.
The duration of the study intervention or follow-up period is sufficient to detect an effect on the outcome of interest.
Potential confounding factors are identified, assessed, and/or controlled.
Subject attrition (subjects leaving the study before the study is completed) is assessed, explained, and reasonable.
Population studied
The sample size is large enough to provide sufficient statistical power to detect a significant effect. (If the study is underpowered, it may be impossible to conclude that the absence of an effect is not due to chance.)
The study population is representative (with respect to factors such as age, gender distribution, race, socioeconomic status, geographic location, family history, health status, and motivation) of the population to which the claim will be targeted.
The criteria for inclusion and exclusion of study subjects were clearly stated and appropriate.
The study used recruitment procedures that minimized selection bias.
For controlled interventions, the subjects were randomized. If matching was employed to assign the subjects to control and treatment groups, appropriate demographic characteristics and other variables were used for the matching. The randomization was successful in producing similar control and intervention groups.
Assessment of intervention or exposure and outcomes
The analytical methodology and quality control procedures to assess dietary intake are adequate.
The dietary supplement serving size is well defined and appropriately measured.
The background diets to which the dietary supplement was added, or the control and interventional diets, are adequately described, measured, and suitable.
In studies with cross-over designs, the "wash-out" period (the period during which subjects do not receive an intervention) between dietary supplement exposures is appropriate. Lack of a sufficient wash-out period between interventions may lead to confusion as to which intervention produced the health outcome.
The form and setting of the intervention are representative of the way the product will be normally used.
Other possible, concurrent changes in diet or health-related behavior (weight loss, exercise, alcohol intake, and smoking cessation) present during the study that could account for the outcome identified, are assessed and/or controlled.
The study's outcomes are well defined and appropriately measured(9)
Efforts were made to detect harmful as well as beneficial effects.
Data Analysis and Assessment
Appropriate statistical analyses were applied to the data.
"Statistical significance" was interpreted appropriately.
Relative and absolute effects were distinguished.
Peer Review - The nature and quality of the written report of the research are also important. Although studies or evidence used to substantiate a claim do not have to be published in a peer-reviewed journal or publication, such publications do give some level of assurance that qualified experts have reviewed the research and found it to be of sufficient quality and validity to merit publication. In contrast, an abstract or informal summary of an article is less reliable, because such documents usually do not give the reader enough insight into how the research was conducted or how the data were analyzed to objectively evaluate the quality of the research data and the conclusions drawn by the authors. Moreover, the mere fact that the study was published does not necessarily mean that the research is competent and reliable evidence adequate to substantiate a particular claim.
Example 17: A dietary supplement label claims, "Randomized, double blind, placebo-controlled studies demonstrate that herbal extract 'Z' is beneficial in relieving menopausal symptoms." The firm is relying on the results of more than one randomized, double blind, placebo-controlled intervention study using menopausal women as subjects, and the results of those studies are in general agreement. The claim would likely be substantiated because it relies on high quality studies in humans that directly addressed conditions described in the claim.
In determining whether there is adequate evidence to substantiate a claim, one should consider the strength of the entire body of evidence, including criteria such as quality, quantity (number of various types of studies and sample sizes), consistency, relevance of exposure, and persuasiveness.
To determine whether the available scientific evidence is adequate to substantiate a claim, it is important to consider all relevant research, both favorable and unfavorable. Ideally, the evidence used to substantiate a claim agrees with the surrounding body of evidence. Conflicting or inconsistent results raise serious questions as to whether a particular claim is substantiated. If conflicts or inconsistencies exist in the scientific evidence, one should determine whether there are plausible explanations for such conflicts or inconsistencies. For example, an inconsistency between two studies might be attributable to different concentrations of the dietary supplement, different test methodologies, different study populations,(10) or other factors.
There is no general rule for how many studies, or what combination of types of evidence, is sufficient to support a claim. However, the replication of research results in independently conducted studies adds to the persuasiveness of the evidence.
Although the quality and persuasiveness of individual pieces of evidence are important, each piece should be considered in the context of all available information; that is, the strength of the total body of scientific evidence is the critical factor in assessing whether a claim is substantiated.
Example 18: A firm intends to promote an herbal product "X" to help maintain cognitive performance of people who are fatigued. The firm has researched the scientific literature and found many studies that demonstrate that the botanical ingredient is effective. However, there are some studies that demonstrate no effect. Still other studies examined the botanical ingredient combined with other ingredients, typically caffeine, which demonstrated mixed positive and negative results. Many reports do not adequately describe the study participants and products examined. Consequently, it is not possible to explain the disparate results. However, the firm's review suggests that either the botanical and/or caffeine are the most likely dietary ingredients that act to maintain better cognition test results in fatigued study participants. As a result, the firm conducts a large, randomized, placebo-controlled study to compare the botanical ingredient against caffeine in the treatment of cognitive performance deficits associated with fatigue. The results demonstrate that caffeine improved cognition test results in all of the fatigued subjects that received caffeine, while test performance was unaffected in all subjects receiving the botanical ingredient. The study cannot explain the results reported in the earlier studies; however, it demonstrates that the botanical ingredient studied is most likely ineffective for improving or maintaining cognitive performance in fatigued people.
Example 19: A firm plans to promote its herbal product for occasional, nocturnal leg cramps. The firm has one study demonstrating the product to be effective in ameliorating nocturnal leg cramps. The firm is also aware of several other randomized controlled trials that do not show a benefit. All these studies are of equal quality and used similar patient populations and test materials. When considered as a whole, even though some evidence to support the claim exists, the totality of the evidence does not support the proposed claim. If no plausible explanation can be found to explain the disparate results, the available evidence would probably not be considered adequate to substantiate the claim.
Example 20: An herbal product is promoted to help a person get to sleep when he or she has difficulty falling asleep. The firm has one randomized, placebo-controlled study in 8 volunteers who had trouble falling asleep. The study showed that those who used the product decreased the amount of time needed time to fall asleep. There are several other large studies, however, that found that the herbal ingredient used in the product did not consistently help people get to sleep. It is not clear whether the different results of the various studies are a consequence of differences in product formulation or dosage or some other factor. Even though the firm's single study is positive, it may not provide adequate substantiation because the totality of existing evidence suggests that the herbal ingredient does not decrease time to fall asleep in persons who have trouble falling asleep. Moreover, the firm's very small study size represents a weakness. Given the weakness of the firm's study and the weight of contrary evidence against the claim, it is unlikely that a sleep-related claim would be substantiated for this product.
Example 21: A company plans to promote its product containing ingredient X to athletes to improve endurance performance. There are some well-designed published studies demonstrating that other products containing ingredient X are effective, but other well-designed studies show no effect for certain products containing ingredient X. The firm sponsored a randomized, blinded, six-month study comparing its product to four other products containing ingredient X in a dose (serving size)-response fashion. The findings demonstrate that the firm's product and two other products that provided the highest amount of ingredient X per day produced substantial, statistically significant improvements in athletic endurance. When the firm compared the results of this study to prior studies, the firm concluded that the explanation for previous conflicting study results is that when the serving size of ingredient X is below a certain amount, there is no measurable benefit. Taken together, the positive results from their study, and the identification of a plausible explanation to explain why some studies showed no positive effects, would likely provide evidence to substantiate adequately the endurance performance claim for the dietary supplement.
Section 403(r)(6) of the Act requires dietary supplement manufacturers to have substantiation that structure/function, nutrient deficiency, and general well-being claims on a dietary supplement product's labeling are truthful and not misleading. To meet this statutory requirement, we recommend that manufacturers possess adequate substantiation for each reasonable interpretation of the claims. We intend to apply a standard that is consistent with the FTC standard of "competent and reliable scientific evidence" to substantiate a claim. We consider the following factors important to establish whether information would constitute "competent and reliable scientific evidence:"
Does each study or piece of evidence bear a relationship to the specific claim(s)?
What are the individual study's or evidence's strengths and weaknesses? Consider the type of study, the design of the study, analysis of the results, and peer review.
If multiple studies exist, do the studies that have the most reliable methodologies suggest a particular outcome?
If multiple studies exist, what do most studies suggest or find? Does the totality of the evidence agree with the claim(s)?
For Further Information, Contact:
Vickey Lutwak
Office of Nutritional Products, Labeling, and Dietary Supplements
Center for Food Safety and Applied Nutrition
Food and Drug Administration
301-436-2375
(1) The Office of Nutritional Products, Labeling, and Dietary Supplements in FDA's Center for Food Safety and Applied Nutrition prepared this guidance document.
(2) Under section 403(r)(6)(A) of the Act (21 U.S.C. 343(r)(6)(A)), such a statement is one that "claims a benefit related to a classical nutritional deficiency disease and discloses the prevalence of such disease in the United States, describes the role of a nutrient or dietary ingredient intended to affect the structure or function in humans, characterizes the documented mechanism by which a nutrient or dietary ingredient acts to maintain such structure or function, or describes general well-being from consumption for a nutrient or dietary ingredient...."
(3) This guidance does not discuss the criteria to determine whether a statement about a dietary supplement is a structure/function claim under section 403(r)(6) of the Act or a disease claim. Please see the Federal Register of January 6, 2000 (65 FR 1000, codified at 21 CFR 101.93) (www.cfsan.fda.gov/~lrd/fr000106.html) for the final rule defining structure/function claims for dietary supplements and the January 9, 2002 Small Entity Compliance Guide for structure/function claims (www.cfsan.fda.gov/~dms/sclmguid.html).
(4) See Report of the Commission on Dietary Supplement Labels, November 1997, at page 42. The Commission's recommendations on substantiation are at pages 42 through 45 of the report.
(5) See Bureau of Consumer Protection, Federal Trade Commission, "Dietary Supplements: An Advertising Guide for Industry," April 2001 (hereinafter referred to as "FTC Advertising Guide"), available at www.ftc.gov.
(6) See, e.g. Vital Basics, Inc., C-4107 (Consent April 26, 2004); see also In Re Schering Corp., 118 F.T.C. 1030, 1123 (1994).
(7) See "Systems to Rate the Strength of Scientific Evidence. Evidence Report/Technology Assessment Number 47, "Agency for Healthcare Research and Quality and Research (AHRQ), Publication No. 02-E016, April 2002.
(8) See Spilker, B. Guide to Clinical Trials. Raven Press, New York, 1991.
(9) If surrogate endpoints were considered, then there should be evidence that the surrogate endpoints are valid. Surrogate endpoints are a type of biomarker, which are parameters from which risk of a condition can be inferred, rather than being a measure of the condition itself (Guidance for Industry: Significant Scientific Agreement in the Review of Health Claims for Conventional Foods and Dietary Supplements, December 22, 1999) (http://www.cfsan.fda.gov/~dms/ssaguide.html). A surrogate endpoint has several characteristics (Food Advisory Committee Transcript, June 7, 2004, pp. 50-52): (1) it is associated with a condition; (2) it mediates the relationship between intake in healthy people and a condition; and (3) its expression is modified by intake of a substance in healthy people.
(10) For example, with respect to human drug products, it is fairly well known that children and the elderly may experience different drug effects compared to those seen in the adult population. These differences may be due to physiological differences (such as hormonal differences, differences in kidney function, etc.) between children, adults, and the elderly.
November 4, 2004: FDA Announces Major Initiatives for Dietary Supplements
