Douglas Kalman, PhD04.01.11
During the last 16 years, since the passage of the Dietary Supplement Health & Education Act (DSHEA), with many opportunities to create a “black and white” definition of substantiation, the regulatory bodies in this sector have been relatively quiet regarding this matter—until now.
Indeed the recent FTC vs. POM as well as the Nestle, Iovate and Mark Dreher PhD cases may offer a clearer definition of substantiation going forward. Further, what FTC has said publicly about potential study designs may be an indication of the future for science-based products in the natural products industry.
The ‘Nuts & Bolts’ of Study Design
The FTC has stated that two well-designed randomized, double-blind, placebo-controlled trials (RCTs) represent the gold standard for substantiation. To gain a full understanding of RCTs it helps to run through some definitions. Understanding these common definitions represents a good first step in creating an original substantiated product.
Randomized is defined as “to make random in arrangement, especially in order to control the variables in an experiment,” while double-blind is defined as “a testing procedure designed to eliminate biased results, in which the identity of those receiving a test treatment is concealed from both administrators and subjects until after the study is completed.” A placebo is considered “an inactive substance or preparation used as a control in an experiment or test to determine the effectiveness of a medicinal drug or other product.”
The FTC also mentions terms such as “adequately powered” in the conversation regarding substantiation. Adequately powered means the study contains enough research volunteers (subjects) and controls so that the results are more likely accurate, rather than occurring by chance alone. For an adequately powered study to produce real results, one also has to look at the effect (effect size) of the intervention (as compared to placebo). Knowing the effect size or predicted effect size allows the study team to determine how many subjects will be needed for the study. Having the right amount of subjects calculated formally during the study design period allows for a cleaner study and clearer results.
One reason FTC requires studies to contain a placebo is to help rule out a placebo effect. The placebo effect occurs when an inert or non-active substance is used and the person taking it gets the same or similar results as the intervention product. Placebo effects typically occur in psychological studies (i.e., forms of depression) as well as in some physical intervention trials (i.e., osteoarthritis trials where quality of life issues are prominent). Many studies have shown a real placebo effect and an improvement equal to or beyond that of the medical or nutritional intervention, posing a dilemma for researchers.
But observing a placebo effect in a study does not necessarily mean that the interventional product did not have a real effect, either physiologically or psychologically. A commonly asked question regarding placebos is why use one if it has the potential to cloud the results. The answer is simple: using a placebo allows for more real results.
There is a way to utilize a placebo in clinical trials without risking good data, by using a “placebo run-in.” In this case, just prior to administering the intervention, researchers will allow study participants to take a placebo for a specific amount of time. The placebo run-in is detailed in the “informed consent” as part of the study and study procedures, however subjects may not know exactly when they are receiving placebo versus the real product.
By using the placebo run-in you allow all subjects to experience a placebo response, which may diminish the placebo response during the actual intervention phase of the study (i.e., when the real product is given and placebo is also given in a controlled, randomized way). One could also argue that utilizing the right placebo run-in might enhance the FTC (and perhaps FDA) view and judgment about your study design and results (i.e., they are more believable, and there is a lower likelihood of placebo effect), especially if the results demonstrate an effect of the intervention.
But perhaps your company isn’t quite ready for an RCT. In this case, it might be most logical, as part of your Product Master File—which is often used as a dossier for substantiation—to consider a “Proof of Concept” study (PoC). These studies typically use a pre-set amount of people, whereas the RCT will utilize a more formal sample size calculation. (The sample size calculation can be affected by many things, including the effect size expected—if known—of the intervention product being evaluated.) The PoC study allows researchers to obtain a rough idea of the effects of the intervention. In other words, the results should be interpreted with some caution and not fully embraced as conclusive.
Some PoC studies will utilize a design in which 90% or so of the subjects are on active product while only one or two people are on placebo. This type of study also gives researchers an indication of the degree of the placebo effect, while also making sure the lab test data are being analyzed correctly. Additionally, the results can help you determine the effect (effect size of your intervention) and thus how many subjects you would need for a larger trial.
A new study design being employed in pharmaceutical research today is “adaptive design.” According to FDA, “There is great interest in the possibility that clinical trials can be designed with adaptive features (i.e., changes in design or analyses guided by examination of the accumulated data at an interim point in the trial) that may make the studies more efficient (e.g., shorter duration, fewer patients), more likely to demonstrate an effect of the drug if one exists, or more informative (e.g., by providing broader dose-response information).” This newer research design has gained popularity because it has the potential to answer multiple questions within the same study.
Ultimately, it is in the best interest of all companies involved in the natural products sector to gather the most data on their products, especially if FTC and consumers are to believe the claims.
Indeed the recent FTC vs. POM as well as the Nestle, Iovate and Mark Dreher PhD cases may offer a clearer definition of substantiation going forward. Further, what FTC has said publicly about potential study designs may be an indication of the future for science-based products in the natural products industry.
The ‘Nuts & Bolts’ of Study Design
The FTC has stated that two well-designed randomized, double-blind, placebo-controlled trials (RCTs) represent the gold standard for substantiation. To gain a full understanding of RCTs it helps to run through some definitions. Understanding these common definitions represents a good first step in creating an original substantiated product.
Randomized is defined as “to make random in arrangement, especially in order to control the variables in an experiment,” while double-blind is defined as “a testing procedure designed to eliminate biased results, in which the identity of those receiving a test treatment is concealed from both administrators and subjects until after the study is completed.” A placebo is considered “an inactive substance or preparation used as a control in an experiment or test to determine the effectiveness of a medicinal drug or other product.”
The FTC also mentions terms such as “adequately powered” in the conversation regarding substantiation. Adequately powered means the study contains enough research volunteers (subjects) and controls so that the results are more likely accurate, rather than occurring by chance alone. For an adequately powered study to produce real results, one also has to look at the effect (effect size) of the intervention (as compared to placebo). Knowing the effect size or predicted effect size allows the study team to determine how many subjects will be needed for the study. Having the right amount of subjects calculated formally during the study design period allows for a cleaner study and clearer results.
One reason FTC requires studies to contain a placebo is to help rule out a placebo effect. The placebo effect occurs when an inert or non-active substance is used and the person taking it gets the same or similar results as the intervention product. Placebo effects typically occur in psychological studies (i.e., forms of depression) as well as in some physical intervention trials (i.e., osteoarthritis trials where quality of life issues are prominent). Many studies have shown a real placebo effect and an improvement equal to or beyond that of the medical or nutritional intervention, posing a dilemma for researchers.
But observing a placebo effect in a study does not necessarily mean that the interventional product did not have a real effect, either physiologically or psychologically. A commonly asked question regarding placebos is why use one if it has the potential to cloud the results. The answer is simple: using a placebo allows for more real results.
There is a way to utilize a placebo in clinical trials without risking good data, by using a “placebo run-in.” In this case, just prior to administering the intervention, researchers will allow study participants to take a placebo for a specific amount of time. The placebo run-in is detailed in the “informed consent” as part of the study and study procedures, however subjects may not know exactly when they are receiving placebo versus the real product.
By using the placebo run-in you allow all subjects to experience a placebo response, which may diminish the placebo response during the actual intervention phase of the study (i.e., when the real product is given and placebo is also given in a controlled, randomized way). One could also argue that utilizing the right placebo run-in might enhance the FTC (and perhaps FDA) view and judgment about your study design and results (i.e., they are more believable, and there is a lower likelihood of placebo effect), especially if the results demonstrate an effect of the intervention.
But perhaps your company isn’t quite ready for an RCT. In this case, it might be most logical, as part of your Product Master File—which is often used as a dossier for substantiation—to consider a “Proof of Concept” study (PoC). These studies typically use a pre-set amount of people, whereas the RCT will utilize a more formal sample size calculation. (The sample size calculation can be affected by many things, including the effect size expected—if known—of the intervention product being evaluated.) The PoC study allows researchers to obtain a rough idea of the effects of the intervention. In other words, the results should be interpreted with some caution and not fully embraced as conclusive.
Some PoC studies will utilize a design in which 90% or so of the subjects are on active product while only one or two people are on placebo. This type of study also gives researchers an indication of the degree of the placebo effect, while also making sure the lab test data are being analyzed correctly. Additionally, the results can help you determine the effect (effect size of your intervention) and thus how many subjects you would need for a larger trial.
A new study design being employed in pharmaceutical research today is “adaptive design.” According to FDA, “There is great interest in the possibility that clinical trials can be designed with adaptive features (i.e., changes in design or analyses guided by examination of the accumulated data at an interim point in the trial) that may make the studies more efficient (e.g., shorter duration, fewer patients), more likely to demonstrate an effect of the drug if one exists, or more informative (e.g., by providing broader dose-response information).” This newer research design has gained popularity because it has the potential to answer multiple questions within the same study.
Ultimately, it is in the best interest of all companies involved in the natural products sector to gather the most data on their products, especially if FTC and consumers are to believe the claims.
