By Craig Weiss, President, Consumer Product Testing Co.09.04.18
When approaching a claims-support study for any dietary supplement product that produces a cosmetic end point, also known as “beauty from within,” it is important to understand the types of claims available. Puffery, or emotive claims, are meant to grab the emotions of consumers but don’t necessarily say much about product benefits. Ingredient-based claims are more meaningful. Under the Dietary Supplement Health and Education Act (DSHEA), if there is historical, scientificallybased evidence that an ingredient has an effect on a biologic structure or function and is in a single entity product, then the marketer can legally make a claim. However, companies must also include the following disclaimer: “This statement has not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure or prevent any disease.” Typically, this disclaimer is only available for products that are ingested. Finally, performance-based claims would need to be supported by a sound, scientifically-based study.
Like any other clinical efficacy trial, the study design is key to the strength of the claims made, and the acceptability of the data produced if the claim is challenged. When approaching any efficacy trial, certain questions must be answered. For example, is the desired claim a subjective claim, or an objective claim? To draw a parallel to standard cosmetic studies, a subjective claim would be a claim such as “masks odor”; it is subjective because it is based on a sensation. Other subjective claim could be based on questionnaire data; they are subjective because often there is no validated measurement in use. Subjective claims can be very effective but are not as strong as objective claims. A subjective claim may state “78% of subjects saw a reduction in wrinkles,” whereas an objective claim may read, “a 78% reduction in wrinkles.”
Supportive data for objective claims are generally more expensive to produce because they require the expertise of highly-trained technicians using instrumentation, or calibrated by expert graders. Expert graders are a good addition to any study, as they can assure clinical relevance. There are two burdens of proof when a claim is made: statistical significance and clinical relevance. The former refers to whether or not the data across the panel of subjects show a statistically significant, or meaningful change in the parameter measured. Clinical relevance means, can the change be observed by the consumer? In today’s competitive marketplace, it is essential to have both statistical significance and clinical relevance in the completed study. Without both, the claim may be rejected legally either in a class action lawsuit or a competitive challenge.
Multiple Choices
Integral to choosing a study design, companies must decide on the type of study to be conducted (e.g., blinded or unblinded: do the volunteers and the study administration know what product is being evaluated?). In a Monadic study, also known as baseline controlled, where only one product is evaluated, intermediate and final measurements are compared to baseline. This type of study is used when a control is not easily made or available. If testing a moisturizing cream, for example, any cream will be moisturizing, so what is the control product?
Crossover studies are another option. In a crossover study, volunteers are randomly assigned into groups. Each group is assigned a different study product to start the study; this way, each product is used first by at least one group, nullifying the first-used product bias, which is especially important in studies that rely on questionnaires for data. There is then a use period followed by a measurement, a rest phase to allow the effects of the first product used to disappear, and a remeasurement. Following the remeasurement, the subject group or cohort begins the use of the next product. This is repeated until all subjects have used all products. The strength of this design is that all products being compared were evaluated on the same subjects. This type of study is especially important to compare similar products.
Finally, and most importantly for “beauty from within” claim support studies, are placebo-controlled studies. These are typically conducted when possible in a double-blind manner, where the investigator and the subject do not know which group is the using the placebo control. There are times when it is not possible to create a true placebo control, and unless the identity is known only to the study sponsor then the study would be conducted as a single blind study.
With an understanding of claims and study design, consider how to measure product performance. Objectively measuring the performance of a product is the same regardless of what is tested, as the actual measurement is endpoint driven. When gathering instrumentation data as a primary measurement of product performance, it is important that the instrumentation is “industry standard,” or has been validated for the measurement being considered. That’s not to say that new instrumentation cannot be used in the study; however, the instrument should be validated against an existing technique. Additionally, the instrument being used should be calibrated either on a routine basis or at time of use.
Clinical photography should always be considered. The adage, “a picture is worth a thousand words,” applies here as well. Incorporate images into marketing programs; if challenged, the picture may be able to enhance legal defense. Clinical photography is typically different from photography used for advertising products as it usually focuses on a very specific area or feature. Additionally, these photographs should be taken on a photographic system that allows for lining up the subject in the same manner each time and that does not allow for the image to be altered. If the image is used for making measurements then the photographic system should be validated.
Some Caveats
Consider the testing environment. If the claims are based on a visual color evaluation, make sure the lighting in the evaluation area is sufficient to see that color. Most skin-related studies must be conducted in an environmentally-controlled room, with appropriate equilibration time, as when measuring the skin, temperature and humidity are critical.
It is advisable to conduct many skin studies in the cold dry months as changes in humidity and elevated temperatures can skew results, and not necessarily for the better. When humidity increases, skin typically appears more hydrated, which can skew results. When temperatures rise, subjects perspire, which could skew moisturization values higher, or the subject may become somewhat dehydrated, which could skew moisturization values lower.
Subject selection is another part of assembling an appropriate claims validation study. Questions to answer include: How many subjects? Who is to be included? Who is to be excluded? Are subjects in the same condition at the start of the study? The number of subjects to be included is difficult to know. If you were to ask a statistician to conduct a power computation, you would need to know the incident rate of the change you are measuring; as most of the products being tested are in development, this can be estimated, but not necessarily known. As of the writing of this article, the industry best practice is 30-35 subjects, simply because many retailers look for that many subjects. The demographics for a study depends on the market for the product. If the product is geared toward a specific age group, ethnicity, or specific condition, the volunteers should fit that demographic and have the specific condition.
Subjects who fall out of the selection criteria should be excluded from the study. Additional exclusion criteria typically include items like medication that would interfere with the study, medical conditions, pregnancy, and inability to follow study directions.
Study duration is usually determined by how long it is estimated it should take the product to produce the desired result. If a product should reach peak efficacy in most subjects in six weeks, many may want to stop the study at that point. However, consider conducting the study for significantly longer, perhaps as much as 12 weeks. As with many population-based statistics, peak efficacy for different people happens at contrasting times; it may even resemble the classic bell curve. Lengthening the study allows time for the slower reactors to hit peak product efficacy and will provide enough time to evaluate any regression. With longer study duration, it is advisable to have periodic evaluations during the course of the study.
In order to bring subjects to the same condition, it is often advisable to have a conditioning phase prior to the taking of baseline data. During a conditioning phase, study volunteers may be prohibited from using their normal cosmetic products and be asked to use only cosmetic products supplied by the laboratory.
Case Study in Design
With the essential parts of a study design in place, consider how this all comes together. In order to make this easily understandable, consider a study design for a nutritional supplement product that wants to make anti-aging claims of “reduces the appearance of fine lines and wrinkles, makes skin firmer, and increases hydration.” This is a good starting point, as these claims are for a dietary supplement and are amenable to a double-blind, placebo-controlled study, which is the most rigorous and least likely to be challenged successfully.
Assume it is estimated that most subjects should see the promised product benefits in six weeks; therefore, the study term should be 12 weeks. A double-blind, placebo-controlled
study requires two groups of 30 randomly-selected subjects.
For a dietary supplement, as the product’s benefits are specific to a population with wrinkles and no specific gender requirement, it would be necessary to test on subjects that
have wrinkles.
What follows is a progressive way to assemble a study design using and reinforcing the points made in this article.
Study Design Outline
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
N=60; two groups of 30 (randomized selection)
As the product’s benefits are specific to a population with wrinkles and no specific gender requirement it would be necessary to test on subjects that have wrinkles:
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
As the study duration is long there should be at least one interim visit:
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
Based on this study design or outline, build a study schedule. This study schedule must include a washout period where the subjects did not use any moisturizing or anti-aging products.
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
Day 14
Baseline visit
Here are a few final suggestions for what to look for when placing a study of this nature at a contract facility: audit the laboratory for calibration, training of expert graders, and technical staff. You are responsible for the claims you make; make sure the data makes sense. There should be some variance in the data, as not all subjects will react the same to your product. The adage “if it looks too good to be true, then it probably is” holds up here.
About the author: Craig R. Weiss is president of Consumer Product Testing, a global leader in the testing of personal care, pharmaceutical and therapeutic products. For more information: 973-808-7111; CRWeiss@cptclabs.com; www.cptclabs.com.
Like any other clinical efficacy trial, the study design is key to the strength of the claims made, and the acceptability of the data produced if the claim is challenged. When approaching any efficacy trial, certain questions must be answered. For example, is the desired claim a subjective claim, or an objective claim? To draw a parallel to standard cosmetic studies, a subjective claim would be a claim such as “masks odor”; it is subjective because it is based on a sensation. Other subjective claim could be based on questionnaire data; they are subjective because often there is no validated measurement in use. Subjective claims can be very effective but are not as strong as objective claims. A subjective claim may state “78% of subjects saw a reduction in wrinkles,” whereas an objective claim may read, “a 78% reduction in wrinkles.”
Supportive data for objective claims are generally more expensive to produce because they require the expertise of highly-trained technicians using instrumentation, or calibrated by expert graders. Expert graders are a good addition to any study, as they can assure clinical relevance. There are two burdens of proof when a claim is made: statistical significance and clinical relevance. The former refers to whether or not the data across the panel of subjects show a statistically significant, or meaningful change in the parameter measured. Clinical relevance means, can the change be observed by the consumer? In today’s competitive marketplace, it is essential to have both statistical significance and clinical relevance in the completed study. Without both, the claim may be rejected legally either in a class action lawsuit or a competitive challenge.
Multiple Choices
Integral to choosing a study design, companies must decide on the type of study to be conducted (e.g., blinded or unblinded: do the volunteers and the study administration know what product is being evaluated?). In a Monadic study, also known as baseline controlled, where only one product is evaluated, intermediate and final measurements are compared to baseline. This type of study is used when a control is not easily made or available. If testing a moisturizing cream, for example, any cream will be moisturizing, so what is the control product?
Crossover studies are another option. In a crossover study, volunteers are randomly assigned into groups. Each group is assigned a different study product to start the study; this way, each product is used first by at least one group, nullifying the first-used product bias, which is especially important in studies that rely on questionnaires for data. There is then a use period followed by a measurement, a rest phase to allow the effects of the first product used to disappear, and a remeasurement. Following the remeasurement, the subject group or cohort begins the use of the next product. This is repeated until all subjects have used all products. The strength of this design is that all products being compared were evaluated on the same subjects. This type of study is especially important to compare similar products.
Finally, and most importantly for “beauty from within” claim support studies, are placebo-controlled studies. These are typically conducted when possible in a double-blind manner, where the investigator and the subject do not know which group is the using the placebo control. There are times when it is not possible to create a true placebo control, and unless the identity is known only to the study sponsor then the study would be conducted as a single blind study.
With an understanding of claims and study design, consider how to measure product performance. Objectively measuring the performance of a product is the same regardless of what is tested, as the actual measurement is endpoint driven. When gathering instrumentation data as a primary measurement of product performance, it is important that the instrumentation is “industry standard,” or has been validated for the measurement being considered. That’s not to say that new instrumentation cannot be used in the study; however, the instrument should be validated against an existing technique. Additionally, the instrument being used should be calibrated either on a routine basis or at time of use.
Clinical photography should always be considered. The adage, “a picture is worth a thousand words,” applies here as well. Incorporate images into marketing programs; if challenged, the picture may be able to enhance legal defense. Clinical photography is typically different from photography used for advertising products as it usually focuses on a very specific area or feature. Additionally, these photographs should be taken on a photographic system that allows for lining up the subject in the same manner each time and that does not allow for the image to be altered. If the image is used for making measurements then the photographic system should be validated.
Some Caveats
Consider the testing environment. If the claims are based on a visual color evaluation, make sure the lighting in the evaluation area is sufficient to see that color. Most skin-related studies must be conducted in an environmentally-controlled room, with appropriate equilibration time, as when measuring the skin, temperature and humidity are critical.
It is advisable to conduct many skin studies in the cold dry months as changes in humidity and elevated temperatures can skew results, and not necessarily for the better. When humidity increases, skin typically appears more hydrated, which can skew results. When temperatures rise, subjects perspire, which could skew moisturization values higher, or the subject may become somewhat dehydrated, which could skew moisturization values lower.
Subject selection is another part of assembling an appropriate claims validation study. Questions to answer include: How many subjects? Who is to be included? Who is to be excluded? Are subjects in the same condition at the start of the study? The number of subjects to be included is difficult to know. If you were to ask a statistician to conduct a power computation, you would need to know the incident rate of the change you are measuring; as most of the products being tested are in development, this can be estimated, but not necessarily known. As of the writing of this article, the industry best practice is 30-35 subjects, simply because many retailers look for that many subjects. The demographics for a study depends on the market for the product. If the product is geared toward a specific age group, ethnicity, or specific condition, the volunteers should fit that demographic and have the specific condition.
Subjects who fall out of the selection criteria should be excluded from the study. Additional exclusion criteria typically include items like medication that would interfere with the study, medical conditions, pregnancy, and inability to follow study directions.
Study duration is usually determined by how long it is estimated it should take the product to produce the desired result. If a product should reach peak efficacy in most subjects in six weeks, many may want to stop the study at that point. However, consider conducting the study for significantly longer, perhaps as much as 12 weeks. As with many population-based statistics, peak efficacy for different people happens at contrasting times; it may even resemble the classic bell curve. Lengthening the study allows time for the slower reactors to hit peak product efficacy and will provide enough time to evaluate any regression. With longer study duration, it is advisable to have periodic evaluations during the course of the study.
In order to bring subjects to the same condition, it is often advisable to have a conditioning phase prior to the taking of baseline data. During a conditioning phase, study volunteers may be prohibited from using their normal cosmetic products and be asked to use only cosmetic products supplied by the laboratory.
Case Study in Design
With the essential parts of a study design in place, consider how this all comes together. In order to make this easily understandable, consider a study design for a nutritional supplement product that wants to make anti-aging claims of “reduces the appearance of fine lines and wrinkles, makes skin firmer, and increases hydration.” This is a good starting point, as these claims are for a dietary supplement and are amenable to a double-blind, placebo-controlled study, which is the most rigorous and least likely to be challenged successfully.
Assume it is estimated that most subjects should see the promised product benefits in six weeks; therefore, the study term should be 12 weeks. A double-blind, placebo-controlled
study requires two groups of 30 randomly-selected subjects.
For a dietary supplement, as the product’s benefits are specific to a population with wrinkles and no specific gender requirement, it would be necessary to test on subjects that
have wrinkles.
What follows is a progressive way to assemble a study design using and reinforcing the points made in this article.
Study Design Outline
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
N=60; two groups of 30 (randomized selection)
As the product’s benefits are specific to a population with wrinkles and no specific gender requirement it would be necessary to test on subjects that have wrinkles:
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
- N=60; men and women age 45-65; two groups of 30 (randomized selection)
As the study duration is long there should be at least one interim visit:
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
- N=60; men and women age 45-65; two groups of 30 (randomized selection)
- Three visits (baseline, six weeks, 12 weeks)
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
- N=60; men and women age 45-65; two groups of 30 (randomized selection)
- Three visits (baseline, six weeks, 12 weeks)
- Evaluations: Fine lines and wrinkles by profilometry; Firmness by viscoelasticity; Hydration by TEWL or conductance capacitance measurement
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
- N=60; men and women age 45-65; two groups of 30 (randomized selection)
- Three visits (baseline, six weeks, 12 weeks)
- Evaluations: Fine lines and wrinkles by profilometry and expert clinical grading; Firmness by viscoelasticity; Hydration by TEWL or conductance capacitance measurement; clinical photography
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
- N=60; men and women age 45-65; two groups of 30 (randomized selection)
- Three visits (baseline, six weeks, 12 weeks)
- Evaluations: Fine lines and wrinkles by profilometry and expert clinical grading; Firmness by viscoelasticity; Hydration by TEWL or conductance capacitance measurement; clinical photography; Questionnaire with appropriate statistics
Based on this study design or outline, build a study schedule. This study schedule must include a washout period where the subjects did not use any moisturizing or anti-aging products.
12 Week Double-Blind, Placebo-Controlled Evaluation of a Dietary Supplement
Day 14
- Possible subjects are asked to sign their informed consent; given the conditioning phase products
Baseline visit
- Subjects are equilibrated in a controlled environmental room (30 minutes); Assigned to a product or placebo group
- Evaluations: Fine lines and wrinkles by profilometry and expert clinical grading; Firmness by viscoelasticity; Hydration by TEWL or conductance capacitance measurement; Clinical photography; Questionnaire (optional)
- Distribution of product or placebo and a product usage diary
- Subjects are equilibrated in a controlled environmental room (30 minutes)
- Evaluations: Fine lines and wrinkles by profilometry and expert clinical grading; Firmness by viscoelasticity; Hydration by TEWL or conductance capacitance measurement; Clinical photography; Questionnaire (optional)
- Compliance check of product usage diary
- Subjects are equilibrated in a controlled environmental room (30 minutes)
- Evaluations: Fine lines and wrinkles by profilometry and expert clinical grading; Firmness by viscoelasticity; Hydration by TEWL or conductance capacitance measurement; Clinical photography; Questionnaire (optional)
- Compliance check of product usage diary
Here are a few final suggestions for what to look for when placing a study of this nature at a contract facility: audit the laboratory for calibration, training of expert graders, and technical staff. You are responsible for the claims you make; make sure the data makes sense. There should be some variance in the data, as not all subjects will react the same to your product. The adage “if it looks too good to be true, then it probably is” holds up here.
About the author: Craig R. Weiss is president of Consumer Product Testing, a global leader in the testing of personal care, pharmaceutical and therapeutic products. For more information: 973-808-7111; CRWeiss@cptclabs.com; www.cptclabs.com.
