Mark Roman11.01.05
The goal of any analytical test method, whether it is for the analysis of pesticides in food, an active in a pharmaceutical product or a marker compound in a botanical raw material, is to provide accurate, reproducible results. How do we know that a method yields accurate, reproducible results? Hopefully, the method has been validated-that is, it has been shown to be valid for its intended purpose. But what is validation, and how do we go about proving that a method is valid for its intended purpose? This topic has become of central importance in the dietary supplement industry as manufacturers are being called upon by FDA to support their ingredient label claims. Proving their label claims requires analytical testing, both of the raw materials and the finished products. It has not been uncommon in the dietary supplement industry, however, for two different labs to test the same material for the same compound and obtain two completely different results. These differences can often be attributed to a variety of factors, including differences in sample extraction procedures; selectivity of method; and scope and applicability of the method.
When the Dietary Supplement Health and Education Act (DSHEA) was passed by Congress in 1994, it contained the following statement (emphasis added):
"The Secretary may by regulation prescribe good manufacturing practices (GMPs) for dietary supplements. Such regulations shall be modeled after current good manufacturing practice regulations for food and may not impose standards for which there is no current and generally available analytical methodology. No standard of current good manufacturing practice may be imposed unless such standard is included in a regulation promulgated after notice and opportunity for comment in accordance with chapter 5 of title 5, United States Code."
While "final" GMPs have yet to be issued by FDA, the "proposed" GMPs make it clear that analytical testing of both raw materials and finished products are of great importance to ensuring the quality of dietary supplements. Unfortunately, for most ingredients there are no "current and generally available" methodologies. As a result, many QC laboratories and contract analytical laboratories use their own in-house methods, but agreement of results between laboratories is often poor. It would be very beneficial to the industry, therefore, if universally accepted "gold standard" methods were available for dietary supplements to ensure consistency in results between laboratories.
There have been some limited attempts to promulgate "official" methods by organizations such as the USP, Institute for Nutraceutical Advancement (INA), American Herbal Pharmacopoeia (AHP), as well as other groups, but often these methods have been inconsistent between organizations and have very limited information about the inter-laboratory reproducibility of the results. As a result, the FDA and NIH Office of Dietary Supplements (ODS) contracted with AOAC International (AOACI) to identify and validate initially 20 analytical methods for use with dietary supplements.
Many people within the dietary supplement industry were not familiar with AOACI when this contract was signed, and many still are not aware of the organization. AOAC International was founded in 1884 as the Association of Official Agricultural Chemists in response to quality concerns of fertilizers used in the agricultural industry. Back then, different companies used different testing methodologies to make claims regarding the nutrient content of their fertilizers-a situation very similar to the one today in the dietary supplement industry. The scope of AOACI's work broadened considerably, and AOACI was incorporated into the FDA in 1927. In 1965, the organization changed its name to the Association of Official Analytical Chemists, and in 1979 the organization was spun off from FDA and today remains an independent not-for-profit organization. In order to reflect the diverse membership, including microbiologists, botanists, pharmacologists, and many other professions, as well as the international character of the organization, the name was officially changed to AOAC International, maintaining the initials but dropping any reference to "analytical chemists."
AOACI's function is to validate methods through collaborative studies. It is recognized in the Code of Federal Regulations as the primary source for analytical testing methods for the food industry, and hence has official regulatory status, much like the USP has for drugs. These methods are published in the AOACI's "Official Methods of Analysis," and many of these methods are recognized as official not only in the U.S., but by other countries and international organizations like the World Health Organization (WHO), CODEX and ISO.
The process by which AOACI accepts an official method of analysis for dietary supplements is based upon peer review and volunteer efforts. An Ingredient Ranking Subcommittee (IRS-not to be confused with the U.S. tax collection agency), which is composed of a mixture of government, academia, and industry representatives familiar with the supplement industry, has currently prioritized several ingredients based upon safety concerns, market share, interest in clinical trials and other factors. Expert Review Panels (ERPs) then meet to discuss the merits and drawbacks of available methods for a particular ingredient. These methods can be culled from the literature or come from in-house methods submitted by industry, government organizations or academic institutions. After extensive review and discussion, the ERP will then select a method (or methods) for that ingredient with which to move forward.
Before moving on to a full collaborative study, it must first be shown that the method can perform satisfactorily in a single laboratory, so it must undergo a "single laboratory validation" (SLV) since the existing validation data often does not meet AOACI's requirements.
When AOACI attempts to validate a method, it measures certain qualities about the method-referred to as "performance characteristics." Performance characteristics are "the functional qualities and statistical measures of the degree of reliability exhibited by the method under specified operating conditions" (AOACI Guidelines for Single Laboratory Validation, 2003). They answer the question "how good is the method?" Performance characteristics are defined by functional qualities and performance limits. The functional qualities of a method include:
• Scope and Applicability
• Selectivity
• Precision (measured as the variability, both intra- and inter-laboratory)
• Reliability (measured as the recovery, often called "accuracy")
• Range
The performance limits are the limits or ranges within which each functional quality measurement must fall for the method to be considered valid. The analyst is then challenged with coming up with a series of experiments to determine each of those functional qualities of the method.
A properly designed single laboratory validation (SLV) can determine all of these functional qualities except the reproducibility of the method-that is, how reproducible the results are when obtained by different laboratories using the same method on the same sample. Results from SLV apply to a specific operator (or operators) in a specific laboratory using specific equipment. These results provide no information about how a method will perform once it goes outside of the validating laboratory.
AOACI will help to identify a study coordinator and validating laboratory for the SLV. The study coordinator will prepare a validation protocol, which will then be reviewed by the Methods Committee on Dietary Supplements ("Committee K") for technical soundness and conformance to AOACI's SLV requirements. Upon approval of the protocol, the selected laboratory will execute the protocol and submit an SLV manuscript for review. Upon approval of the manuscript by Committee K, it will be published in the Journal of AOACI, and assuming the validation data meet the required performance limits, the method will be recommended for collaborative study.
The next phase of method validation is carried out through collaborative study. A collaborative study requires at least eight participating laboratories, and will often involve more-AOACI usually tries to target at least 12 laboratories in case some drop out or there are any "outliers." All Official Methods of Analysis (OMA) of the AOACI have undergone full collaborative studies. Samples are sent out usually as "blind duplicates" to the participating laboratories; that is, the laboratories are not given the identities or the expected concentrations of analyte in the samples in order to avoid potential bias in the results. The laboratories test the samples using the proposed method exactly as written; deviations from the written method can cause the data to be invalidated. The results are sent to the study director, who performs a statistical analysis on the results. If the statistical analysis shows that the reproducibility of the method, expressed as a % Relative Standard Deviation (% RSD) is within an acceptable range, the study director can recommend that method be adopted as "Official First Action." After review by Committee K, and usually two members of the AOACI Official Methods Board, the method can be accepted as Official First Action. The collaborative study manuscript is published in the Journal of AOACI, and the method, along with the appropriate validation data, is published in the Official Methods of Analysis, with the designation of Official First Action.
After two years, if no problems with the method have been reported to the study director, the method can be recommended for Final Action by Committee K to the Official Methods Board. Upon approval by the Official Methods Board, the method is considered final, and then published in the Official Methods of Analysis as such.
If a method has made it this far through the process, a couple of things will become evident: First, there are numerous reviews by various panels, committees and boards of any method that makes through the entire process. The description of the process makes it sound straightforward and linear, when in fact, it is not-revisions, retests, and rewrites are often required. Second, by the time a method has made it through the collaborative study process, it has become the most studied and validated method for that particular analyte in the world.
Currently, three methods have been accepted as Official First Action for dietary supplements: (1) ephedrine alkaloids in botanicals in dietary supplements; (2) glucosamine in raw materials and dietary supplements; and (3) beta-carotene in raw materials and dietary supplements. A method for the determination of flavonol glycosides in ginkgo biloba raw materials and dietary supplements recently completed the collaborative study phase, and a method for the determination of phytosterols in saw palmetto is currently undergoing a collaborative study. In addition, methods for St. John's Wort, aristolochic acid and terpene lactones in ginkgo biloba are undergoing SLVs. ERPs have met for CoQ10, omega 3 fatty acids, chondroitin sulfate and phenolics.
When the FDA and NIH-ODS contract was first signed with AOACI, it generally took two years (or more) for a method to go through the full validation process. Now methods can be validated in under a year, although sometimes it can take longer. If questioning why it takes so long for methods to go through the validation process, there are a couple of things to consider. First, these methods are to be used for a wide variety of matrices, such as botanical raw materials, extracts and finished dosage forms, and developing rugged methods and validating them for these products is very challenging. Second, most of the work is carried out by volunteers. In fact, study directors, ERP and committee members, and participating laboratories are all working on a volunteer basis. The fact that so much has already been accomplished by volunteers is a testament to their dedication.
There are still several more methods that need to be developed and validated. Indeed, only the surface has been scratched. AOACI has put out a call for methods for an additional 14 ingredients, including ginseng, echinacea and black cohosh.
The benefits of using an AOAC Official Methods of Analysis to assure product quality and support label claims are manifold. First, FDA recognizes these methods as official, much like it recognizes USP monographs as official for drug testing. Also, U.S. court systems recognize these methods as official in the case of disputes. Because these methods are considered "reference" methods, they can be used to validate in-house methods. Lastly, because all the validation data is published, the amount of variability in results that can be expected between laboratories will be known. In the end, the use of validated Official Methods of Analyses will help level the playing field in the dietary supplement industry, ensure that manufacturers are meeting label claims, help evaluate data from clinical trials performed on dietary supplements, and reduce the discrepancy in results between laboratories.NW
About the author: Mark Roman is chair of AOAC International's "Committee K." He is also the director of Research & Development for Chromadex, Clearwater, FL. He can be reached at 727-573-9229; Fax: 727-573-9220; E-mail: mark.roman@chromadex.com; Website: www.chromadex.com; www.aoac.org.
When the Dietary Supplement Health and Education Act (DSHEA) was passed by Congress in 1994, it contained the following statement (emphasis added):
"The Secretary may by regulation prescribe good manufacturing practices (GMPs) for dietary supplements. Such regulations shall be modeled after current good manufacturing practice regulations for food and may not impose standards for which there is no current and generally available analytical methodology. No standard of current good manufacturing practice may be imposed unless such standard is included in a regulation promulgated after notice and opportunity for comment in accordance with chapter 5 of title 5, United States Code."
While "final" GMPs have yet to be issued by FDA, the "proposed" GMPs make it clear that analytical testing of both raw materials and finished products are of great importance to ensuring the quality of dietary supplements. Unfortunately, for most ingredients there are no "current and generally available" methodologies. As a result, many QC laboratories and contract analytical laboratories use their own in-house methods, but agreement of results between laboratories is often poor. It would be very beneficial to the industry, therefore, if universally accepted "gold standard" methods were available for dietary supplements to ensure consistency in results between laboratories.
The Evolution of 'Official' Analytical Methods
There have been some limited attempts to promulgate "official" methods by organizations such as the USP, Institute for Nutraceutical Advancement (INA), American Herbal Pharmacopoeia (AHP), as well as other groups, but often these methods have been inconsistent between organizations and have very limited information about the inter-laboratory reproducibility of the results. As a result, the FDA and NIH Office of Dietary Supplements (ODS) contracted with AOAC International (AOACI) to identify and validate initially 20 analytical methods for use with dietary supplements.
Many people within the dietary supplement industry were not familiar with AOACI when this contract was signed, and many still are not aware of the organization. AOAC International was founded in 1884 as the Association of Official Agricultural Chemists in response to quality concerns of fertilizers used in the agricultural industry. Back then, different companies used different testing methodologies to make claims regarding the nutrient content of their fertilizers-a situation very similar to the one today in the dietary supplement industry. The scope of AOACI's work broadened considerably, and AOACI was incorporated into the FDA in 1927. In 1965, the organization changed its name to the Association of Official Analytical Chemists, and in 1979 the organization was spun off from FDA and today remains an independent not-for-profit organization. In order to reflect the diverse membership, including microbiologists, botanists, pharmacologists, and many other professions, as well as the international character of the organization, the name was officially changed to AOAC International, maintaining the initials but dropping any reference to "analytical chemists."
AOACI's function is to validate methods through collaborative studies. It is recognized in the Code of Federal Regulations as the primary source for analytical testing methods for the food industry, and hence has official regulatory status, much like the USP has for drugs. These methods are published in the AOACI's "Official Methods of Analysis," and many of these methods are recognized as official not only in the U.S., but by other countries and international organizations like the World Health Organization (WHO), CODEX and ISO.
The process by which AOACI accepts an official method of analysis for dietary supplements is based upon peer review and volunteer efforts. An Ingredient Ranking Subcommittee (IRS-not to be confused with the U.S. tax collection agency), which is composed of a mixture of government, academia, and industry representatives familiar with the supplement industry, has currently prioritized several ingredients based upon safety concerns, market share, interest in clinical trials and other factors. Expert Review Panels (ERPs) then meet to discuss the merits and drawbacks of available methods for a particular ingredient. These methods can be culled from the literature or come from in-house methods submitted by industry, government organizations or academic institutions. After extensive review and discussion, the ERP will then select a method (or methods) for that ingredient with which to move forward.
Before moving on to a full collaborative study, it must first be shown that the method can perform satisfactorily in a single laboratory, so it must undergo a "single laboratory validation" (SLV) since the existing validation data often does not meet AOACI's requirements.
Performance Characteristics
When AOACI attempts to validate a method, it measures certain qualities about the method-referred to as "performance characteristics." Performance characteristics are "the functional qualities and statistical measures of the degree of reliability exhibited by the method under specified operating conditions" (AOACI Guidelines for Single Laboratory Validation, 2003). They answer the question "how good is the method?" Performance characteristics are defined by functional qualities and performance limits. The functional qualities of a method include:
• Scope and Applicability
• Selectivity
• Precision (measured as the variability, both intra- and inter-laboratory)
• Reliability (measured as the recovery, often called "accuracy")
• Range
The performance limits are the limits or ranges within which each functional quality measurement must fall for the method to be considered valid. The analyst is then challenged with coming up with a series of experiments to determine each of those functional qualities of the method.
Single Laboratory Validation (SLV)
A properly designed single laboratory validation (SLV) can determine all of these functional qualities except the reproducibility of the method-that is, how reproducible the results are when obtained by different laboratories using the same method on the same sample. Results from SLV apply to a specific operator (or operators) in a specific laboratory using specific equipment. These results provide no information about how a method will perform once it goes outside of the validating laboratory.
AOACI will help to identify a study coordinator and validating laboratory for the SLV. The study coordinator will prepare a validation protocol, which will then be reviewed by the Methods Committee on Dietary Supplements ("Committee K") for technical soundness and conformance to AOACI's SLV requirements. Upon approval of the protocol, the selected laboratory will execute the protocol and submit an SLV manuscript for review. Upon approval of the manuscript by Committee K, it will be published in the Journal of AOACI, and assuming the validation data meet the required performance limits, the method will be recommended for collaborative study.
Collaborative Study
The next phase of method validation is carried out through collaborative study. A collaborative study requires at least eight participating laboratories, and will often involve more-AOACI usually tries to target at least 12 laboratories in case some drop out or there are any "outliers." All Official Methods of Analysis (OMA) of the AOACI have undergone full collaborative studies. Samples are sent out usually as "blind duplicates" to the participating laboratories; that is, the laboratories are not given the identities or the expected concentrations of analyte in the samples in order to avoid potential bias in the results. The laboratories test the samples using the proposed method exactly as written; deviations from the written method can cause the data to be invalidated. The results are sent to the study director, who performs a statistical analysis on the results. If the statistical analysis shows that the reproducibility of the method, expressed as a % Relative Standard Deviation (% RSD) is within an acceptable range, the study director can recommend that method be adopted as "Official First Action." After review by Committee K, and usually two members of the AOACI Official Methods Board, the method can be accepted as Official First Action. The collaborative study manuscript is published in the Journal of AOACI, and the method, along with the appropriate validation data, is published in the Official Methods of Analysis, with the designation of Official First Action.
After two years, if no problems with the method have been reported to the study director, the method can be recommended for Final Action by Committee K to the Official Methods Board. Upon approval by the Official Methods Board, the method is considered final, and then published in the Official Methods of Analysis as such.
If a method has made it this far through the process, a couple of things will become evident: First, there are numerous reviews by various panels, committees and boards of any method that makes through the entire process. The description of the process makes it sound straightforward and linear, when in fact, it is not-revisions, retests, and rewrites are often required. Second, by the time a method has made it through the collaborative study process, it has become the most studied and validated method for that particular analyte in the world.
Status of Validated Methods
Currently, three methods have been accepted as Official First Action for dietary supplements: (1) ephedrine alkaloids in botanicals in dietary supplements; (2) glucosamine in raw materials and dietary supplements; and (3) beta-carotene in raw materials and dietary supplements. A method for the determination of flavonol glycosides in ginkgo biloba raw materials and dietary supplements recently completed the collaborative study phase, and a method for the determination of phytosterols in saw palmetto is currently undergoing a collaborative study. In addition, methods for St. John's Wort, aristolochic acid and terpene lactones in ginkgo biloba are undergoing SLVs. ERPs have met for CoQ10, omega 3 fatty acids, chondroitin sulfate and phenolics.
Summary
When the FDA and NIH-ODS contract was first signed with AOACI, it generally took two years (or more) for a method to go through the full validation process. Now methods can be validated in under a year, although sometimes it can take longer. If questioning why it takes so long for methods to go through the validation process, there are a couple of things to consider. First, these methods are to be used for a wide variety of matrices, such as botanical raw materials, extracts and finished dosage forms, and developing rugged methods and validating them for these products is very challenging. Second, most of the work is carried out by volunteers. In fact, study directors, ERP and committee members, and participating laboratories are all working on a volunteer basis. The fact that so much has already been accomplished by volunteers is a testament to their dedication.
There are still several more methods that need to be developed and validated. Indeed, only the surface has been scratched. AOACI has put out a call for methods for an additional 14 ingredients, including ginseng, echinacea and black cohosh.
The benefits of using an AOAC Official Methods of Analysis to assure product quality and support label claims are manifold. First, FDA recognizes these methods as official, much like it recognizes USP monographs as official for drug testing. Also, U.S. court systems recognize these methods as official in the case of disputes. Because these methods are considered "reference" methods, they can be used to validate in-house methods. Lastly, because all the validation data is published, the amount of variability in results that can be expected between laboratories will be known. In the end, the use of validated Official Methods of Analyses will help level the playing field in the dietary supplement industry, ensure that manufacturers are meeting label claims, help evaluate data from clinical trials performed on dietary supplements, and reduce the discrepancy in results between laboratories.NW
About the author: Mark Roman is chair of AOAC International's "Committee K." He is also the director of Research & Development for Chromadex, Clearwater, FL. He can be reached at 727-573-9229; Fax: 727-573-9220; E-mail: mark.roman@chromadex.com; Website: www.chromadex.com; www.aoac.org.
