EPA method will curtail science used in chemical evaluations

EPA recently released its new TSCA “systematic review” method that establishes how the Agency will use science to make decisions about whether to limit toxic chemicals in our air, water, food and everyday products. Unfortunately, our analysis finds that instead of ensuring a comprehensive, unbiased evaluation, like a systematic review is supposed to do, the TSCA method will severely narrow the science the Agency considers, leading to poor decisions and putting the public’s health at risk.

The method lays out the ‘rules’ EPA will use to find, assemble and interpret scientific evidence on chemical hazards and health effects. What studies EPA considers, and how EPA evaluates those studies can change the final conclusion about whether or not a chemical poses risks. Thus the ‘rules’ have profound implications for public health– especially for populations like pregnant women and children who are more vulnerable to chemical exposures. In particular, a serious concern with the TSCA approach is that it will result in relevant, high-quality studies being excluded from the Agency’s consideration.

Systematic review methods (such as Cochrane) are the standard for evidence evaluation in clinical medicine because they are demonstrated to save lives and money by providing a comprehensive, unbiased evaluation of the scientific evidence. Building from the clinical sciences, there are established, empirically-based methods for systematic review in the environmental health sciences that are validated, peer-reviewed and demonstrated in case studies over the last decade. These include the National Toxicology Program’s OHAT method and the Navigation Guide, developed here at UCSF. Of course we think the Navigation Guide is pretty great, but don’t take our word for it—the National Academy of Sciences (NAS) reviewed both methods and found them to be scientifically robust and exemplary of the type of systematic review method EPA should utilize in chemical evaluations.

What is particularly dismaying about the TSCA method is that it ignores the best practices for systematic review that have already been rigorously tested and validated– a waste of money and time. It is inconsistent with all the current methods for systematic review, which are informed by decades of research and development; the TSCA method is a far departure from OHAT, the Navigation Guide, and the recommended NAS approach.

First, the TSCA method is incomplete- it covers 2 of the 9 required steps, as shown in the diagram. An especially critical missing step is “Develop protocols for systematic review” which must occur prior to identifying and evaluating the evidence to ensure an unbiased review. Further, EPA’s regulation for risk evaluations also requires this—it mandates that the Agency use “a pre-established protocol” to conduct assessments.

Second, the TSCA systematic review method sets up an unscientific ‘scoring’ scheme for evaluating study quality by assigning numerical scores to various study components, such as exposure characterization, and then calculating an overall “quality score.” The NAS, Cochrane methodology, and the academic community have already found based on analysis of empirical data that this type of ‘scoring’ approach does not measure actual study quality, and can lead to biased, inaccurate results. Further, EPA has chosen to score numerous study components that have nothing to do with the quality of the underlying research, such as how completely the authors reported the methods used to carry out the study.

Even worse, the TSCA method then uses this scoring scheme to inappropriately exclude studies based on a single reporting or methodological limitation, saying these studies are ‘unacceptable for use.’ The TSCA method’s arbitrary, unscientific scoring and exclusion of studies is right in line with other attempts to restrict the science EPA relies on, such as the recent “censored science” proposal from former Administrator Pruitt. And, we have a preview of how application of the TSCA method will throw out science—EPA applied the TSCA method in its evaluations of persistent, bioaccumulative and toxic chemicals, resulting in exclusion of almost 500 studies because they “did not meet evaluation criteria.” Excluding relevant studies can only result in a biased evaluation and inaccurate conclusions.

The point of a systematic review is to evaluate the entire body of scientific evidence, not exclude relevant science— no valid systematic review method excludes studies in this way and the TSCA method is far outside the scientific mainstream in this and many other respects. EPA needs to scrap the TSCA method and employ an existing empirically-based method (such as OHAT or Navigation Guide) that has already been tested and approved by the NAS. True systematic reviews are comprehensive, transparent, and unbiased and would help the Agency make the best science-based decisions to ensure protection of the environment and human health.

This post was co-written by Veena Singla, PhD, Associate Director of Science & Policy at PRHE. Public comments on EPA’s TSCA systematic review method were written by Patrice Sutton, MPH, Research Scientist at PRHE.