National Academies of Sciences: EPA should follow IRIS lead

Last week, the National Academies of Sciences (NAS) said EPA’s Integrated Risk Information System (IRIS) Assessments “can serve as a model for other EPA programs that are implementing systematic review methods.” We commend the IRIS program and urge EPA to use the IRIS method across every office that conducts chemical evaluations so that we can better protect people from harmful chemicals.

EPA’s IRIS program develops science-based assessments on toxicity of chemicals. It is considered the gold standard for chemical toxicity assessments in the US. PRHE has encouraged IRIS and EPA’s Office of Chemical Safety and Pollution Prevention (OCSPP) to strengthen the health protectiveness of their chemical risk evaluations further by incorporating scientifically valid systematic review methods. As the NAS report states, “the IRIS program is clearly helping to advance the science of systematic review, as applied to hazard identification.”

Earlier this year, the NAS’ report on the Toxic Substances Control Act (TSCA) systematic review method recommended that “OCSPP consider components of the OHAT, IRIS, and Navigation Guide methods that could be incorporated directly and specifically into hazard assessment.” OCSPP, however, is yet to reveal what systematic review method it is currently using.

In reviewing the IRIS “handbook” that guides the process of IRIS chemical assessments, last week’s NAS report acknowledged progress but also found that the handbook and IRIS assessments could be improved in several areas consistent with NAS recommendations to EPA on how to improve TSCA systematic review, including:

Publish a method protocol for IRIS assessments prior to conducting the assessments

A pre-published protocol is critical for completing a transparent and unbiased review. It is essentially the recipe for how the assessment will be conducted. The approaches to evidence selection, evaluation, synthesis and integration must be made before reviewing the evidence so that the results of the study do not bias the risk evaluation. The NAS committee recommended that “EPA should create a time-stamped, read-only, final version of each document that details the planned methods for an IRIS assessment prior to conducting the assessment.”

Publishing research methods in advance safeguards against changes being made to these approaches without the reader being aware of it.

Do not exclude studies from an assessment solely on study evaluation ratings

Recent approaches to evaluating a study’s risk of bias – an assessment of the internal validity of a study based on how study design, conduct or analysis can lead to an over- or underestimation of the true result – have included removing a study from an assessment based on the study having only one “critically deficient” limitation. There is no scientific justification for this approach, and it can lead to important evidence being excluded from an assessment.

Data from recent IRIS assessments that eliminated studies based on one “critically deficient” limitation showed that the proportion of human studies excluded from consideration in the IRIS assessment ranged from 0 to 50 percent, and 0 to 41.5 percent for animal studies. These findings were consistent with concerns we raised in a previous study that applied the IRIS risk of bias tool to a sample of studies. The NAS committee stated that “study evaluation ratings should not be used to exclude studies.”

Quantify disease risk level in the entire population at every level of exposure and no longer assume chemicals do not pose harm at low levels of exposure

Current approaches for estimating non-cancer risk (e.g. neurodevelopment, reproductive and cardiovascular disease outcomes) are based on the idea that there is a “safe level” for a chemical. Yet, this “safe level” is not necessarily safe, particularly when considering the true risk level among a diverse population. Thus, the NAS recommends that the level of risk (e.g. how many people per 1,000 are at risk) should be specified so that EPA can better protect the most vulnerable populations, including fenceline communities and communities of color. 

The NAS endorsed the previous recommendations from the NAS to the IRIS program which recommended EPA transition away from “using traditional deterministic approaches for deriving reference values and instead use probabilistic methods to derive risk-specific values”.

Assess the impact of funding bias on the results of an assessment

The NAS committee highlighted that chemical, pharmaceutical and nutrition research have been demonstrated that industry-sponsored studies or authors with a financial conflict of interest (COI) are more likely to have favorable results and conclusions in favor of the sponsor’s product than studies without. This bias must be identified and accounted for when evaluating the results across a body of evidence in EPA risk assessments so that evidence is not inappropriately skewed from showing harm (thus, favorable to industry).  

One of the simplest ways to do this is to include financial COI in the risk of bias tools EPA uses to evaluate individual studies. Importantly, including financial COI as a risk of bias does not lead to the exclusion of industry-sponsored studies, it only means identifying it as potential bias and evaluating its impact on the overall quality of the evidence and result of the assessment.

In response to the NAS recommendations, the IRIS Program stated that it “will incorporate feedback from the report and continue to advance the science of conducting robust and rigorous systematic reviews for human health assessments.” We applaud IRIS for this work and fully support adoption of the IRIS methods across all Agency offices that conduct risk assessments.