The Environmental Working Group’s annual Shopper’s Guide to Pesticides in Produce separates fruits and vegetables into groups of produce that are naughty (the “Dirty Dozen”), nice (the “Clean 15”), or somewhere in between. It purports to help families eat a diet rich in fruits and vegetables while avoiding toxic pesticides in produce.
While it is important to reduce consumption of pesticides and other toxins for the sake of both our health and the planet’s health, when I browsed the EWG website for information on how it calculated the risks of the foods named to the “Dirty Dozen” list, I was disappointed, both as a consumer and as a former risk assessment analyst.
The problem is that the method used by the EWG to compile its list of 48 fruits and vegetables is incapable of illuminating the magnitude of risk consumers face by eating fruits and vegetables contaminated by pesticide residues. In reality, the pesticide residues on, say, number seven, sweet peppers, don’t vary greatly from those found on, say, number 19, pears. And yet the list encourages consumers to always buy organic when choosing sweet peppers, but has nothing to say about pears because pears are not in the “Dirty Dozen.”
“It’s extremely arbitrary and ignores the three primary components of risk assessment,” said Dr. Carl K. Winter, a food scientist at the University of California-Davis who studies the risks from pesticide residues and naturally occurring toxins in foods. “The three primary components of risk assessment are: 1) the actual amount of the specific pesticides showing up on each of the specific foods, 2) the amount of those foods that we consume, and 3) the toxicity of the various pesticides—or, put a different way, how much exposure to the pesticides is necessary to make people concerned about them.”
Having myself spent several years of work as a consultant supporting the U.S. Environmental Protection Agency toxic substances programs and as a staff risk assessor for the Virginia Department of Environmental Quality, I have a pretty good idea of what to look for in any statement of risk. I analyzed the 2012 report from EWG to see if it met accepted standards of scientific rigor. The methodology EWG used to produce its 2013 list is unchanged.
The Science Behind the Guide
Traditionally, scientific reports begin with an abstract or brief summary followed by an introduction and sections containing a description of the materials and methods, listing the results, and offering a discussion of those results along with conclusions where appropriate.
The Shopper’s Guide falls short on these items. The Guide does a great job of packaging and marketing its study of pesticide contamination—but all it consists of are two lists, the “Dirty Dozen” and the “Clean 15,” with list members ranked in order of “dirtiness” or “cleanliness,” respectively. You can print it out and fold it in your wallet to take with you to the grocery store. The science behind the lists’ generation, materials, methods, and results appear not in the Shopper’s Guide, but on a separate “Methodology” web page.
What I did not find on the “Methodology” page, however, was a decent discussion and justification of the EWG’s methods. What little I found was at the bottom of the page. Here’s what it says:
“The Shopper's Guide to Pesticides in Produce ranks pesticide contamination on 48 popular fruits and vegetables based on an analysis of more than 28,000 samples taken by the USDA and FDA.”
However, it does not state what U.S. Department of Agriculture or Food and Drug Administration data it used. (In case you are interested, the USDA data can be downloaded from the “Databases and Annual Reports” portion of its Pesticide Data Program website and the FDA data can be obtained from the “Residue Monitoring Reports” section of its food safety website.)
- According to its website, the EWG measured contamination in six ways:
- Percent of samples tested with detectable pesticides
- Percent of samples with two or more detectable pesticides
- Average number of pesticides found on a single sample
- Average amount (in parts per million) of all pesticides found
- Maximum number of pesticides found on a single sample
- Total number of pesticides found on the commodity
For each of its six criteria, EWG then ranked each fruit or vegetable and “normalized” the results on a scale of 1 to 100—those with the highest values on each measurement are set at 100, those with the least are set at 1—and adds the six scaled values to get the final score.
The EWG justifies its approach with the following statement:
“The EWG’s Shopper’s Guide is not built on a complex assessment of pesticide risks but instead reflects the overall pesticide loads of common fruits and vegetables. This approach best captures the uncertainties of the risks of pesticide exposure. Since researchers are constantly developing new insights into how pesticides act on living organisms, no one can say that concentrations of pesticides assumed today to be safe are, in fact, harmless.”
There is a huge problem, however, with EWG’s approach, says Winter—there will always be a Dirty Dozen:
“Regardless of the actual amounts of pesticides that will ever be detected in foods, using this methodology you can always identify your 'Dirty Dozen,’ the top 12 commodities. But that doesn’t tell you anything. If the levels were 10,000 times lower or 10,000 times higher, you’d still just have a ”˜Dirty Dozen.’” My biggest concern is that this type of information—that is very widely and very uncritically disseminated—may result in consumers reducing their consumption of a lot of fruits and vegetables. I think that’s the worst thing consumers can do in terms of their health.”
Winter, who along with his colleague Josh Katz, published a paper in the Journal of Toxicology in 2011 that questioned the scientific validity of the EWG’s approach, said some of the criteria make no sense.
“Average number of pesticides found on a single sample, average number of all pesticides found, maximum number of pesticides found on a single commodity, total number of pesticides found on a commodity—that doesn’t address the risk assessment aspect,” Winter said. “It’s really just presence or absence and nothing else.”
Even the one criterion that is based on the amount of pesticide residues found on the produce is problematic, Winter said, because it doesn’t tell you anything about actual exposure—which is dependent on how much a person actually eats.
“Pesticides can be detected at very, very different levels. They’re used at different levels. They have effectiveness at very different levels. When you look at residues, you might find very, very little amount of the newer chemicals and maybe higher amounts of the other ones; so an average really tells you nothing.”
EWG did not make its researchers available for comment, and its communications staff was defensive to questions regarding their methods. When asked to comment on their methodology, Alex Formuzis, EWG’s Vice President for Media Relations, said, “We've spent enough time answering your questions, so this will be my final comment to you. As we've mentioned to you before, this is not a list based on risk, but rather a comprehensive look at which conventional fruits and vegetables consistently carry the highest and lowest numbers of pesticide residues.” He added that “Many people in the public interest community believe a number of the current tolerances set by EPA under FQPA are not adequate enough to protect young children. Just because levels detected on fruits and veggies are legal, doesn't mean they're safe.”
Organic or Nothing?
On their website, the top of the Shopper’s Guide cautions that the list is not meant to scare people into eating fewer fruits and vegetables if they cannot afford organics, “The health benefits of a diet rich in fruits and vegetables outweigh the risks of pesticide exposure.”
However, it must be asked if whether such a widely distributed list creates a general distrust of fruits and vegetables in the mind of the consumer. Belinda Zeidler, a professor of nutrition at Portland State University notes that “this type of ranking might discourage Americans from eating some very nutrient-dense fruits and vegetables. I buy many of the items on the dirty dozen in their conventional form because I can't always afford the organic version. My bigger concern is the impact of pesticides on farm workers.”
Zeidler advises cleaning your fruits and veggies well: “Food-borne illnesses from pathogens like E. coli and salmonella can be more of an immediate threat than some of these pesticides. Wash your produce well, whether it’s organic, conventional, on the "Dirty Dozen" list, or one of the "Clean 15."
Dirty Apples: How Bad is the Risk?
From the 2013 “Dirty Dozen” list, one might conclude that conventionally grown apples—ranked No. 1 on the naughty list—should be avoided. The question is, how bad is the risk?
To calculate the risk from exposure to a contaminant, a risk assessor must consider the exposure pathway—in this case, ingestion. They must know, or estimate, the concentration of the contaminant in the food or drink consumed, the amount consumed, and the frequency of consumption. Then the risk assessor must specify the population exposed to the contaminant, which in this case would be children and adults. Because children are smaller than adults, they are more vulnerable to toxic effects. Two types of chronic daily intakes are calculated: one to use in estimates of cancer risk, and one to use in estimates of other toxic effects. For cancer risk, in general, a one-in-one-million or greater risk is considered excessive. For other toxic effects, hazard quotient ratios of one or more are considered to be of concern.
When multiple contaminants are present, the cancer risks and hazard quotients are added together. Again, a total cancer risk of at least one-in-one-million or a total hazard quotient of at least one are considered excessive.
I brushed up on my risk assessment skills to determine how bad conventional apples were. To be fair to EWG, I considered only a subset of the data it analyzed. According to Formuzis, the 2012 Shopper’s Guide included apple data collected by the USDA in 2004, 2005, 2009, and 2010. I only looked at the data listed in the USDA’s Pesticide Data Program annual summary for the 2010 calendar year. In that report, Appendix K lists pesticides residues by commodity for combinations in which the specific pesticide was found in at least five percent of samples. Despite the limitations of my brief analysis, I find the results illustrate the weaknesses of EWG’s approach.
I estimated cancer and non-cancer risk for both children and adults for the 21 apple-pesticide combinations listed in the EWG’s 2010 report. In doing so, I used both the maximum measurement for each pesticide and the average (arithmetic mean) measurement for each pesticide. I assumed that the target person would eat one apple (weighing about 150 grams or five ounces) a day.
Of the 21 pesticides detected on more than five percent of apple samples, only one had a slope factor and was evaluated as an actual or potential carcinogen. (There were several others on the list, but the EPA finds that evaluating them on the basis of their non-cancer risks is sufficiently protective of human health, so does not provide a slope factor for them.)
With the average concentrations reported in the USDA’s national samples, none of the individual pesticides had a hazard quotient greater than one. The only way a child could possibly cross the hazard threshold of one would be to eat an apple each day that was contaminated with the average amount of every pesticide detected on all of the apples. Considering that growing seasons and practices, locations, harvests, pesticides used and application methods vary so widely, such an occurance would be a statistical anomaly. Using the maximum concentrations of pesticides detected in the samples, three—diazinon, diphenylamine, and thiabendazole—did have hazard quotients greater than one for children. However, the only way to reach a level of concern of exposure would be for a child to eat an apple every day that was contaminated at that maximum level. Because those maximum-level samples were statistical outliers, it would be highly unlikely. The one pesticide evaluated as a carcinogen did not reach the one-in-one-million threshold of concern in either case of children or adults.
Appropriate Levels of Caution
We can and should do better to reduce the amount of pesticides applied to conventionally grown crops—and EWG should be applauded for its advocacy work in this area—but we also need more and better scientific studies designed to more accurately determine dose-response relationships of potentially toxic substances.
And in the meantime, we need to set appropriate levels of caution while not discouraging people from eating fresh fruits and vegetables. Reducing pesticides and other toxins is a worthwhile goal for any organization; EWG has done a great job of creating a dialog around toxins in our food supply and in other consumer products. However, by creating and aggressively marketing the Shopper’s Guide in ways that seem conceived to maximize its profile and its fundraising (there are multiple donation links from the Shopper’s Guide web page), EWG has created a degree of consumer anxiety around the consumption of fruits and vegetables—especially for those who cannot afford to buy organic.
In the end, as Winter points out, “There’s still no way that EWG can scientifically justify why it is that consumers should only purchase those top 12 fruits and vegetables in organic form, even if the levels were 100,000 times lower than that, they would still say these are the 12 you need to avoid.”
Winter, C.K. and J.M. Katz. 2011. Dietary exposure to pesticide residues from commodities alleged to contain the highest contamination levels. Journal of Toxicology. doi: 10.1155/2011/589674.
Dave Lawrence is a journalist, author, and scientist who has worked for the EPA, the Virginia Department of Environmental Quality, Red River Agricultural Research Station, Woods Hole Oceanographic Institution, and the Sea Education Association. He has also taught biology, geography, meteorology, and ecology at several colleges and universities in the Richmond, VA, area.