4 Research Skills That Explain Why Not to Panic About Arsenic in Baby Formula

by ParentCo. January 05, 2018

The Clean Label Project recently released a study of more than 500 infant formulas and baby foods representing 60 brands. The results, which the CLP published as an infographic, look terrifying:

• 65 percent of baby food and 80 percent of formula contained arsenic.
• 36 percent of baby food contained lead.
• 58 percent of baby food contained cadmium.

Myth-busting Snopes was on the case almost as soon as the CLP published its results. The CLP did not publish its findings in a peer-reviewed journal. It did not provide any data to support its conclusions. It did not disclose any conflicts of interest. The Snopes investigation also identified previous controversies with Ellipse Labs, the company that did the product testing for CLP. This is not the first time critics have addressed methodological problems and conflicts of interest at the CLP. When the organization published results about pet foods earlier this year, its methods drew a great deal of criticism, including a lively Reddit Ask Me Anything in which two of the researchers answered very little. All of these are reasons enough not to trust the CLP's findings about baby food and infant formula. But that debunking isn't necessarily helpful to parents who want to avoid being taken in by the next headline-grabbing "study" of a baby danger. Here are four research skills to help you digest the CLP's study of baby food and prepare yourself to be a more critical reader of the next big baby scare.

Just because something sounds dangerous doesn't mean it is dangerous

The CLP finding you're most likely to hear about in the news is the shocking conclusion that eighty percent of infant formulas contain arsenic. That's bound to stick with you, because arsenic is probably the most famous poison. If you know arsenic to be poisonous, you were probably shocked to learn that the food you're feeding your baby contains it. The problem here is that many foods contain arsenic, especially rice, which just so happens to be a component of many baby foods. It's also present in another kid favorite, juice.

The CLP's arsenic finding is one of many variations on the same rhetorical strategy: name a scary-sounding ingredient, identify a non-food purpose for that ingredient, and voila! You've made viral news. It's the same approach that has worked for the Food Babe countless times... Subway is feeding you yoga mats! Macaroni and Cheese is stuffing your kids with the same stuff you use to power your car! The CLP benefits from our collective chemophobia, such that it can merely mention the name of a scary-sounding chemical in order to stoke fear. Take, for example, the CLP's finding that 10 percent of its samples tested positive for acrylamide, which, according to its infographic, "is a chemical created during manufacturing linked to brain damage, cancer, and reproductive harm." Let's take those claims one at a time. Although it's technically true that acrylamide would be created during the "manufacturing" process of some baby foods, that's only true because "cooking" is part of the manufacturing process.

Acrylamide is used in the manufacture of paper, ink, and other materials, but that's not the use we're talking about in baby food. Acrylamide is created during the cooking of starchy foods like french fries and potato chips, as well as some types of crackers. Although the CLP does not name the products that tested positive for acrylamide, it's reasonable to assume that they were fried or otherwise cooked with very high heat: teething biscuits, puffs, and other crispy snacks. The second claim, that acrylamide has been "linked to brain damage, cancer, and reproductive harm," is another one of those true but not true claims. Acrylamide was unfairly maligned as a cancer-causing agent in the early 2000s, but it was found to be a natural part of the cooking process not necessarily linked with human cancers. Takeaway: If a chemical sounds scary, find out all of its uses before deciding it is scary.

Beware of inappropriate comparisons

Let's return to arsenic for a moment. It may be true that 80 percent of baby foods tested by the CLP contained arsenic. That finding, however, is not evidence that these baby foods pose danger to children. The CLP did not compare its results to any arsenic level standard. Instead, it tested for the presence of arsenic and compared products to each other. Because arsenic is naturally present in many foods, it's reasonable that some products tested positive for arsenic, and that some would have higher levels than others. Imagine Baby Cereal A and Baby Cereal B. Cereal A was found to have 50 parts per billion (ppb) of arsenic. Cereal B was found to have 100 ppb of arsenic. Is Cereal B dangerous? Not according to the FDA, which recently proposed a limit of 100 ppb for all rice-containing baby cereals. The CLP, however, would call Cereal B more dangerous, because it places products on a spectrum. Those with less detectable arsenic are "better" than the ones with more detectable arsenic, even if all of the products tested meet the FDA guidelines. The CLP could also claim that the "worse" cereal has 100 percent more arsenic than the better cereal, which makes that cereal sound absolutely terrifying. Takeaway: When you see scary numbers in the news, look for the comparisons. Beware of dangerous items compared to each other instead of to a standard.

Always search for a methods section

The CLP describes its "unique" methods for obtaining its data as follows: "We do not make assumptions for product recommendations based on manufacturer supplied data, peer-reviewed research reports, or data from other consumer advocacy groups." Assuming that "make assumptions for product recommendations" means something like "we don't allow the following sources to influence our product recommendations," it's both reasonable and ethical that neither manufacturer-supplied data nor consumer advocacy group data was included in the study. Wedged in between those two groups, however, is "peer-reviewed research reports." The entire notion of scientific knowledge is that it is built, piece by piece, upon the work of previous science. If you aren't identifying yourself within a particular field, and you're not building your work on the publications of others in that field, you're not doing scientific research. It's possible for a research method to be "unique," but the methods section of your study should not be. Researchers include detailed methods sections in their research so that other researchers can replicate those findings. That replication is essential to demonstrating a phenomenon actually exists. Takeaway: If you can't identify the methods the researchers used, you can't reproduce their results. If you can't reproduce their results, it's not scientific research.

Ask if the researchers are trying to sell you something

The most concerning issue here is that the CLP is telling parents what to buy. There is nothing wrong with consumer-advocacy groups recommending one product over another. There is nothing wrong with groups like Consumer Reports or even blogs like The Sweet Home offering detailed reviews of their products. There is a problem when a group claiming to be doing "independent" research is profiting directly off of the results of its research. The CLP's website features "buy now" links to all the products included in its reviews. It's possible that those links represent an undisclosed conflict of interest. The CLP could be a part of Amazon's Affiliate marketing program. If so, the CLP would earn four percent from the Amazon sales of all its recommended products. The CLP happens to recommend more expensive brands more highly, therefore, with each click of a CLP five-star product, they would be earning more than they would if they had highly rated a cheaper product. Even if the CLP is not an Amazon affiliate, it has not disclosed sources of funding or possible conflicts of interest, which has been a source of controversy for the organization before. Takeaway: If the same people conducting the research are also trying to sell you something, be suspicious. If they aren't telling you how they earn their money, be even more suspicious.

---

---

Also in Conversations