Over the past few years, the argument over cutting salt from your diet has been heating up. Some research shows too much salt can negatively affect your brain, heart, and bone health. On the other hand, reducing how much salt you eat can be harmful if you’re a heart failure patient. Is cutting back on salt good for you or not? Researchers from Columbia University’s Mailman School of Public Health found the scientific evidence is pretty mixed.

The meta analysis, published in the International Journal of Epidemiology, aimed to analyze the health effects of a salty diet by looking at 269 academic reports — primary studies, systematic reviews, and guidelines, as well as comments, letters, and reviews — published between 1979 and 2014. They recorded which supported cutting back on salt, which were against it, and which were inconclusive. More than half of the reports were published after 2011, which the researcher say shows how hot the salt debate has gotten.

The researchers found that reports on both sides were more likely to cite another report that came to a conclusion similar to their own than they were to cite ones with different conclusions. They also found there were a small number of studies with strong cases made for and against limiting salt intake, and these were among the most cited papers.

"There are two almost distinct bodies of scholarship — one supporting and one opposing the claim that salt reduction in populations will improve clinical outcomes," said David Johns, a doctoral student in Sociomedical Sciences at the Mailman School of Public Health and an affiliate of the Data & Society Research Institute in a press release. "Each is driven by a few prolific authors who tend to cite other researchers who share their point of view, with little apparent collaboration between the two sides."

The researchers found many of the systematic reviews summarizing research to date had little to no consistency in their selection of primary reviews. When multiple studies cite the same primary review, there’s greater certainty as to what counts as evidence regarding cutting salt from our diets. Unfortunately, the researchers found that when a review selected a primary study, the odds of another review selecting that same primary study were less than a third.

The lack of consistency in these reports can lead to difficult decisions for health officials when deciding how to regulate salt intake. "Decision-makers often must choose a course of action in the face of conflicting, uncertain evidence," said coauthor Ludovic Trinquart in the release. "Both the misuse of uncertainty and the exaggeration of certainty can shape the outcomes of public health decision-making processes."

Source: Trinquart L, Johns D, Galea S. Why do we think we know what we know? A metaknowledge analysis of the salt controversy. International Journal of Epidemiology. 2016.