For years, the standard rule of dieting stated that you must consume fewer calories than you burn (3,500 to be exact) in order to lose one pound. A new study adds to the mounting research that weight loss is a more complex subject, and that sugar substitutes can actually hinder weight loss efforts.

“A calorie is not a calorie,” senior study author Dana Small, professor of psychiatry at Yale University School of Medicine, said in a university blog post.

According to a study conducted at Yale, the problem begins when something is too sweet or not sweet enough to account for the calories being consumed. This disrupts the metabolic response as well as the communication to your brain about the nutritional makeup that can be expected from your food and drinks. Something that tastes sweet but has few calories, like diet sodas, will increase the metabolic response. When the sweet taste matches up with calories, the calories are then metabolized, thus satisfying the area in your brain associated with reward. But when your body is expecting to get a surge of calories because it was tricked by the sugary taste, your brain is disappointed and doesn’t register the calories consumed.

“In other words, the assumption that more calories trigger greater metabolic and brain response is wrong,” Small said. “Calories are only half of the equation; sweet taste perception is the other half.”

Diet sodas often get a lot of attention as being “bad,” but sugar substitutes are in many other processed foods.

“Our bodies evolved to efficiently use the energy sources available in nature,” Small said. “Our modern food environment is characterized by energy sources our bodies have never seen before.”

Gum, candy, baked goods, juice, ice cream and yogurt are all common sources of artificial sugar.

Substitutes like Stevia and Aspartame have been approved by the Food and Drug Administration, but that hasn’t stopped the safety debate surrounding these faux sugars. According to WebMD, much of the criticism began after scientific studies from the 1970s associated the sweetener saccharin to bladder cancer in rats. However, the National Cancer Institute has determined that there is not enough evidence to conclude that any sweetener deemed safe by the FDA can cause cancer, writes WebMD.

Fox News explored the murky world of sugar substitutes, and wrote that saccharin was an accidental discovery after a researcher tasted something sweet on his hands while searching for new ways to use coal tar derivatives. This led to finding the key ingredient of benzoic sulfimide, which was responsible for the sweet substance he discovered.

In the early 1900s, saccharine was found in many foods. The head of the United States Department of Agriculture’s chemical department actually tried to get the substance banned, believing it could be toxic, to no success. While it was no longer allowed to be used beginning in 1912, WWI ended the the ban as sugar was rationed and people needed another option.

After saccharine came cyclamate, which you likely never heard of because it was proven to cause cancer in lab rats, resulting in a ban from the FDA since the 1970s. Following that was aspartame, and then sucralose, the main ingredient in Splenda.