It’s not uncommon to hear people poke fun at their fathers for forgetting things. They’ll ask the same question two or three times, call your friend Lindsey something far from it, and lose their keys just to realize it was left inside the lock. While forgetfulness is a normal part of aging, a new study finds men might be on the faster track to it than women.

Researchers from the Mayo Clinic found age-related memory loss affected men at a faster pace than women, and that when it happened, the volume of the brain’s memory center, the hippocampus, went with it. More importantly, though, the memory loss was unrelated to having a gene — APOE4 — linked to the development of the characteristic amyloid deposits in Alzheimer’s patients. This particular finding refutes a common belief in the medical community that memory lapses among older adults can be early signs of Alzheimer’s.

Instead, Dr. Clifford Jack, lead author of the study, says the amyloid deposits may be after the fact. “There seems to a profound effect of aging, itself, on memory — independent of amyloid,” he told HealthDay. “We think that [amyloid] pathology tends to arise late in life, to accelerate a pre-existing decline in memory.”

If this is the case, then it’s good news for people who experience memory decline as they age, as it won’t always mean they’re on the path toward full-blown Alzheimer’s. Memory loss with the development of dementia or Alzheimer’s is much different. According to Mayo Clinic, this kind of memory loss goes beyond simply forgetting someone’s name or forgetting where the keys are to include forgetting common words while speaking, taking longer to complete familiar tasks, and misplacing items in inappropriate places.

For the study, the researchers had 1,200 adults between the ages of 30 and 95 from a single county in Minnesota take memory tests and undergo brain scans — while an MRI looked at hippocampal volume and a PET scan looked for amyloid deposits. They found that memory and brain volume both declined between the ages of 30 and 95, despite little evidence of amyloid plaques growing. Among men, overall memory became worse than women's at around 40, and around 60, their hippocampal volume became lower than women's. It wasn’t until about 70 years old that many of these people began developing amyloid plaques; those with the APOE4 gene, who seemed to develop them at an earlier age, also experienced a sudden increase in their development around 70.

“Understanding the basic biology of these early processes are likely to substantially inform us about ways in which we can maintain cognitive health and optimize resistance to late-life dementia,” said Dr. Charles DeCarli, of the University of California at Davis, Sacramento, in an editorial about the study. “Establishing what is normal creates avenues for new research, increasing the likelihood of discovering novel therapeutics for late-life disease states, which is a laudable goal indeed.”

Source: Jack C, Wiste H, Weigand S, et al. Age, Sex, and APOE ε4 Effects on Memory, Brain Structure, and β-Amyloid Across the Adult Life Span. JAMA Neurology. 2015.

DeCarli C. A Call for New Thoughts About What Might Influence Human Brain Aging: Aging, Apolipoprotein E, and Amyloid. JAMA Neurology. 2015.