Ancel Keys and the Seven Countries Study: A Response to "The Sugar Conspiracy"
UPDATE: On 8/1/17, the True Health Initiative released a thorough and complete historical assessment of the Seven Countries Study. I worked with co-authors as well as researchers from the ORIGINAL Seven Countries team to put the study, and modern misinterpretations to the test. See my blog post about it here, and read the paper!
A week ago, a wonderfully written, albeit scathing, longform article entitled, “The Sugar Conspiracy” was published in the Guardian by Ian Leslie. It detailed the rise and fall of British scientist, John Yudkin, whose career was unfairly destroyed because he attempted to prove that sugar causes heart disease. Interestingly, the article implies that Dr. Yudkin might not have suffered such intensely negative attention if the United States and the United Kingdom had not been locked into the belief that dietary fat, not sugar, was to blame for chronic cardiac diseases. Furthermore, the proof was right in front of us the entire time, according to Leslie, but we collectively chose to ignore it because of Ancel Keys, an influential nutritionist from the 50s and 60s, and his well-connected cronies.
The narrative of Yudkin’s life is fascinating, and it is no doubt true that Keys eviscerated Yudkin’s theories about sugar in the literature—that’s easy enough to find. Perhaps this was due to Keys feeling threatened by Yudkin’s assertion; perhaps it was the result of a simple superiority complex. But regardless of motivation, Keys was successful at downplaying Yudkin’s research, and theories that saturated fat, not sugar, was related to heart disease became the norm, informing scientific inquiry and policy for the next 50 years, at least.
However, Leslie goes on to argue, Keys’ prominence over Yudkin has done the world a tremendous disservice. Sugar, or carbohydrate depending who you ask, IS in fact the culprit in chronic heart disease. Saturated fat isn’t anywhere near as bad for us, even. According to Leslie, it goes beyond one simple mistake. In fact, he claims that all the dietary guidelines we have today are based on shaky science.
As a science and epidemiology student, I was naturally intrigued. Sure, all the jabs painting nutrition scientists as pretentious idiots stung a little bit, but I am a student not a scientist. If there are major errors and I realize them, I could feasibly influence the next generation of nutritionists and scientists, right?
So I decided to look into the Seven Countries Study and I found a number of occasions where “The Sugar Conspiracy” misinterprets the evidence.
So buckle yourselves up, conspiracy theorists, because in this post, I’m going to cover the history of the diet-heart hypothesis – namely The Seven Countries Study and the subsequent research mentioned in “The Sugar Conspiracy.”
Let’s Talk about Ancel Keys for a Moment… A While
Leslie’s first example illustrating the scope of the heart disease problem comes from 1955, when President Eisenhower suffers his first heart attack. Eisenhower, who had quit smoking in the 40’s, now goes on a low-fat diet and completely eliminates saturated fat and cholesterol from his diet. He continues this diet, Leslie reports, “right up until his death, in 1969, from heart disease.” The fact that Eisenhower dies a full 14 years after his first heart attack at the age of 78 does not suggest that a low-fat diet hastened his demise, even if it didn’t help. Fourteen years isn’t an unusually long time to live after a first heart attack, but it isn’t a swift end. Eisenhower’s age at death was 8 years over the US life expectancy for men in 1969. (1)
But why did Eisenhower’s doctor advise him to undertake such a restrictive diet? It turns out this doctor was a follower of the leading nutrition scientist, Ancel Keys. And Ancel Keys had undertaken what, at the time, was a monumental ethnographic cohort study showing an association between saturated fat and heart disease. You may have heard of it. It’s called “The Seven Countries Study.”
Here is a picture of Ancel Keys. Lots of people do not like him even though his glasses are from Warby Parker.
The Seven Countries Study
From 1958 to 1964, Ancel Keys’ team went to seven countries (surprise!) and collected extensive data on their habits, lifestyles and biomarkers (including serum cholesterol and electrocardiograms). He then correlated that data with heart disease outcomes in a series of regressions—plotting the fat intake in each country against the heart disease deaths of that same country and then assessing how closely heart disease deaths tracked with fat intake. Considering that there were no modern computers, this was an incredible accomplishment of international cooperation and organization. But the question is, are those correlations even worthwhile?
“The Sugar Conspiracy” contends that they weren’t. “There was no objective basis for the countries chosen by Keys, and it is hard to avoid the conclusion that he picked only those he suspected would support his hypothesis. After all, it is quite something to choose seven nations in Europe and leave out France and what was then West Germany, but then, Keys already knew that the French and Germans had relatively low rates of heart disease, despite living on a diet rich in saturated fats,” writes Leslie.
Leslie is insinuating that Keys wasted huge amounts of resources in order to commit what was essentially methodological fraud. Not a small accusation by any stretch of the imagination.
Here is another thing about The Seven Countries study – people have a tendency to report on the study without having looked through it. It’s like a bunch of undergrads faking book reports on Infinite Jest by reading the Wikipedia. “Ugh, DFW, footnotes, amiright?” We get the same shrugging off of Keys' ginormous body of work. But guys, Infinite Jest is a masterpiece.
Did Ancel Keys Use Shoddy Methodology?
So is there a reason for the countries he chose? Because Leslie makes a great point—why leave out France and West Germany?
The answer is pretty boring, and yet very, very common in the field of science: Logistics and budget.
In Keys’ two major publications that came out in 1970, he details the reasoning for countries’ inclusion. Namely, these were the countries that could provide the funding and infrastructure necessary to complete the study. While descriptions of Keys’ work make it sound like he traveled by himself to collect the research, it was actually an enormous international team of academics. Inter-institutional and governmental cooperation was required and Keys’ team needed “reliable census data and well-organized medical systems.” (2)
Additionally, in Keys earlier epidemiological work in Europe, which would have been in the very early 50’s, he had excluded countries that had been recently occupied by Nazis, since food scarcity and rationing during the occupied years could confound health outcomes. At a time when huge multivariate regressions couldn’t be performed automatically by statistical software, this was a real consideration. (3) And, it’s possible this historical context had something to do with excluding West Germany as well.
Also, Keys’ team started collecting data from countries where they had worked before and already had institutional affiliations. “The plan,” Keys stated, “was to organize long-range studies in these countries in which contacts had been established.” These included Italy, Spain, and England (1952) and then in South Africa, Japan and Finland, and the US in 1956 and 57. As time went on, they added sites in Yugoslavia, two more in Italy, Zutphen, the Netherlands, and Greece. It is important to note that the cohorts from each country studied were funded in part by that country. Science takes collaboration and lots and lots of money. (4)
Finland, Finland, Finland!
Thus a country could be eliminated from the data collection if that country didn’t want to fund the research. Spain was not included because of “lack of funds and interested local personnel.” South Africa? “…Great travel expense and the fact that no one seemed to be interested in imitating the ways of the Bantu.” (5) Ouch, Keys, let’s keep the snark to a minimum, okay?
Interestingly, both Greece and the Netherlands were included precisely because it seemed that these populations ate a high fat diet but at once had a low or unknown rate of heart disease. This implies that Keys was at least aware of potential challenges to his theory and arguably willing to investigate them, like any self-respecting scientist.
Keys found rural Greece intriguing, noting that “although nothing was known about the population in regard to blood lipids and very little could be said about heart disease… the reputed large use of olive oil in the diet should be interesting.” And of the Netherlands he remarked, “Mortality ascribed to CHD was very low, although the Dutch were pictured as growing fat on a diet high in saturated fats.”
Leslie is sort of right on one point though. Japan and Finland were included because they represented extremes of heart disease prevalence. In an ethnographic study of this sort, each cohort is plotted individually, so in order to see associations, you need data points that across a wide spectrum. If everyone has the same diet and the same heart disease risk, you’ll see a blob in the middle of your graph. I’m sure if Keys were alive he would probably see it that way too, staring through those hipster glasses.
But, Ze French!?
Ok, ok but wait. Let’s just say that even if Keys didn’t do this on purpose, isn’t leaving out France and West Germany neglectful because they both disprove this theory? Leslie points out that French and Germans had “relatively low rates of heart disease despite living on a diet rich in saturated fats.” (6)
And this is a valid point. It certainly would have been interesting to have this data included. However, the diet was not as “rich” in animal fats as Leslie would lead you to believe. Before 1980, the French still lagged behind the UK and US for animal fat consumption and trumped us in wine consumption, which is frequently posited as an explanation for the lower risk of heart disease in that population. (7) In a 1995 Lancet article, authors noted that “Ecological findings are consistent with a large and growing number of case-control and cohort studies within populations that show a protective effect for CHD morbidity and mortality with light to moderate (1 or 2 drinks per day) intake of alcohol.” (8) Additionally, between 1965 and 1988, percent animal fat consumption in France jumped up 25% and researchers pointed out the “time lag between an increase in fat consumption and its maximal effect on heart disease risk is at least 25 years.” (9)
Anyway, the whole point of that paragraph was to say that while the French certainly do experience the “French Paradox” and if I could go back and redesign the Seven Countries model I personally would include them, the exclusion of French and West German populations doesn’t invalidate the study. Furthermore, the French were eating less fat during the time of the study than they are now, and most of them would have gone through a long period of dietary restriction during the war, which also could have had an independent effect, a fact Keys himself acknowledged.
Also, I watched Bridge of Spies recently and I wouldn’t have wanted to be in any part of Germany right after WWII. Just a fun thought.
So can we conclude, as Leslie so breezily does, that the outcomes of The Seven Countries Study are a deliberate result of fraudulent data cherry picking? There is always the possibility that this is true, but I think not. Even if the exclusion of France and Germany seem convenient, the inclusion of Greece and the Netherlands precisely for their high fat intake seems to disprove the idea that Keys was worried about disproving his hypothesis or engaging in shady methodologies.
Data Quality and the RCT
Ok fine, so Keys wasn’t an evil scientist bent on bugging poor railroad workers about their eating habits just to meet his own evil, attention-grubbing ends. But the data is observational! It can only find correlations and it cannot attribute a cause, as science teachers repeat, exhaustedly, at least forty times a day. And, particularly in the early days of nutrition epidemiology, researchers could not control for many confounding factors – while early studies usually add in smoking and biomarkers, for example, they leave out many things that epidemiological studies usually include today, like physical activity, income, occupation, and education level.
But it is important to think about what we already knew when Keys’ research started:
People with heart disease tended to have higher cholesterol.
People with diabetes or kidney disease tended to have higher cholesterol.
In experimental studies, if you induced high cholesterol in animals, they would develop heart disease.
Atherosclerotic plaques consisted of cholesterol. (10)
Keys’ detractors portray the diet-heart hypothesis as if it simply popped into his head one day and he then proceeded to club everyone with it until we were convinced. Not so fast, haters. The ideas of serum lipids and of fat in the diet being somehow connected with chronic disease had existed for much longer. In fact, by the time he wrote Atherosclerosis, in 1953, there already had been trials testing varying diets on cholesterol levels in adult men. (11) What did these show? That feeding high-fat diets raised cholesterol levels and that reducing fat in the diet reduced cholesterol levels. These basic experiments had been done, giving Keys a base for his theories.
However, at the time of Keys’ initial inquiries, it was still unclear how fat and cholesterol and heart disease might fit together. Before a large scale, long term RCT could reasonably be funded, there would need to be a larger-scale epidemiological study done in order to see if the correlation between cholesterol and heart disease held up across populations. Because while correlation cannot PROVE causation (repeat that five hundred times a day to everyone you know), it would create a solid hypothesis around which to design an RCT (randomized, controlled trial).
So what did the Seven Countries Study do? It went to each country and collected massive amounts of data in various cohorts: resting pulse, blood pressure, cholesterol, dietary intake, weight, physical activity level, cigarette smoking. Then, this data was used to do multivariate regressions to look for associations.
Seven Countries Conclusions
Here is what Ian Leslie said Keys found:
“[The Seven Countries study] showed a correlation between intake of saturated fats and deaths from heart disease, just as Keys had predicted. The scientific debate swung decisively behind the fat hypothesis.” Leslie goes on to imply that because Keys dismissed the idea of sugar as a cause from the outset, he never bothered looking for it in the data.
Here is what Keys concludes, in Seven Countries, the book length 10 year follow up on the study. I’ll do my best to translate this to English (the spacing is added for emphasis).
“When the cohort median values for both systolic blood pressure and for serum cholesterol were taken as independent variables in multiple regression analysis, two-thirds of the cohort variance in coronary death rate are explained.”
“The estimated mean percentage of calories provided by saturated fatty acids in the diets of the cohort was equally successful in explaining the variance in coronary death rate of the cohorts.”
BUT
“Inclusion of this dietary variable in the multiple regression equation with blood pressure and serum cholesterol did not significantly reduce the unexplained variance in coronary death rate.”
WHY?
“This is understandable in view of the very high correlation (r=0.84) between the average values of serum cholesterol and of saturated fats in the diets of the cohorts.”
BUT WHAT ABOUT SUGAR?
“Incidence rates were significantly correlated with sugar and with cholesterol in the diet, but those variables make no significant independent contribution…”
WHY?
“Their relationships with incidence are explained by their high correlation with saturated fats in the diets; r = 0.84 for sucrose, 0.89 for dietary cholesterol.” (12)
Um, what was I supposed to learn from that?
What Keys and his team did was called a multivariate regression or multiple linear regression. If you took statistics in college, you probably did some of these.
Essentially they plot a graph and on the Y-axis they have the outcome they want, in this case it was “coronary death.” On the X-axis, they put the variable they are looking at. In this case, let’s say that variable is percent of calories from dietary fat.
Now, you plot the data for each cohort with their number of coronary heart disease deaths on the Y and the % fat intake on the X.
THEN, you create a line through that data that has the minimum vertical distance between each point and the line. This is your regression line. The data above doesn’t have a regression, because in this part of the book Keys is giving us a “crude correlation,” that is, a correlation unadjusted for confounders like smoking.
The variable on that regression line generates a correlation coefficient. A correlation coefficient tells you how much of the difference in the locations of the dots can be “explained” by the regression line.
As you can see, all the dots don’t line up perfectly – that means there is variation that isn’t explained by the variable on the X axis. So in a multiple linear regression, you add in more variables and see if you can increase the correlation coefficient – so that you can “explain” more of the correlation.
What did Keys’ long statement mean? He noted that most of the difference in heart disease deaths could be explained by blood pressure and serum cholesterol. But what about diet?
Keys also found that individually, saturated fat was highly correlated with heart disease risk, but when he added it to the multiple linear regression model, it didn’t significantly increase the r, or correlation coefficient.
Why is that? Keys noted that because fat increases cholesterol in the blood, the amount of variability explained by serum cholesterol is likely that same amount of variability explained by saturated fat.
Ok, so what about sugar? Well it turned out that total sugar and cholesterol intakes were very highly correlated with saturated fat intake- i.e., the people who ate a lot of sugar also ate a lot of fat. But when you think about most sugary treats, like pastries or ice cream or cake, they tend to be high in both ingredients. Unless a huge percentage of the study population was mainlining Swedish fish (or Snackwells, but they weren’t invented yet), the correlations between sugar and fat would have been totally tied together based on the simple dietary fact that the two ingredients often occur simultaneously.
So why did Keys assert that saturated fat is more significantly connected to the deaths than sugar? This is because there was significant physiological evidence that eating saturated fat increased cholesterol and cholesterol was significantly associated with risk of death from heart disease.
While Leslie is right to say that Keys found an association between saturated fat and heart disease, it’s a bit of an understatement to assume he made some back-of-napkin calculations and then rode his high horse straight to McGovern and demanded to tell us all what to eat. There was a little more math on the front end.
Meanwhile, in Jolly Old England
What about Yudkin, the guy claiming sugar was responsible for heart disease? I should point out that Yudkin’s work was also based on correlations, just like Keys’. However, in the Yudkin paper cited by Leslie, he didn’t even calculate correlation coefficients. (13) Yudkin plotted the figures and used the mathematical technique known as “eyeballing it.” Even if he was 100% right, he did not have higher quality evidence. I’m not positive, but I imagine that this could have led to some of the public critique of his work. It’s possible that, because of his lack of political will and perhaps the pressure he got from Keys and the industry he wasn’t able to secure funding to do more in depth research.
It is also worth pointing out, I think, that in Yudkin’s 1957 paper, he doesn’t even hypothesize that sugar is causing increased heart disease rates. All he says is that there are many associations one can make between dietary and lifestyle factors and disease and he doesn’t think that dietary fat is a good explanation for the increase in heart disease. He concludes that, “Statistical studies such as we have here been considering are also of value, to the extent that they may, if used cautiously, yield useful hints as to possible causes.” Hard to disagree with that! (14)
That same paper contains this fun correlation
But, you might think, this still doesn’t solve the fact that correlation doesn’t equal causation!
This is true.
I don’t want people to think that I disagree with that point – just because you find a great correlation doesn’t mean there is cause there. For many public health issues, it is advisable to act on these correlations anyway if they are strong enough, but that’s another conversation.
I also want to point out that all of this evidence isn’t undeniable proof that Keys was right and we should all go home. I just think it’s incorrect to so easily dismiss his research as biased or his influence as the result of politicizing.
One final caveat should be that there isn’t an “either or” in nutrition. Even if too much saturated fat were bad that doesn’t mean too much sugar isn’t bad or vice versa. Possibly neither is bad if you maintain a healthy weight and don’t smoke. Perhaps they’re both equally bad. I don’t think that someone has to pick a side in order to suggest that Keys’ research is being unfairly maligned.
The Post-Keys Seven Countries Study Analysis
Leslie points out that, “Years later, the Seven Countries study’s lead Italian researcher, Alessandro Menotti, went back to the data, and found that the food that correlated most closely with deaths from heart disease was not saturated fat, but sugar.” (15)
This is false. Menotti et al. did not find this.
In 1999, Menotti wanted to try something new with the data. You probably realize that Ancel Keys originally just looked at percent of calories from fat or total fat intake and not individual foods. Well Menotti et al wanted to look at specific food groups rather than blaming this or that thing on macronutrients or saturated fatty acids. (16)
Then, he did the same type of analysis that was done in the Seven Countries study, except for two key differences:
He looked at individual foods and food groups instead of nutrients
He did NOT add multiple variables into his analysis. They did what is called a “crude correlation.” Namely, there is no adjusting for confounders at all.
So, if you’re worried that the original Seven Countries study didn’t control for confounders, then you won’t like these results because they controlled for nothing. No control. Brakes off!
What did they find? Sugar had a correlation coefficient of 0.6. Was this higher or lower than the correlation coefficient for saturated fat? Quick! Tell me!
Trick question. There was no correlation coefficient for fat because this study looked at foods and food groups, not nutrients. So you cannot possibly conclude that the correlation for sugar was stronger than that of fat.
If you’re interested, the correlation coefficient for meat was 0.645, the coefficient for butter was 0.887, and the correlation for pastries was 0.752. All animal foods together had a correlation coefficient of 0.798 and foods labeled “sweets” (all sugar products and pastries, which are usually high in both fat and sugar) had a correlation coefficient of 0.821.
Even these authors, though, made the similar conclusions to Keys about the food groups. Namely that there are “direct correlations among animal foods and sugar products.” This means that though both sugar and meat are associated with heart disease, they are also highly associated with each other.
Are There Clinical Trials that Prove Causation?
Leslie stated, “In the 30 years after Eisenhower’s heart attack, trial after trial failed to conclusively bear out the association he [Keys] claimed to have identified in the Seven Countries Study.”
Leslie’s statement is a little confusing. Did the trials fail to show a strong association between saturated fat and cholesterol? Is that what he means?
Because that isn’t true at all. They definitely did.
In fact, in a series of controlled human experiments, researchers replaced much of the saturated fat in participants’ diets with polyunsaturated fats. The result? Their cholesterol went down. (17)
The fact that saturated fat raises cholesterol is not debated today. It is a physiological fact. It happens. Saturated fat raises cholesterol.
Now, people will still debate whether high cholesterol actually causes heart disease – but from a population health perspective, it is a pretty good metric as a risk factor. If your nation's LDL cholesterol goes way up, your nation wants to talk to a doctor. If people are really interested in the cholesterol controversy we’d be happy to write more about it! Let us know!
What about that meta-analysis in 2010?
“Another landmark review,” Leslie writes, “authored by, among others, Ronald Krauss, a highly respected researcher and physician a the University of California, stated “there is no significant evidence for concluding that dietary saturated fat is associated with an increased risk of CHD or CVD [coronary heart disease and cardiovascular disease].”
This is a really interesting study; it combined the results of numerous studies and then assessed them all together to look at whether saturated fat was significantly associated with heart disease outcomes. In this study, there truly was no significant result. (18)
But unfortunately, this doesn’t mean what it sounds like it means. “No significant evidence” doesn’t mean there that there is evidence to the contrary. You’ve probably heard the saying, “absence of evidence isn’t evidence of absence” before. This is one of those cases. In science, “significance” means the statistical likelihood that something you observe isn’t just chance. If something is not significant, it doesn’t mean it isn’t there, it just means the effect wasn’t so big you could confidently determine it wasn’t from chance.
I spoke with one of the authors of the Krauss study, Dr. Frank Hu, an epidemiologist at Harvard, and asked him to explain what we should take from the study.
So can one say that this paper shows that saturated fat has no effect on heart disease? According to Dr. Hu, “This is a simplistic interpretation of the data. This meta-analysis and an accompanying article in the same issue of AJCN suggests that compared to carbohydrates, intake of saturated fat is not associated with CHD risk. But the meta-analysis did not compare saturated fat with other macronutrients, such as unsaturated fat. In a recent meta-analysis, we found that replacing saturated fat with PUFA is beneficial for CHD prevention.”
So, at best, it shows that if you replace saturated fat with carbohydrates, there is no benefit. But, if you want to improve outcomes, switch out your saturated fat with unsaturated fat.
Not exactly the bombshell conclusion The Sugar Conspiracy was going for.
In conclusion:
In “The Sugar Conspiracy,” the author makes a lot of assumptions about intent, the usefulness of epidemiology, and even the conclusions of papers. However, upon closer inspection, a lot of his evidence doesn’t hold up under scrutiny.
The idea that Keys claimed in his Seven Countries Study that correlation proved causation is false. Keys just said that cholesterol is a mediator for heart disease and that saturated fat raises cholesterol, both of which later turned out to be true. The Menotti “reanalysis” did not find that sugar is more closely correlated with heart disease than fat, and even if it did, it is a simple regression – it controlled for zero confounders, way fewer than were controlled for in the original Seven Countries Study. Finally, there is plenty of evidence to suggest that substituting saturated fatty foods in favor of unsaturated fats is a good idea.
It is absolutely worthwhile to debate the merits of all scientific findings or even the merits of an entire field, like epidemiology. Scientists, even nutrition scientists, do this all the time. The problem, though, is that if basic facts are actually based on misinformation, you can’t build a real case for or against anything.
If people are interested, Nutrition Wonk can continue with a Part II about the first dietary guidelines.
Notice any mistakes? Email contact@thenutritionwonk.com Thanks!
1. http://www.indexmundi.com/facts/united-states/life-expectancy-at-birth
2. Coronary Heart Disease in Seven Countries, Keys et al., 1970
3. Arteriosclerosis a Problem in Newer Public Health, Ancel Keys, 1953
4. Epidemiological Studies Related to Coronary Heart Disease: Characteristics of Men Aged 40-59 in Seven Countries, Keys et al., 1966
5. Ibid ii
6. The Sugar Conspiracy, Ian Leslie, The Guardian, 2016
7. Why heart Disease mortality is low in France: the time lag explanation, Malcolm Law & Nicholas Wald, BMJ, 1999
8. Does Diet or Alcohol explain the French Paradox?, M H Criqui, Brenda L Ringel, The Lancet 1994
9. Ibid vii
10. Ibid iii
11. Ibid iii
12. Seven Countries: A multivariate analysis of death and coronary heart disease, Keys et al., 1970 Diet and Coronary Thrombosis, John Yudkin, The Lancet, 1957
13. Ibid xiii
14. Ibid vi
15. Food intake patterns and 25-year mortality from coronary heart disease: Cross-cultural correlations in the Seven Countries Study, Menotti et al., European Journal of Epidemiology, 1999
16. Thematic Review Series: The Pathogenesis of Atherosclerosis. An interpretive history of the cholesterol controversy: part II: the early evidence linking hypercholesterolemia to coronary disease in humans, Daniel Steinberg, Journal of Lipid Research, 2004
17. Meta-analysis of prospective cohort studies evaluating the association of saturated fat with cardiovascular disease, Ronald M Krauss et al., The American Journal of Clinical Nutrition, 2010