Ancel Keys and the Seven Countries Study: A Response to "The Sugar Conspiracy"
UPDATE: On 8/1/17, the True Health Initiative released a thorough and complete historical assessment of the Seven Countries Study. I worked with co-authors as well as researchers from the ORIGINAL Seven Countries team to put the study, and modern misinterpretations to the test. See my blog post about it here, and read the paper!
A week ago, a wonderfully written, albeit scathing, longform article entitled, “The Sugar Conspiracy” was published in the Guardian by Ian Leslie. It detailed the rise and fall of British scientist, John Yudkin, whose career was unfairly destroyed because he attempted to prove that sugar causes heart disease. Interestingly, the article implies that Dr. Yudkin might not have suffered such intensely negative attention if the United States and the United Kingdom had not been locked into the belief that dietary fat, not sugar, was to blame for chronic cardiac diseases. Furthermore, the proof was right in front of us the entire time, according to Leslie, but we collectively chose to ignore it because of Ancel Keys, an influential nutritionist from the 50s and 60s, and his well-connected cronies.
The narrative of Yudkin’s life is fascinating, and it is no doubt true that Keys eviscerated Yudkin’s theories about sugar in the literature—that’s easy enough to find. Perhaps this was due to Keys feeling threatened by Yudkin’s assertion; perhaps it was the result of a simple superiority complex. But regardless of motivation, Keys was successful at downplaying Yudkin’s research, and theories that saturated fat, not sugar, was related to heart disease became the norm, informing scientific inquiry and policy for the next 50 years, at least.
However, Leslie goes on to argue, Keys’ prominence over Yudkin has done the world a tremendous disservice. Sugar, or carbohydrate depending who you ask, IS in fact the culprit in chronic heart disease. Saturated fat isn’t anywhere near as bad for us, even. According to Leslie, it goes beyond one simple mistake. In fact, he claims that all the dietary guidelines we have today are based on shaky science.
As a science and epidemiology student, I was naturally intrigued. Sure, all the jabs painting nutrition scientists as pretentious idiots stung a little bit, but I am a student not a scientist. If there are major errors and I realize them, I could feasibly influence the next generation of nutritionists and scientists, right?
So I decided to look into the Seven Countries Study and I found a number of occasions where “The Sugar Conspiracy” misinterprets the evidence.
So buckle yourselves up, conspiracy theorists, because in this post, I’m going to cover the history of the diet-heart hypothesis – namely The Seven Countries Study and the subsequent research mentioned in “The Sugar Conspiracy.”
Let’s Talk about Ancel Keys for a Moment… A While
Leslie’s first example illustrating the scope of the heart disease problem comes from 1955, when President Eisenhower suffers his first heart attack. Eisenhower, who had quit smoking in the 40’s, now goes on a low-fat diet and completely eliminates saturated fat and cholesterol from his diet. He continues this diet, Leslie reports, “right up until his death, in 1969, from heart disease.” The fact that Eisenhower dies a full 14 years after his first heart attack at the age of 78 does not suggest that a low-fat diet hastened his demise, even if it didn’t help. Fourteen years isn’t an unusually long time to live after a first heart attack, but it isn’t a swift end. Eisenhower’s age at death was 8 years over the US life expectancy for men in 1969. (1)
But why did Eisenhower’s doctor advise him to undertake such a restrictive diet? It turns out this doctor was a follower of the leading nutrition scientist, Ancel Keys. And Ancel Keys had undertaken what, at the time, was a monumental ethnographic cohort study showing an association between saturated fat and heart disease. You may have heard of it. It’s called “The Seven Countries Study.”
Here is a picture of Ancel Keys. Lots of people do not like him even though his glasses are from Warby Parker.
The Seven Countries Study
From 1958 to 1964, Ancel Keys’ team went to seven countries (surprise!) and collected extensive data on their habits, lifestyles and biomarkers (including serum cholesterol and electrocardiograms). He then correlated that data with heart disease outcomes in a series of regressions—plotting the fat intake in each country against the heart disease deaths of that same country and then assessing how closely heart disease deaths tracked with fat intake. Considering that there were no modern computers, this was an incredible accomplishment of international cooperation and organization. But the question is, are those correlations even worthwhile?
“The Sugar Conspiracy” contends that they weren’t. “There was no objective basis for the countries chosen by Keys, and it is hard to avoid the conclusion that he picked only those he suspected would support his hypothesis. After all, it is quite something to choose seven nations in Europe and leave out France and what was then West Germany, but then, Keys already knew that the French and Germans had relatively low rates of heart disease, despite living on a diet rich in saturated fats,” writes Leslie.
Leslie is insinuating that Keys wasted huge amounts of resources in order to commit what was essentially methodological fraud. Not a small accusation by any stretch of the imagination.
Here is another thing about The Seven Countries study – people have a tendency to report on the study without having looked through it. It’s like a bunch of undergrads faking book reports on Infinite Jest by reading the Wikipedia. “Ugh, DFW, footnotes, amiright?” We get the same shrugging off of Keys' ginormous body of work. But guys, Infinite Jest is a masterpiece.
Did Ancel Keys Use Shoddy Methodology?
So is there a reason for the countries he chose? Because Leslie makes a great point—why leave out France and West Germany?
The answer is pretty boring, and yet very, very common in the field of science: Logistics and budget.
In Keys’ two major publications that came out in 1970, he details the reasoning for countries’ inclusion. Namely, these were the countries that could provide the funding and infrastructure necessary to complete the study. While descriptions of Keys’ work make it sound like he traveled by himself to collect the research, it was actually an enormous international team of academics. Inter-institutional and governmental cooperation was required and Keys’ team needed “reliable census data and well-organized medical systems.” (2)
Additionally, in Keys earlier epidemiological work in Europe, which would have been in the very early 50’s, he had excluded countries that had been recently occupied by Nazis, since food scarcity and rationing during the occupied years could confound health outcomes. At a time when huge multivariate regressions couldn’t be performed automatically by statistical software, this was a real consideration. (3) And, it’s possible this historical context had something to do with excluding West Germany as well.
Also, Keys’ team started collecting data from countries where they had worked before and already had institutional affiliations. “The plan,” Keys stated, “was to organize long-range studies in these countries in which contacts had been established.” These included Italy, Spain, and England (1952) and then in South Africa, Japan and Finland, and the US in 1956 and 57. As time went on, they added sites in Yugoslavia, two more in Italy, Zutphen, the Netherlands, and Greece. It is important to note that the cohorts from each country studied were funded in part by that country. Science takes collaboration and lots and lots of money. (4)
Finland, Finland, Finland!
Thus a country could be eliminated from the data collection if that country didn’t want to fund the research. Spain was not included because of “lack of funds and interested local personnel.” South Africa? “…Great travel expense and the fact that no one seemed to be interested in imitating the ways of the Bantu.” (5) Ouch, Keys, let’s keep the snark to a minimum, okay?
Interestingly, both Greece and the Netherlands were included precisely because it seemed that these populations ate a high fat diet but at once had a low or unknown rate of heart disease. This implies that Keys was at least aware of potential challenges to his theory and arguably willing to investigate them, like any self-respecting scientist.
Keys found rural Greece intriguing, noting that “although nothing was known about the population in regard to blood lipids and very little could be said about heart disease… the reputed large use of olive oil in the diet should be interesting.” And of the Netherlands he remarked, “Mortality ascribed to CHD was very low, although the Dutch were pictured as growing fat on a diet high in saturated fats.”
Leslie is sort of right on one point though. Japan and Finland were included because they represented extremes of heart disease prevalence. In an ethnographic study of this sort, each cohort is plotted individually, so in order to see associations, you need data points that across a wide spectrum. If everyone has the same diet and the same heart disease risk, you’ll see a blob in the middle of your graph. I’m sure if Keys were alive he would probably see it that way too, staring through those hipster glasses.
But, Ze French!?
Ok, ok but wait. Let’s just say that even if Keys didn’t do this on purpose, isn’t leaving out France and West Germany neglectful because they both disprove this theory? Leslie points out that French and Germans had “relatively low rates of heart disease despite living on a diet rich in saturated fats.” (6)
And this is a valid point. It certainly would have been interesting to have this data included. However, the diet was not as “rich” in animal fats as Leslie would lead you to believe. Before 1980, the French still lagged behind the UK and US for animal fat consumption and trumped us in wine consumption, which is frequently posited as an explanation for the lower risk of heart disease in that population. (7) In a 1995 Lancet article, authors noted that “Ecological findings are consistent with a large and growing number of case-control and cohort studies within populations that show a protective effect for CHD morbidity and mortality with light to moderate (1 or 2 drinks per day) intake of alcohol.” (8) Additionally, between 1965 and 1988, percent animal fat consumption in France jumped up 25% and researchers pointed out the “time lag between an increase in fat consumption and its maximal effect on heart disease risk is at least 25 years.” (9)
Anyway, the whole point of that paragraph was to say that while the French certainly do experience the “French Paradox” and if I could go back and redesign the Seven Countries model I personally would include them, the exclusion of French and West German populations doesn’t invalidate the study. Furthermore, the French were eating less fat during the time of the study than they are now, and most of them would have gone through a long period of dietary restriction during the war, which also could have had an independent effect, a fact Keys himself acknowledged.
Also, I watched Bridge of Spies recently and I wouldn’t have wanted to be in any part of Germany right after WWII. Just a fun thought.
So can we conclude, as Leslie so breezily does, that the outcomes of The Seven Countries Study are a deliberate result of fraudulent data cherry picking? There is always the possibility that this is true, but I think not. Even if the exclusion of France and Germany seem convenient, the inclusion of Greece and the Netherlands precisely for their high fat intake seems to disprove the idea that Keys was worried about disproving his hypothesis or engaging in shady methodologies.
Data Quality and the RCT
Ok fine, so Keys wasn’t an evil scientist bent on bugging poor railroad workers about their eating habits just to meet his own evil, attention-grubbing ends. But the data is observational! It can only find correlations and it cannot attribute a cause, as science teachers repeat, exhaustedly, at least forty times a day. And, particularly in the early days of nutrition epidemiology, researchers could not control for many confounding factors – while early studies usually add in smoking and biomarkers, for example, they leave out many things that epidemiological studies usually include today, like physical activity, income, occupation, and education level.
But it is important to think about what we already knew when Keys’ research started:
People with heart disease tended to have higher cholesterol.
People with diabetes or kidney disease tended to have higher cholesterol.
In experimental studies, if you induced high cholesterol in animals, they would develop heart disease.
Atherosclerotic plaques consisted of cholesterol. (10)
Keys’ detractors portray the diet-heart hypothesis as if it simply popped into his head one day and he then proceeded to club everyone with it until we were convinced. Not so fast, haters. The ideas of serum lipids and of fat in the diet being somehow connected with chronic disease had existed for much longer. In fact, by the time he wrote Atherosclerosis, in 1953, there already had been trials testing varying diets on cholesterol levels in adult men. (11) What did these show? That feeding high-fat diets raised cholesterol levels and that reducing fat in the diet reduced cholesterol levels. These basic experiments had been done, giving Keys a base for his theories.
However, at the time of Keys’ initial inquiries, it was still unclear how fat and cholesterol and heart disease might fit together. Before a large scale, long term RCT could reasonably be funded, there would need to be a larger-scale epidemiological study done in order to see if the correlation between cholesterol and heart disease held up across populations. Because while correlation cannot PROVE causation (repeat that five hundred times a day to everyone you know), it would create a solid hypothesis around which to design an RCT (randomized, controlled trial).
So what did the Seven Countries Study do? It went to each country and collected massive amounts of data in various cohorts: resting pulse, blood pressure, cholesterol, dietary intake, weight, physical activity level, cigarette smoking. Then, this data was used to do multivariate regressions to look for associations.
Seven Countries Conclusions
Here is what Ian Leslie said Keys found:
“[The Seven Countries study] showed a correlation between intake of saturated fats and deaths from heart disease, just as Keys had predicted. The scientific debate swung decisively behind the fat hypothesis.” Leslie goes on to imply that because Keys dismissed the idea of sugar as a cause from the outset, he never bothered looking for it in the data.
Here is what Keys concludes, in Seven Countries, the book length 10 year follow up on the study. I’ll do my best to translate this to English (the spacing is added for emphasis).
“When the cohort median values for both systolic blood pressure and for serum cholesterol were taken as independent variables in multiple regression analysis, two-thirds of the cohort variance in coronary death rate are explained.”
“The estimated mean percentage of calories provided by saturated fatty acids in the diets of the cohort was equally successful in explaining the variance in coronary death rate of the cohorts.”
“Inclusion of this dietary variable in the multiple regression equation with blood pressure and serum cholesterol did not significantly reduce the unexplained variance in coronary death rate.”
“This is understandable in view of the very high correlation (r=0.84) between the average values of serum cholesterol and of saturated fats in the diets of the cohorts.”
BUT WHAT ABOUT SUGAR?
“Incidence rates were significantly correlated with sugar and with cholesterol in the diet, but those variables make no significant independent contribution…”
“Their relationships with incidence are explained by their high correlation with saturated fats in the diets; r = 0.84 for sucrose, 0.89 for dietary cholesterol.” (12)
Um, what was I supposed to learn from that?
What Keys and his team did was called a multivariate regression or multiple linear regression. If you took statistics in college, you probably did some of these.
Essentially they plot a graph and on the Y-axis they have the outcome they want, in this case it was “coronary death.” On the X-axis, they put the variable they are looking at. In this case, let’s say that variable is percent of calories from dietary fat.
Now, you plot the data for each cohort with their number of coronary heart disease deaths on the Y and the % fat intake on the X.
THEN, you create a line through that data that has the minimum vertical distance between each point and the line. This is your regression line. The data above doesn’t have a regression, because in this part of the book Keys is giving us a “crude correlation,” that is, a correlation unadjusted for confounders like smoking.
The variable on that regression line generates a correlation coefficient. A correlation coefficient tells you how much of the difference in the locations of the dots can be “explained” by the regression line.
As you can see, all the dots don’t line up perfectly – that means there is variation that isn’t explained by the variable on the X axis. So in a multiple linear regression, you add in more variables and see if you can increase the correlation coefficient – so that you can “explain” more of the correlation.
What did Keys’ long statement mean? He noted that most of the difference in heart disease deaths could be explained by blood pressure and serum cholesterol. But what about diet?
Keys also found that individually, saturated fat was highly correlated with heart disease risk, but when he added it to the multiple linear regression model, it didn’t significantly increase the r, or correlation coefficient.
Why is that? Keys noted that because fat increases cholesterol in the blood, the amount of variability explained by serum cholesterol is likely that same amount of variability explained by saturated fat.
Ok, so what about sugar? Well it turned out that total sugar and cholesterol intakes were very highly correlated with saturated fat intake- i.e., the people who ate a lot of sugar also ate a lot of fat. But when you think about most sugary treats, like pastries or ice cream or cake, they tend to be high in both ingredients. Unless a huge percentage of the study population was mainlining Swedish fish (or Snackwells, but they weren’t invented yet), the correlations between sugar and fat would have been totally tied together based on the simple dietary fact that the two ingredients often occur simultaneously.
So why did Keys assert that saturated fat is more significantly connected to the deaths than sugar? This is because there was significant physiological evidence that eating saturated fat increased cholesterol and cholesterol was significantly associated with risk of death from heart disease.
While Leslie is right to say that Keys found an association between saturated fat and heart disease, it’s a bit of an understatement to assume he made some back-of-napkin calculations and then rode his high horse straight to McGovern and demanded to tell us all what to eat. There was a little more math on the front end.
Meanwhile, in Jolly Old England
What about Yudkin, the guy claiming sugar was responsible for heart disease? I should point out that Yudkin’s work was also based on correlations, just like Keys’. However, in the Yudkin paper cited by Leslie, he didn’t even calculate correlation coefficients. (13) Yudkin plotted the figures and used the mathematical technique known as “eyeballing it.” Even if he was 100% right, he did not have higher quality evidence. I’m not positive, but I imagine that this could have led to some of the public critique of his work. It’s possible that, because of his lack of political will and perhaps the pressure he got from Keys and the industry he wasn’t able to secure funding to do more in depth research.
It is also worth pointing out, I think, that in Yudkin’s 1957 paper, he doesn’t even hypothesize that sugar is causing increased heart disease rates. All he says is that there are many associations one can make between dietary and lifestyle factors and disease and he doesn’t think that dietary fat is a good explanation for the increase in heart disease. He concludes that, “Statistical studies such as we have here been considering are also of value, to the extent that they may, if used cautiously, yield useful hints as to possible causes.” Hard to disagree with that! (14)
That same paper contains this fun correlation
But, you might think, this still doesn’t solve the fact that correlation doesn’t equal causation!