Charismatic Paleo Bloggers: Rigorous Caution Required

The peer review process

The difference between charismatic bloggers and published research scientists is that the latter must present their ideas, work and experiments before a panel of scientific peers prior to publication. The peer review process certainly is not infallible and clearly does not always insure the accuracy, generalizability or validity of any experiment or idea. Nevertheless, it generally does insure that the paper or concept has been examined by a panel of scientists and experts who usually are trained in the discipline, but who also are typically trained in universal research design and statistical concepts and procedures, without which experiments and data are meaningless and un-interpretable.

Almost universally, charismatic bloggers have little or no understanding of how research design and statistical issues can make or break the interpretation of any experiment or hypothesis, yet as I will show you they proudly offer their opinions regardless.

Research design and statistics

I graduated from the University of Utah in 1981 with a Ph.D. in Health Sciences (emphasis: Exercise Physiology). Besides course work, one of the requirements for the Ph.D. was the successful completion of an experimental project and the subsequent write-up of this research via a Doctoral Dissertation. During my years of coursework, I took almost two quarters worth of specific graduate level classes that focused upon 1) a variety of statistical procedures, 2) research design issues, and 3) computer assisted data compilation and interpretation. Whew! These classes were not fun, and I struggled to get through some of them. But without the knowledge and experience I gained from these classes, I wouldn’t have had a clue about designing, statistically analyzing, and writing up the physiological/respiratory experiment that eventually became my Doctoral Dissertation.

When I finally completed my two year long experiment, wrote the dissertation and finally graduated, I breathed a sigh of relief in the mistaken belief that I would never again have to go through this ordeal. Wrong! At the time, little did I realize the research design, statistical and computer skills I had utilized for my Ph.D. project would never leave me, and that I would have to repeat this process again, again and again on a regular basis for the next 32 years.

It is sometimes said that the best way to better learn about any topic or skill is to have to teach it to others. As a rookie, Assistant Professor at Colorado State University in the fall of 1981, I was immediately assigned to teach a graduate course in Research Design and Statistics to both Master’s and Ph.D. students. As it turned out, I would go on to teach this course for the next 32 years, but more importantly I continually honed my research design and statistical tools not only for my own research, but also as I taught my graduate students to implement their research projects. Increasingly as my career developed I fully appreciated the magnitude of these powerful scientific tools as I served as a reviewer for scientific journal articles and governmentally sponsored grants.

“By using it, you will not lose it,” or so goes the truism. In the case of research design and statistics, almost all charismatic bloggers, never learned these scientific tools in the first place, so their Paleo diet interpretations of the scientific literature and subsequent subjective pronouncements need to be rigorously evaluated if we are to place any credence whatsoever upon their writings.

Below are just a few key questions almost any scientist familiar with research design and statistical procedures would be able to answer. I suspect that none of our charismatic Paleo bloggers whose names you all recognize would be able to answer any of these questions off the top of their heads. Familiarity with these concepts is essential in correctly interpreting and fully understanding the scientific literature.

Questions

1. What is statistical power and how does it influence hypothesis testing?
2. What is the null hypothesis. Can it be answered in either the affirmative or negative or only singly and why does it matter?
3. What is a two-tailed statistical test? How does it affect alpha and subsequently hypothesis testing?
4. What is the relationship between alpha (a type 1 statistical error) and beta (a type 2 statistical error) and how does sample size (n) interact with these concepts to affect hypothesis testing?
5. Why is sample size crucial when evaluating the internal and external validity of an experiment?
6. What are the four levels of data and how does this information influence the type of statistic to be employed in the analysis and why?
7. What are the differences among 1) pre-experimental, 2) quasi experimental and 3) true experimental designs. How do these considerations influence internal validity and generalizability of the experiment?
8. When is a repeated measures ANOVA used to analyze data and why should multiple t-tests not be used in making repeated comparisons?
9. What are the differences between parametric and non-parametric statistical tests and how does the level of data influence their choice?
10. Do descriptive statistics show causality? How about inferential statistics? What are common differences between the two?
11. Is it possible to generate a standard deviation greater than the mean? How are large standard deviations generally interpreted with small sample sizes? How about with large sample (n) sizes?
12. When should the standard error of the mean (SEM) be employed in lieu of the standard deviation?
13. With the inclusion of more and more variables into a forward, stepwise multiple regression equation, what is the effect upon “R”; what is the effect upon “p”. Why does this matter?
14. How does the use of partial correlation techniques help to unravel relationships among a series of variables?

OK, OK – ENOUGH! You get my point; we could go on endlessly with these obscure statistical and research design concepts. For most of you, not only can you not answer these questions, the answers are irrelevant anyway.

What you want to find out from your charismatic blogger is a simple answer to a simple question – should I drink milk or not? How about kefir? Should I regularly consume legumes and beans? How about sea salt – is it OK? Do contemporary Paleo diets require supplements?

I’ll give you some insight into your charismatic blogger – off the top of their heads, without the input of skilled professionals, they could not answer these research design and statistical questions either – they simply lack the training. Like you, they are barely even familiar with these terms and concepts known to most research scientists.

Without the knowledge or understanding of research design and statistical notions our charismatic, influential Paleo bloggers simply cannot understand the subtleties, limitations and flaws in the scientific papers they may read. Accordingly, their advice and pronouncements about a variety of Paleo Diet issues are at best incomplete, and at worst flat out wrong.

Cause and effect

One of the challenges faced by nutritional scientists when they ultimately make recommendations regarding what we should and should not eat is to establish cause and effect between a dietary element and the subsequent development or prevention of disease. Some foods and some dietary habits promote good health whereas others promote disease. Figure 1 demonstrates the four primary procedures by which causality is established between diet and disease.¹

Figure 1. The four primary procedures by which causality is established between diet and disease.^{1, 2}

No single procedure alone can establish cause and effect,^{1, 2} nor can any single study prove causality.³ Observational epidemiological studies can only show relationships among variables and are notorious for showing conflicting results⁴ and cannot provide decisive evidence by themselves either for or against specific hypotheses.⁵

For example increased animal protein has been associated with a decreased risk for coronary heart disease (CHD) in a large group of nurses (The Nurses Health Study),⁶ whereas exactly the opposite association was found for markers of CHD and meat consumption in people from rural China.^{7, 8} An analogy here may be appropriate to show you why observational epidemiological studies can only show relationships and not establish causality. In New York City, there is a strong association between the size of a structure fire and the number of fire trucks at the fire, but can we conclude that more fire trucks cause bigger fires?

In order to establish cause and effect between diet and disease, it takes more than just observational epidemiological evidence.⁵ There must also be what is referred to as “biological plausibility” in which evidence gathered from tissue, animal and short term human metabolic studies support causality.² When observational epidemiological evidence is augmented by biological plausibility studies and confirmed by randomized controlled human trials, the case for causality becomes ever more convincing. Unfortunately, charismatic Paleo bloggers seem to be unaware of these basic research design parameters when they read, evaluate and report upon the scientific nutritional literature.

Summary

Over the past five or six years, The Paleo Diet, has grown into a household concept known to millions of people worldwide and has beneficially affected the health and well-being of countless individuals. Nevertheless, the original message, developed primarily in the scientific literature, has now increasingly become diluted as certain charismatic, non-scientific bloggers turn the original concept into a non-factual, personal belief system without consensus or support by the scientific Paleo Diet literature.

If your charismatic Paleo blogger promotes any of the following nutritional guidelines below, you may want to rigorously research each of these concepts for yourself and then reconsider these so-called “experts” as spokespersons for contemporary Paleo diets.

Charismatic Paleo bloggers myths debunked

These four guidelines promoted by charismatic Paleo bloggers have been debunked and are not part of a real Paleo Diet:

• Dairy products can be a regular components of contemporary Paleo diets.
• Sea salt can be used in lieu of regular table salt in contemporary Paleo diets.
• Legumes and beans are nutritious foods and should be regularly included in contemporary Paleo diets.
• Contemporary Paleo diets cannot be successfully implemented without the use of various supplements or supplement mixtures conveniently concocted and sold only by your friendly charismatic Paleo blogger.*

* Clearly humans are no longer wild hunter-gatherers and in the 21st century we are definitely operating in a foreign niche. Consequently, a few dietary exceptions are required.

Most of us stay indoors in buildings 24/7, whereas our ancestors had no such thing as "indoors." Accordingly, to insure our blood concentrations of Vitamin D are equivalent to our outdoor living ancestors, we need to either sunbathe regularly or supplement with Vitamin D.

If we eat fatty fish (salmon, herring, mackerel, sardines, etc.) 2-3 times per week, there is absolutely no need to supplement with fish oil. Nevertheless, a significant number of people in westernized countries don't like or can’t afford fresh fish. Hence the need to supplement with fish oil, so that our blood concentrations of long chain omega 3 fatty acids are comparable to our pre-agricultural ancestors who ate the entire carcass of their prey animals (brains, gonads, liver, kidney) which are highly concentrated sources of long chain omega 3 fatty acids. For most westerners it is culturally offputting and not consistent with our modern tastes to regularly eat brains etc.

The calcium issue is a tricky whicket, but we have devoted an entire blog and soon to be peer review scientific paper on this topic: “Got Bones? The Paleo Solution for Building Strong Bones While Keeping Arteries Soft and Supple” Evolution through natural selection has completely figured out the calcium conundrum in hominins from 2 MYA until the agricultural revolution. Outdoor living in a wild environment to which we are genetically adapted solves the calcium issue. Modern people who habitually consume salt (whether Paleo or not), unless they consume fresh fruits and veggies daily to the tune of about 25-35 % of energy may require calcium supplements.

References

1. Sempos CT, Liu K, Ernst ND. Food and nutrient exposures: what to consider when evaluating epidemiologic evidence. Am J Clin Nutr. 1999 Jun;69(6):1330S-1338S.
2. Potischman N, Weed DL. Causal criteria in nutritional epidemiology. Am J Clin Nutr. 1999 Jun;69(6):1309S-1314S.
3. Freudenheim JL. Study design and hypothesis testing: issues in the evaluation of evidence from research in nutritional epidemiology. Am J Clin Nutr. 1999 Jun; 69(6): 1315S-1321S.
4. Fraser GE. A search for truth in dietary epidemiology. Am J Clin Nutr. 2003 Sep;78(3 Suppl):521S-525S.
5. Flegal KM. Evaluating epidemiologic evidence of the effects of food and nutrient exposures. Am J Clin Nutr. 1999 Jun;69(6):1339S-1344S.
6. Hu FB, Stampfer MJ, Manson JE, Rimm E, Colditz GA, Speizer FE, Hennekens CH, Willett WC. Dietary protein and risk of ischemic heart disease in women. Am J Clin Nutr. 1999 Aug;70(2):221-7.
7. Campbell TC, Junshi C. Diet and chronic degenerative diseases: perspectives from China. Am J Clin Nutr. 1994 May;59(5 Suppl):1153S-1161S.
8. Campbell TC, Parpia B, Chen J. Diet, lifestyle, and the etiology of coronary artery disease: the Cornell China study. Am J Cardiol. 1998 Nov 26;82(10B):18T-21T.