Psych Colloq: Andrew Gelman

Date: 

Thursday, January 29, 2015, 12:00pm to 1:30pm

Location: 

William James Hall 1

The Statistical Crisis in Science

Andrew Gelman, Dept of Statistics and Dept of Political Science, Columbia University

Top journals in psychology routinely publish ridiculous, scientifically implausible claims, justified based on “p < 0.05.”  And this in turn calls into question all sorts of more plausible, but not necessarily true, claims, that are supported by this same sort of evidence.  To put it another way:  we can all laugh at studies of ESP, or ovulation and voting, but what about MRI studies of political attitudes, or embodied cognition, or stereotype threat, or, for that matter, the latest potential cancer cure?  If we can’t trust p-values, does experimental science involving human variation just have to start over?  And what to we do in fields such as political science and economics, where preregistered replication can be difficult or impossible?  Can Bayesian inference supply a solution?  Maybe.  These are not easy problems, but they’re important problems.

Here are the slides from the last time I gave this talk:
http://www.stat.columbia.edu/~gelman/presentations/psych_crisis.pdf

And here are some relevant articles:

[2014] Beyond power calculations: Assessing Type S (sign) and Type M (magnitude) errors. {\em Perspectives on Psychological Science} {\bf 9}, 641--651. (Andrew Gelman and John Carlin):
http://www.stat.columbia.edu/~gelman/research/published/retropower_final.pdf

[2014] The connection between varying treatment effects and the crisis of unreplicable research: A Bayesian perspective. {\em Journal of Management}. (Andrew Gelman):
http://www.stat.columbia.edu/~gelman/research/published/bayes_management.pdf

[2013] It’s too hard to publish criticisms and obtain data for replication. {\em Chance} {\bf 26} (3), 49--52. (Andrew Gelman):
http://www.stat.columbia.edu/~gelman/research/published/ChanceEthics8.pdf

[2012] P-values and statistical practice. {\em Epidemiology}. (Andrew Gelman):
http://www.stat.columbia.edu/~gelman/research/published/pvalues3.pdf