skip to content

Lost in Stats: Pitfalls of the Scientific Method

[Organizers: David Gross and Felipe Montealegre-Mora; Wednesdays 4pm, Seminar Room 0.03 New Theory Building; digital participation possible at]

Science employs a collection of methods that are designed to increase our knowledge about the real world. However, it is increasingly appreciated that the scientific method as it is actually used will sometimes lead to systematically biased results. This phenomenon is best-understood in the quantitative social sciences, which have faced a replication crisis in recent years -- i.e. a situation where widely-accepted effects have turned out to be spurious. A fairly good theoretical understanding has been achieved of how a field employing state of the art inferential methods can end up accepting non-existing effects as real.

This debate has not been as prominent in physics. There are reasons to believe that physics is less affected (access to more data, quantitative theories, restriction to less complex situations). But there are also signs that we should be more concerned and would benefit from absorbing the lessons learned elsewhere.

In this seminar, we will explore the systematic failure modes of the scientific method and scientific institutions, with particular attention to their applicability to physics.


First meeting is on Wednesday, the 13th of October, 4pm. If you are interested in participating, just show up or write an email to Felipe ( Registration via KLIPS is not currently possible. Most topics will be covered by 1h talks by participants, some fundamentals might be covered by the organizers, lecture-style. Update: we are fully booked! Look at the list of presentations and preliminary materials here. (You might want to zoom in into the pdf to see the text.)

Preliminary List of Topics


  • We are only interested in systematic biases of the scientific method. That is distinct from hypotheses/theories/programs failing to live up to their hopes. Failure is an expected part of exploration. Systematic biases are not.
  • Pseudoscience ("quantum homeopathy...") or crackpots. (Though the typology of crackpots is a fascinating topic, as is its relation to physics)
  • Conspiracy theories, nihilism, unfocused cultural criticism
  • Fraud. (Fraud is boring).
  • "Culture wars" and hot-button political issues
  • Activism. Here, we aim to understand, not to reform.

Miscellaneous Resources