Statistical standards for scientific discovery in linguistics: a practical introduction

04/10/2017-06/10/2017: Over the last years, the current standards for scientific discovery have repeatedly been questioned. Especially the social sciences are going through a «replicability crisis» triggered by three main causes: 1) observations and experiments are carried out on samples that are too small to properly observe the investigated effects, 2) the current standard for claiming significant discoveries is based on p-values, which pose several problems (e.g. they do not indicate the size of the observe effects, they can suffer important alterations depending on small modifications of the data, eventually leading to “p-hacking”, and they are poorly understood by many researchers), and 3) strong pressure for publishing only significant results has lead to a publication bias caused by the reluctance to publish negative results, which precludes a great amount of relevant data highly needed to reject type I errors (i.e. false positives) from getting published. Some disciplines, like social psychology, are increasingly drawing attention to these issues,while others, like linguistics, have not yet addressed them satisfactorily. This workshop aims at providing an opportunity to discuss these challenges.
Particularly junior linguistics researchers, who have questions on these problems but seldom find answers within the discipline, are invited to participate. The workshop will bring together renowned experts to give plenary lectures on relevant topics like sampling methods and sample sizes, p-values vs. effect sizes and bayesian statistics, cognitive biases affecting the scientific method, and strong-inferences techniques. The workshop will have a strong practical focus, combining these plenaries with short presentations by participants as well as practical sessions accompanied by the experts.

We call for presentations of 15 minutes in which doctoral and postdoctoral researchers discuss quantitative issues from, or related to, their own research. The presentations are expected to point at potential problems, which can subsequently be addressed in practical sessions with the invited experts.