QRP Survey 2020

This topic is to resurrect the STORK QRP survey.

Link to the ethics submission draft: https://docs.google.com/document/d/1H6f2IXm3DUquDeDscNy9z295ccZkluPPPRgVpG6_CeM/edit

Link to survey development: https://docs.google.com/spreadsheets/d/1Y6BMeoKERBF7NWCZ98rguMir0HgLOiRUwuceR_lwjE8/edit#gid=1054969288

Recent suggestion from Ian: I wonder also, if it’s worth rather than asking explicitly whether someone has HARKed or p-hacked, could we just ask questions by just describing the behaviour? E.g., “Have you ever collected or selected data or statistical analyses until nonsignificant results become statistically significant (e.g., fell below some statistically significant threshold such as p=.05).” or “Have you ever presented a post hoc hypothesis (i.e., one based on or informed by your results) in your research report as if it were, in fact, an a priori hypotheses?”. We have done this already in places but often put the term in brackets after. My reasoning is pretty straightforward, people are more likely to have an automatic negative/defensive response to the word p-hacking but not really know what it is. So, they may p-hack not knowing what they’re doing is bad, although they have a sense that something called p-hacking is bad. Hope this makes sense. I think our idea of taking an approach similar to Anderson, Martinson and De Vries: asking what researchers subscribe to (what they believe is good practice), what they do (own behaviour), and what they believe others do (others behaviour). https://journals.sagepub.com/doi/abs/10.1525/jer.2007.2.4.3

Links to two pilot/draft surveys from Brandon & Ian:

Just saw this on twitter. Might be useful. https://www.nature.com/articles/d41586-020-00687-0 reporting on https://www.biorxiv.org/content/10.1101/2020.02.19.955328v1