Series of webinars on reproducible research. Episode X. Statistical and Reproducibility Issues in Human Computer Interactions

To improve our practice in terms of reproducible research, some colleagues and I organize a series of webinars where we will share our experience and thoughts and on those different themes. These webinars are open to anyone (PhD students, post-doc, engineers, researchers, …) and will be in English (possibly with a strong French accent ;-). The precedent ones have addressed various topics such as:

• the management of a laboratory notebook and of replicable articles,
• the control of an environment,
• numerical reproducibility,
• activity logging and backing up,
• software and data archiving.
• reproducible science in bioinformatics.
• publication Modes Favoring Reproducible Research.
• artifact evaluation in computer systems' conferences.
• experimental Testbeds in Computer Science.

 All the slides and videos are available here: https://github.com/alegrand/RR_webinars

The next webinar will be given by:

• Pierre Dragicevic (Inria Saclay, Aviz team) on "Statistical and Reproducibility Issues in Human Computer Interactions":

We need to improve the way we do statistics in HCI, but more trainingin statistical theory is not enough. We also need good "intuitionpumps" to develop our statistical thinking skills. In this talk Iexplore the basic concept of statistical dance. The dance analogy hasbeen used by Geoff Cumming to describe the variability of p-valuesacross hypothetical replications. Through visual examples, PierreDragicevic will show why any statistical analysis and any statisticalchart actually dances across replications. He will discuss why mostattempts at stabilizing statistical dances are either insufficient ormisguided. The solution is to embrace the uncertainty and messiness inour data. We need to develop a good intuition of this uncertainty andcommunicate it faithfully to our peers. Finally, he will give a fewtips for conveying and interpreting interval estimates in our papersin a honest and truthful way.

• Yvonne Jansen (UPMC Jussieu, ISIR lab) on "Failure to replicate - a case study":

In 2010 Dana Carney, Amy Cuddy, and Andy Yap published a study on the effects of “power poses” on hormone levels and risk-taking behavior. Their findings were widely publicized through Amy Cuddy’s TED talk and have inspired much work building on these findings. Yet recent developments suggest that the effect might not actually exist and that instead many failed replications fell victim to a publication bias. In this talk I will show how we ourselves attempted to apply power poses to interface designs, how we failed to replicate effects of power poses, and the difficulties we encountered in publishing our negative findings. I conclude by discussing the value of sharing negative findings with the research community and how it conflicts with our current publication system.

It will take place on Thursday 22nd June 2017, from 1:30PM to 4:30PM (UTC+1) in the amphitheater of the IMAG building in Grenoble. It will be screencast and interactions during the presentations will take place through written interactive documents (pad, hangout, …). The resulting videos will be edited within a few days for a better exploitation by people that would not have been able to attend the event. All the information are gathered here: https://github.com/alegrand/RR_webinars/blob/master/10_statistics_and_replication_in_HCI/index.org.

If you are interested in following these talks, you should check on the previous web page whether a room has been reserved nearby and, if not, book one and update the web page accordingly as it is always much more fun/easy to learn in company of others rather than alone in a dark office. :)

We hope to see you soon.
Best,
Arnaud Legrand