Readings & Assignments

8/26 Scientific method & hypothesis testing

Wikipedia: Scientific Method

Assignment: What is "the scientific method"? What are hypotheses? How can they be tested?

Optional Advanced Reading:
Wagenmakers 2007
Bayesian primer

8/31 - 9/7 Project assignments; PI presentations, (IRB approval process)

Complete the CITI certification course (the minimal risk research module)
Submit your ID to your mentor/lab coordinator so you will be added to their experimental protocol.

Assignment: Read all the target papers and write down at least 2 questions that you want to ask during the class discussion. After the class, read your assigned paper in detail and spend 30min on background research.

9/09 Presentation Skills

watch the YouTube video: "Happy secret to better work" by Shawn Achor

Take a look at student presentation examples

9/14 – 9/16 Target Study, Research questions and hypothesis

Assignment: Read assigned papers carefully and write down their hypotheses and how they tested them. When it's your turn to present, prepare notes and slides

9/21 - 9/23 Background of target study

Assignment: Read 3-5 papers related to your target paper and be prepared to summarize them and present them

9/28 Experiment Design

Discussion Article:
Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective attention. Nature, 423(6939), 534-537.

Also useful to read:
Levitin, D. J. (2002). Foundations of Cognitive Psychology: Core readings. Chapter 6: Experimental Design in Psychological Research, pp. 115 - 126. Cambridge: MIT Press.

Shepard, R. N., & Metzler, J. (1971). Mental Rotation of Three-Dimensional Objects. Science, 171, 701-703.

Assignment: Identify the independent variables and levels, dependent variables, and within- and between-subjects factors

09/30 Replicability, Reproducibility, and Replication crisis in Psychology

Assignment: Think about factors contributing to the systematic non-replicability of many studies. What actions can be taken to overcome this problem?

The 9 Circles of Scientific Hell
Ioannidis, J. P. a. (2005). Why most published research findings are false. PLoS Med, 2(8), e124.
Pashler, H., & Harris, C. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536. doi:10.1177/1745691612463401

Optional Advanced Reading:
Pashler, H., & Wagenmakers, E. J. (2012). Editors' introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530.
Brandt, M. J., Ijzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R., … van 't Veer, A. (2014). The Replication Recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50(1), 217–224. doi:10.1016/j.jesp.2013.10.005
Button, K. S., Ioannidis, J. P. a., Mokrysz, C., Nosek, B. a., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(May). doi:10.1038/nrn3475

10/5 - 10/7 Experimental paradigms used in the target projects

Assignment: Be prepared to present (and answer questions about) the design and methods of your target study. Go back to all the target papers and understand the basic experimental design/methods and why they were chosen (over other alternatives)

Optional discussion topic: Preregistration
Which parts of each study could have been preregistered? Should preregistration be required for all research? In what ways is a preregistered study more valuable than a non-preregistered study?
Psychologists Call for Preregistration Open Letter to The Guardian
Why Psychology Needs Preregistration by Dorothy Bishop
How to Preregister a Study by Chris Chambers

10/14 Academic Writing Workshop (Methods)

Assignment: TBD

10/19 Team communication and organization (notes, files, codes)

Assignment: TBD

10/21 Data format plus figures

Assignment: Ask your PI to gain access to a raw data format (a record of your subjects' responses). Choose one figure in your target paper and understand the flow between the raw data format and the final visualization. What is plotted in the figure? What kinds of computation is necessary?

10/24 Understanding Data and Results (confidence intervals and statistical significance)

Assignment: TBD

Wallis papers + examples
Understanding p-values: Wagenmakers 2007

10/26-10/28 Progress Updates (present your pilot data)

Assignment: Be prepared to talk about current states of your projects. Ask questions to solicit feedback from the instructors and peers.

11/2-11/4 Data Wrangling and Visualization, Introduction to Matlab & R

Assignment: Install R and R studio on your laptop
Matlab. You will receive datasets to work with.

Checking Data
Descriptive Statistics
What are Outliers
Detecting Outliers
Dealing with Missing Data
Blog Comparison of Database Software

11/9 - 11/11 Data Curation, Visualization, and Understanding

Assignment: TBD. We will continue from where we left off in the previous week. We aim to learn how to plot your actual data and read graphs in papers. What's a confidence interval? Why is it important?

11/16 - 11/18 Progress Updates, Working with Data Output

Ten simple rules for better figures
Beyond bar and line graphs

11/23 Academic Writing Workshop (Results)

Assignment: TBD

Optional reading materials
A Brief Guide to Writing a Psychology Paper -- from Harvard University
Sternberg, R.J (1993). How to win acceptances by psychology journals: 21 tips for better writing. APS Observer 5, 12-14.
How to Publish a Replication in Perspectives in Psychological Science

11/25 In-class worktime (Bring your questions and discuss them with the instructors)

11/30 -12/2 Final Progress Updates, Final Presentation Dry-runs

12/9 -12/11 Final Presentations