No Evidence of Experimenter Demand Effects in Three Online Psychology Experiments.

Experimenter demand effects occur when participants alter their behavior to align with perceived study hypotheses, threatening internal validity. Concern about demand effects is pervasive in psychology. Experimenter demand may be especially acute in studies relying on experienced participants recruited online (e.g., via Prolific), who may readily guess hypotheses, or when using common paradigms (e.g., vignette studies and interventions) where study goals are transparent. We conducted three preregistered experiments (N = 2,252) examining whether explicit demand cues influence online participants' behavior across three paradigms commonly used in psychology: a dictator game, replicating prior work on demand effects (Experiment 1); a moral dilemma vignette (Experiment 2); and an intervention on group attitudes (Experiment 3). We randomly assigned participants on Prolific to receive information about the study's hypothesis or to a no-information control. As expected, we find that receiving such information significantly shifts participants' beliefs about the study's hypothesis, creating the potential for experimenter demand effects. Yet we find no evidence that learning any study's hypothesis alters participants' behavior, judgments, or attitudes, suggesting that demand effects may be elusive in online samples. Bayesian region-of-practical-equivalence (ROPE) analyses indicate that experimenter demand effects in these paradigms are typically confined to small magnitudes (|d| < 0.20), suggesting that effects larger than Cohen's d = 0.20 are unlikely to be fully explained by experimenter demand alone. These findings offer important insights for the design and interpretation of modern online psychology experiments.