Kelley’s Paradox and strength skewness in research on unconscious mental processes

A widely adopted approach in research on unconscious perception and cognition involves contrasting behavioral or neural responses to stimuli that have been presented to participants (e.g., old items in a memory test) against those that have not (e.g., new items), and which participants do not discriminate in their conscious reports. We demonstrate that such contrasts do not license inferences about unconscious processing, for two reasons. One is Kelley’s Paradox, a statistical phenomenon caused by regression to the mean. In the inevitable presence of measurement error, true awareness of the contrasted stimuli is not equal. The second is a consequence, within the framework of Signal Detection Theory, of unequal skewness in the strengths of target and nontarget items. The fallacious reasoning that underlies the employment of this contrast methodology is illustrated through both computational simulations and formal analysis, and its prevalence is documented in a narrative literature review. Additionally, a recognition memory experiment is reported which tests and confirms a prediction of our analysis of the contrast methodology and corroborates the susceptibility of this method to artifacts attributable to Kelley’s Paradox and strength skewness. This work challenges the validity of conclusions drawn from this popular analytic approach.