g-Distance: On the Comparison of Model and Human Heterogeneity

Models are often evaluated when their behavior is at its closest to a single, sometimes averaged, set of empirical results, but this evaluation neglects the fact that both model and human behavior can be heterogeneous. Here, we develop a measure, g-distance, which considers model adequacy as the extent to which models exhibit a similar range of behaviors to the humans they model. We define g as the combination of two easily interpretable dimensions of model adequacy: accommodation and excess flexibility. We apply this measure to five models of an irrational learning effect, the inverse base-rate effect. g-Distance identifies two models, a neural network with rapid attentional shifts (NNRAS) and a dissimilarity-similarity generalized context model (DGCM18), which outperform the previously most supported exemplar-based attention to distinctive input model (EXIT). We show that this conclusion holds for a wide range of beliefs about the relative importance of excess flexibility and accommodation. We further show that a preexisting metric, the Bayesian information criterion, misidentifies a known-poor model of the inverse base-rate effect as the most adequate model. Along the way, we discover that some of the models accommodate human behavior in ways that seem unintuitive from an informal understanding of their operation, thus underlining the importance of formal expression of theories. We discuss the implications of our findings for model evaluation generally, and for models of the inverse base-rate effect in particular, and end by suggesting future avenues of research in computational modeling. (PsycInfo Database Record (c) 2025 APA, all rights reserved).