Which R2 should be used to benchmark polygenic score performance against heritability?

The narrow-sense heritability of a trait refers to the proportion of phenotypic variance attributable to additive genetic effects and provides a predictive upper bound for linear polygenic scores. Performance benchmarking in the context of continuous traits is generally performed using the coefficient of determination (R2). Several definitions of R2 exist, however, and here we demonstrate that the correlation-based R2 is the appropriate formulation for the specific task of heritability benchmarking. For non-linear and machine-learning-based polygenic scores, the more relevant performance ceiling is the broad-sense heritability. However, the uncritical use of R2 for the assessment of non-linear models has long been cautioned against. We show that correlation-based R2 can still be meaningfully interpreted as the proportion of phenotypic variance recoverable from a model's predictions, even when generated through non-linear means. We therefore highlight the importance of reporting correlation-based R2 when benchmarking both traditional and machine-learning-based genomic prediction models against trait heritability.