Quantifying spectral information and redundancy in high-dimensional personality taxonomies: an entropy-based approach.

BACKGROUND

Personality assessment involves the collection and organization of information into multidimensional taxonomies. However, no routinely used index directly quantifies the amount of information encoded in such taxonomies. Effective dimensionality (ED) is a Shannon entropy-based metric that uses eigenvalue distributions to quantify a system's spectral information as the effective number of non-redundant components. Although ED has been applied across fields ranging from physics to political science, it remains underutilized in psychological assessment.

METHODS

After establishing an interpretive framework through Monte Carlo simulation, we applied ED to quantify informational content and redundancy across five widely used taxonomies of normal and pathological personality in large community and clinical samples (n = 2,510-7,953), as well as across hierarchical levels of measurement (domains, facets, items), alternative factor-analytic solutions, and populations.

RESULTS

On average, domains conveyed 21% less spectral information than their nominal dimensionality implies, and facets 45% less, reflecting substantial trait overlap. Across levels, items contained 9- to 31-fold greater spectral information than domains, reflecting both specific variance and measurement error. Although residualized facets and items contained unique variance that predicted clinically relevant life outcomes, gains did not reach significance under stringent out-of-sample comparisons. In factor analysis, ED quantified factor redundancy across rotation methods, with oblique rotations producing up to 25% less spectral information. ED also informed cutoff calibration to mitigate the curse of dimensionality-which can artifactually inflate disorder rates in high-dimensional systems-maintaining prevalence estimates within 1-15% of target levels under multivariate normality.

CONCLUSION

ED provides a model-agnostic, continuous index for quantifying spectral information, structural redundancy, and complexity in multidimensional psychological systems. However, it does not necessarily coincide with psychologically meaningful information. ED may support the development of informationally parsimonious taxonomies and well-calibrated diagnostic thresholds in high-dimensional personality assessment.