Large language models' knowledge of children's memory and suggestibility: Evaluating model predictions of prior experimental results.

BACKGROUND

Accurately predicting children's memory and suggestibility in forensic contexts, such as child sexual abuse (CSA) investigations, remains challenging for human professionals. Large Language Model (LLM), as an advanced natural language processing tool, has demonstrated excellent capabilities in both medical decision-making and assisted investigation.

OBJECTIVE

The present benchmark study evaluated the responses of four LLMs-GPT-4, Claude 3.5 Sonnet, Llama-3-405b, and Erniebot-in predicting children's performance in memory and suggestibility studies, comparing their accuracy to human professionals.

SETTING

Four state-of-the-art LLMs were tested in a controlled setting, generating 4800 predictions across 30 questions from eight published studies.

METHODS

The models were assessed on their prediction accuracy, with overall and model-specific performance analyzed using chi-square tests.

RESULTS

LLMs achieved a 65 % accuracy rate, outperforming the 57 % accuracy of human professionals in prior studies. Claude 3.5 Sonnet, GPT-4, and Llama-3-405b outperformed Erniebot, particularly excelling in tasks related to false memory implantation.

CONCLUSIONS

LLMs show potential in supporting CSA interviews by providing accurate predictions of children's behavior, although further research is needed to validate their use in real-world forensic settings.