Temporal linguistic shifts in oncology randomized controlled trials following large language model availability: A corpus analysis of 21,392 publications.

BACKGROUND

Public availability of large language models (LLMs) from late 2022 has raised concerns about AI-assisted writing in scientific publishing. Oncology randomized controlled trials (RCTs) underpin cancer treatment guidelines and regulatory decisions worldwide, yet whether linguistic changes have followed the emergence of LLMs has not been examined at scale.

METHODS

We conducted a retrospective corpus linguistics analysis of 21,392 oncology RCTs indexed in PubMed (2019-2026), stratified into pre-LLM (2019-2022; n = 11,308) and post-LLM (2023-2026; n = 10,084) eras. Full text was retrieved for 10,483 papers (49.0%) via PubMed Central; abstracts were used otherwise. The primary outcome was the frequency of 34 formulaic phrases documented as disproportionately prevalent in post-LLM biomedical text. A secondary outcome applied the GRIM test. Mann-Whitney U tests, chi-squared analyses, and multivariable linear regression.

RESULTS

Post-LLM papers contained significantly more formulaic phrases than pre-LLM papers (mean 0.96 [SD 1.50] vs. 0.65 [SD 1.14]; U=51,334,465, Z = -14.56, p < 0.001; r = 0.10). The increase was consistent across full-text and abstract-only subgroups and largest in discussion sections (1.23 vs. 0.88; +40%). Year-by-year analysis showed a progressive rise from 2019 (0.45) through 2025 (1.13). GRIM results were limited by data availability (48 papers; null result, p = 0.085).

CONCLUSION

This study identified a statistically significant increase in formulaic linguistic patterns in oncology RCTs following the public release of LLMs. Although these population-level findings cannot confirm AI-assisted writing in individual papers or authors, they are consistent with increasing LLM use and highlight the need for transparent AI disclosure, ongoing surveillance of scientific writing, and evidence-informed editorial policies.