Large language model-based evaluation of the impact of gender in medical research.

OBJECTIVE

Gender disparities in academic medicine have been previously reported, but prior analyses have relied on either manual labor or fixed databases of name-gender pairs that fail to generalize across different populations and cultures. The objective of this work is to evaluate the utility of large language models (LLMs) as a potential tool to facilitate systematic bibliometric analysis of academic research trends.

METHODS

We introduce an LLM-based pipeline that aggregates gender labels from multiple LLM instances to predict the genders of manuscript authors based on their first names.

RESULTS

Our proposed method outperforms alternative algorithms relying on lookup from finite databases of name-gender pairs, while also offering the scalability to tens of millions of authors that is unfeasible with other manual, human-based methods alone.

DISCUSSION AND

CONCLUSION

Our results suggest that LLMs can be a powerful tool to scalably track gender-based trends in academic medical research.