DeepSeek R1 Distilled Fails to Perform Well Against the USMLE and Other LLMs with and Without Semantics.

DeepSeek-R1 Distilled Large Language model (LLM) was touted to be almost as good and much cheeper to generate than traditional LLMs. We compared its ability to answer medical questions from the United States Medical Licensing Examinations (USMLE) and found that the performance drop from the original model to DeepSeek was significant. DeepSeek is not yet a good alternative for medical question answering.