大型语言模型生成的面向公众的贲门失弛缓症健康信息：一项多维比较研究
Public-facing health information on achalasia generated by large language models: a multidimensional comparative study.
文献信息
| PMID | 42812281 |
|---|---|
| 原文 | 在 PubMed 查看原文 ↗ |
| 发表日期 | 2026 |
| 作者 | Junzhen Li |
| 作者单位 | Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University, Shenzhen, China. |
| 期刊 | Frontiers in public health |
| SCI 分区 | Q1 |
| IF | 4.3 |
| 研究类型 | AI/ML · 临床 |
| 所属专科 | 咽喉科 |
中文摘要
背景： 大型语言模型（LLMs）越来越多地被公众用于获取健康信息，但其为贲门失弛缓症提供可靠信息的能力仍不清楚。因此，我们比较了六种LLMs在回答公众关于贲门失弛缓症（一种不常见的食管动力障碍）问题时的表现，重点关注安全性、准确性、共情、信息质量与可靠性以及可读性。
方法： 在这项横断面比较研究中，于2026年5月7日至5月12日期间，将40个面向患者的贲门失弛缓症相关问题分别一次性提交给六种LLMs（ChatGPT 5.5、Claude Opus 4.7、DeepSeek V4 Pro、Gemini 3.1 Pro Thinking、Grok-4.3和Qwen3-Max）。三名胃肠病学家使用DISCERN、EQIP、评估作者身份、归属、披露和时效性的JAMA基准标准以及全球质量评分（GQS），独立评估了240份回答的安全性、准确性、共情、信息质量和可读性。可读性使用六种既定的可读性指数进行评估。
结果： 总体而言，240份回答中有27份（11.3%）被归类为可能不安全，各模型的比例范围为7.5%至15.0%；模型间安全性的总体差异无统计学意义。准确性在各模型间存在显著差异，尽管效应量较小。在共情、信息可靠性与质量以及可读性方面观察到更大的差异。ChatGPT 5.5和Gemini 3.1 Pro Thinking获得了更高的共情评分，而Claude Opus 4.7显示出更大的阅读难度。JAMA基准评分总体较低，表明来源透明度有限。
结论： 当前LLMs对公众关于贲门失弛缓症的问题提供了总体准确且通常有用的回答。然而，在安全性、共情、透明度、信息质量和可读性方面发现了一些差异。面向公众的LLM对公众关于贲门失弛缓症问题的回答应与指南一致、明确引用来源、以共情方式沟通并使用通俗语言。
英文摘要
BACKGROUND: Large language models (LLMs) are increasingly used by the public to obtain health information, but their ability to provide reliable information for achalasia remains unclear. We therefore compared six LLMs in answering public questions about achalasia, an uncommon esophageal motility disorder, focusing on safety, accuracy, empathy, information quality and reliability, and readability.
METHODS: In this cross-sectional comparative study, 40 patient-oriented questions about achalasia were submitted once to each of six LLMs (ChatGPT 5.5, Claude Opus 4.7, DeepSeek V4 Pro, Gemini 3.1 Pro Thinking, Grok-4.3, and Qwen3-Max), between May 7 and May 12, 2026. Three gastroenterologists independently assessed 240 responses for safety, accuracy, empathy, information quality, and readability using DISCERN, EQIP, the JAMA benchmark criteria assessing authorship, attribution, disclosure, and currency, and the Global Quality Score (GQS). Readability was evaluated using six established readability indices.
RESULTS: Overall, 27 of 240 responses (11.3%) were classified as potentially unsafe, with proportions ranging from 7.5 to 15.0% across models; the overall between-model difference in safety was not statistically significant. Accuracy differed significantly among models, although the effect size was small. Larger differences were observed in empathy, information reliability and quality, and readability. ChatGPT 5.5 and Gemini 3.1 Pro Thinking achieved higher empathy scores, whereas Claude Opus 4.7 showed greater reading difficulty. JAMA benchmark scores were low overall, indicating limited source transparency.
CONCLUSION: Current LLMs provided generally accurate and often useful answers to public questions about achalasia. However, some differences were found in safety, empathy, transparency, information quality, and readability. Public-facing LLM responses to public questions about achalasia should be consistent with guidelines, clearly cite sources, communicate with empathy, and use plain language.