ChatGPT-4o与Gemini-1.5-Pro在AI辅助视频喉镜评估中的准确性

文献信息

中文摘要

目的： 探讨两种大型语言模型（LLMs）作为常见咽喉部疾病管理辅助临床工具的实用性，比较其在有无患者症状和检查结果时的表现。
研究设计： 前瞻性对照研究。
地点： 大学医学中心。
方法： 由ChatGPT-4o和Gemini-1.5-Pro分两个连续轮次分析40例喉科病例：第一轮仅使用视频喉频闪镜检查结果，第二轮附加症状和检查结果。LLMs提供诊断和治疗方案，由两名设盲的喉科医师使用人工智能性能工具（AIPI）进行评估。视频准确性针对喉频闪镜各项条目采用5点Likert量表评估。
结果： 基于合并的患者病史和视频喉频闪镜检查，Gemini-1.5-Pro正确诊断18例（45%），而ChatGPT-4o为19例（47.5%），Gemini-1.5-Pro在鉴别诊断和额外检查建议方面表现更优。仅使用视频喉频闪镜检查时，ChatGPT-4o保持一致的诊断准确性（47.5%），而Gemini-1.5-Pro的表现显著下降（17.5%），导致ChatGPT-4o的AIPI总分显著更高（p = 0.001）。添加临床信息对ChatGPT-4o的表现影响极小，但显著改善了Gemini-1.5-Pro的AIPI条目、子项和总分。ChatGPT-4o在描述解剖结构、声带运动和病变特征方面始终优于Gemini-1.5-Pro。评估者间信度极佳，所有评估的ICC值范围为0.920至0.978。
结论： ChatGPT-4o在仅使用视频喉频闪镜检查时表现出更优的诊断性能，而Gemini-1.5-Pro的准确性在附加临床信息后显著提高。这些发现提示LLMs在喉科评估方面具有不同的能力，对喉科临床实践中的潜在应用具有重要意义。

英文摘要

OBJECTIVE: To investigate the usefulness of two large language models (LLMs) as adjunctive clinical tools for the management of common laryngopharyngeal conditions, comparing their performance with and without patient symptoms and findings.
STUDY DESIGN: Prospective controlled study.
SETTING: University medical center.
METHODS: Forty laryngology cases were analyzed by ChatGPT-4o and Gemini-1.5-Pro in two sequential rounds: first using only videolaryngostroboscopy findings, then with additional symptoms and findings. LLMs provided diagnoses and treatment plans, assessed by two blinded laryngologists using the Artificial Intelligence Performance Instrument (AIPI). Video accuracy was evaluated on a 5-point Likert scale for laryngeal stroboscopic items.
RESULTS: Based on combined patient history and videolaryngostroboscopy, Gemini-1.5-Pro correctly diagnosed 18 cases (45%) versus ChatGPT-4o's 19 cases (47.5%), with Gemini-1.5-Pro showing superior performance for differential diagnoses and additional examination recommendations. Using videolaryngostroboscopy alone, ChatGPT-4o maintained consistent diagnostic accuracy (47.5%), while Gemini-1.5-Pro's performance declined significantly (17.5%), resulting in ChatGPT-4o demonstrating significantly higher AIPI total scores (p = 0.001). Adding clinical information minimally affected ChatGPT-4o performance but substantially improved Gemini-1.5-Pro AIPI item, sub- and total scores. ChatGPT-4o consistently outperformed Gemini-1.5-Pro in describing anatomical structures, vocal fold movements, and lesion characterization. Inter-rater reliability was excellent, with ICC values ranging from 0.920 to 0.978 for all assessments.
CONCLUSION: ChatGPT-4o demonstrated superior diagnostic performance with videolaryngostroboscopy alone, while Gemini-1.5-Pro's accuracy significantly improved with additional clinical information. These findings suggest distinct capabilities between LLMs for laryngological assessment, with potential implications for clinical use in laryngology practice.