Toward Foundation Models in Radiology? Quantitative Assessment of GPT-4V’s Multimodal and Multianatomic Region Capabilities

URN zum Zitieren dieses Dokuments:: urn:nbn:de:bvb:355-epub-767898
DOI zum Zitieren dieses Dokuments:: 10.5283/epub.76789

Strotzer, Quirin D.

; Nieberle, Felix

; Kupke, Laura S. ; Napodano, Gerardo ; Muertz, Anna Katharina ; Meiler, Stefanie

; Einspieler, Ingo ; Rennert, Janine ; Strotzer, Michael ; Wiesinger, Isabel ; Wendl, Christina ; Stroszczynski, Christian

; Hamer, Okka W.

; Schicho, Andreas

Lizenz: Creative Commons Namensnennung 4.0 International
PDF - Veröffentlichte Version
(1MB)

Veröffentlichungsdatum dieses Volltextes: 30 Mai 2025 14:05

Alternative Links zum Volltext:DOI Verlag

Details

Vorschau

Bibliographische Daten exportieren

Zusammenfassung

Zusammenfassung

Background
Large language models have already demonstrated potential in medical text processing. GPT-4V, a large vision-language model from OpenAI, has shown potential for medical imaging, yet a quantitative analysis is lacking.
Purpose
To quantitatively assess the performance of GPT-4V in interpreting radiologic images using unseen data.
Materials and Methods
This retrospective study included single representative abnormal and healthy control images from neuroradiology, cardiothoracic radiology, and musculoskeletal radiology (CT, MRI, radiography) to generate reports using GPT-4V via the application programming interface from February to March 2024. The factual correctness of free-text reports and the performance in detecting abnormalities in binary classification tasks were assessed using accuracy, sensitivity, and specificity. The binary classification performance was compared with that of a first-year nonradiologist in training and four board-certified radiologists.
Results
A total of 515 images in 470 patients (median age, 61 years [IQR, 44–71 years]; 267 male) were included, of which 345 images were abnormal. GPT-4V correctly identified the imaging modality and anatomic region in 100% (515 of 515) and 99.2% (511 of 515) of images, respectively. Diagnostic accuracy in free-text reports was between 0% (0 of 33 images) for pneumothorax (CT and radiography) and 90% (45 of 50 images) for brain tumor (MRI). In binary classification tasks, GPT-4V showed sensitivities between 56% (14 of 25 images) for ischemic stroke and 100% (25 of 25 images) for brain hemorrhage and specificities between 8% (two of 25 images) for brain hemorrhage and 52% (13 of 25 images) for pneumothorax, compared with a pooled sensitivity of 97.2% (1103 of 1135 images) and pooled specificity of 97.2% (1084 of 1115 images) for the human readers across all tasks. The model exhibited a clear tendency to overdiagnose abnormalities, with 86.5% (147 of 170 images) and 67.7% (151 of 223 images) false-positive rates for the free-text and binary classification tasks, respectively.
Conclusion
GPT-4V, in its earliest version, recognized medical image content and reliably determined the modality and anatomic region from single images. However, GPT-4V failed to detect, classify, or rule out abnormalities in image interpretation.

Nur für Besitzer und Autoren: Kontrollseite des Eintrags

Downloadstatistik

Altmetric

Alternative Statistik (altmetrics)

Weitere Literatur (mittels CORE)

Details

Vorschau

Bibliographische Daten exportieren

Zusammenfassung

Zusammenfassung

Downloadstatistik

Downloads

Alternative Statistik (altmetrics)

Weitere Literatur (mittels CORE)

Universitätsbibliothek