Loading ...

TOPLINE:

ChatGPT4 outperformed gastroenterology practitioners and showed better concordance with the 2020 US Multi-Society Task Force on Colorectal Cancer (USMSTF) guidelines in determining rescreening and surveillance colonoscopy intervals.

METHODOLOGY:

• Researchers compared the accuracy, concordance, and reliability of the colonoscopy rescreening and surveillance intervals recommendations provided by ChatGPT4 with those made by the reference standards from the USMSTF Panel and in the endoscopist's procedure reports from real-world gastroenterology practitioners.
• They recruited patients undergoing colonoscopy at two US centers between January 1, 2023, and April 30, 2023.
• De-identified text regarding history of present illness, age, gender, family history, colonoscopy procedure, and pathology report was entered into ChatGPT4, which was then queried for the next recommended follow-up interval.
• The primary outcome was the overall accuracy of ChatGPT4 recommendations and their alignment with the USMSTF guidelines in determining rescreening and surveillance intervals for colorectal cancer.

TAKEAWAY:

• Investigators included 505 patients (mean age, 56.3 years; 54% women), of whom 38 (7%) had a family history of colon cancer.
• ChatGPT4 successfully generated a follow-up recommendation in 99.2% of patients, with only four deferred to a gastroenterologist.
• ChatGPT4 recommendations were concordant with the recommendations of the USMSTF Panel in 85.7% of cases, whereas real-time recommendations of gastroenterologists were consistent with the recommendations of the USMSTF Panel in 75.4% of cases (McNemar's test, P < .001).
• Inter-rater reliability was good (Fleiss' kappa [κ], 0.786; P < .001) between ChatGPT4 and the USMSTF Panel and fair (κ, 0.543; P < .001) between ChatGPT4 and gastroenterology practitioners.
• Researchers cautioned that ChatGPT4 incorrectly advised delayed surveillance in 26 patients despite its excellent reliability and concordance with the USMSTF Panel.

IN PRACTICE:

"After further refinement, ChatGPT4 may help in a support role for routine clinical duties, such as suggesting colonoscopy follow-up intervals for low-risk colon polyps," the authors wrote.

SOURCE:

The study, conducted by Patrick W. Chang, MD, Division of Gastrointestinal and Liver Diseases, University of Southern California, Los Angeles, was published online in Clinical Gastroenterology and Hepatology.

LIMITATIONS:

The screening cohort analyzed in this study may have had a higher-than-normal risk for colorectal cancer. The interpretations of ChatGPT4 were according to preexisting electronic health record databases, which may have limited data for advanced adenoma in first-degree relatives and colorectal cancer in second-degree relatives.

DISCLOSURES:

No funding details were provided for this study. A few authors reported being consultants for Boston Scientific, Cook Medical, Olympus America, or Noah Medical, and others declared that they had no conflicts of interest.