In this installment of Med Op-Ed: Does AI promote deskilling in gastroenterologists? What’s the best design for human-AI workflows in the endoscopy suite? How should clinicians calibrate and validate AI-based tools?

Will AI Deskilling Overshadow Adenoma Detection Gains?

AI-based computer‐aided detection (CADe) in fecal immunochemical test (FIT)-positive colonoscopy raises adenoma detection counts without improving advanced adenoma detection in high-performing screening programs, write Ignasi Puig, MD, PhD, and Maria Pellisé, MD, PhD, in the United European Gastroenterology Journal. Commenting on randomized trials in organized FIT-based screening, the authors note there are also potential “deskilling signals” in the performance of endoscopists using CADe. They say these findings warrant cautious, value-based adoption of AI-powered CADe prioritizing clinically meaningful outcomes, surveillance burden, and operator behavior.

The Context

• Routine exposure to CADe may alter endoscopist behavior, with an observational study noting a reduced adenoma detection rate during non-AI procedures, raising potential deskilling concerns.

• For example, despite modest gains in identifying small, proximal, and nonpolypoid lesions, endoscopists using CADe in the CADILLAC trial showed no improvement in detection of advanced colorectal neoplasia.

• Recent guidance from the European Society of Gastrointestinal Endoscopy reflects trade-offs: a weak recommendation for CADe, citing limited and uncertain effects on colorectal cancer incidence and mortality alongside increased detection and surveillance burden.

In Their Own Words

“Artificial intelligence is not a universal solution, but a context-sensitive tool whose value depends on baseline performance, operator expertise, and clinically meaningful outcomes.”

Read it: Artificial Intelligence in FIT ‐ Positive Colonoscopy: Balancing Detection Metrics and Clinical Impact

Designing Human-AI Workflows for Effectiveness

According to a commentary in Gastroenterology, the benefits seen in randomized trials of CADe do not uniformly translate to practice because endoscopists’ expertise and trust calibration determine how much detection work they delegate to AI. Alan Barkun, MD, and colleagues note that interviews with endoscopists reveal that CADe performance and provider trust shape primary, supervisory, or minimal delegation to AI tools. This underscores the need for targeted training, workflow integration, and monitoring to maintain effective use of AI tools such as CADe.

The Context

• Randomized trials and meta-analyses show that CADe can increase adenoma detection and reduce miss rates, yet effectiveness studies outside controlled conditions have reported variable results, with integration challenges including false positives and clinician distraction.

• Trust in CADe is undermined by false positives — some studies report approximately 26-27 false activations per colonoscopy — adding noise, distraction, and cognitive load; clinicians sometimes deactivated CADe as a result.

• Endoscopist delegation to CADe exists on a continuum. Early-career clinicians tend to respond to most alerts, mid-career clinicians use CADe as a safety net after adapting to ignore false positives, while many late-career clinicians disengage due to perceived limited benefit and added cognitive load.

In Their Own Words

“False-positive alerts and associated cognitive load should be actively monitored, as they directly influence trust and sustained use. Understanding and designing [AI tools] for specific delegation patterns may constitute the key to consistent effectiveness.”

Read it: When AI Meets Expertise: Trust Calibration and Delegation in Computer-Aided Polyp Detection (CADe)

Calibration Counts: Validating AI Tools

Using AI tools in routine gastroenterology practice has been hindered by inconsistent evaluation practices and limited clinical interpretability of reported metrics, wrote Federico Cabitza, PhD, and his colleagues in Digestive and Liver Disease. Clinical evaluation of AI should be anchored to prespecified, clinically justified decision thresholds, they noted, with performance reported at those thresholds and complemented by calibration and decision-analytic assessments. External validation of medical AI on independent data and transparency about dataset similarity and regulatory requirements are essential to gauge generalizability and guide safe deployment, they conclude.

The Context

• One intuitive way endoscopists using AI can select a decision threshold is to ask, “How many false positives are we willing to accept in order to avoid one false negative?” In other words, how many unnecessary alarms or procedures (eg, additional biopsies for benign findings) are acceptable to avoid missing one true case of disease? This is the “K-heuristic.” Thresholds are rarely set above 50% and sometimes as low as 1% depending on risk tolerance.

• Low-prevalence settings can yield many false alarms even with high sensitivity/specificity; eg, in one study, a CADe system with approximately 90% sensitivity and approximately 80% specificity, applied to 1000 patients at a 0.5% cancer prevalence, produced approximately 0.5 false negatives and 199 false positives.

• Compare AI models based on accuracy or utility at meaningful thresholds to reflect their true capabilities and avoid tuning to a single threshold of convenience. This approach prevents a scenario where a model is tuned at a single threshold, which may not be the threshold clinicians actually care about.

In Their Own Words

“The key point is that clinically specified thresholds ground the performance metrics in real-world priorities. Thus, independent validation remains one of the most important safeguards for credible AI performance assessment.”

Read it: Towards transparent and reliable AI in gastroenterology: pragmatic recommendations for evaluation and reporting

This article was created using several editorial tools, including AI, as part of the process. Human editors reviewed this content before publication.