🟡 Preliminary Evidence
A new framework is being developed to address a critical gap in medical intelligence/" class="gmj-dict-autolink" title="Dictionary: Artificial Intelligence">artificial intelligence: the ability of neural networks to recognise when they encounter data outside their training experience. According to researchers presenting the framework, current AI systems used in cancer diagnosis can confidently misclassify unfamiliar tumour subtypes, creating potential risks for patient care.
Key takeaways
- AI systems trained on specific cancer datasets cannot reliably recognise when they encounter unfamiliar tumour subtypes
- A new uncertainty quantification framework aims to make neural networks aware of the limits of their training data
- The approach could reduce overconfident misclassifications in clinical oncology settings
- Implementing this framework requires changes to how AI models are developed and validated before clinical deployment
The confidence problem in medical AI
Medical artificial intelligence systems, particularly deep neural networks, face a fundamental limitation: they are “largely unaware of the limits of their training data,” according to the framework developers. When presented with novel inputs—such as a rare cancer subtype the network has never encountered during training—these systems cannot signal uncertainty. Instead, they produce confident predictions even when operating far outside their knowledge domain.
This phenomenon mirrors a widely cited analogy: a neural network trained to distinguish African mammals would confidently misidentify an unfamiliar jaguar from South America as a leopard, rather than acknowledging the animal lay beyond its training experience. In clinical oncology, such overconfidence carries direct consequences for patient care.
The uncertainty quantification challenge in AI systems
How neural networks behave when encountering unfamiliar data domains
Conceptual representation based on AI framework research, 2026 | Georgian Medical Journal News
How the framework recognises boundaries of knowledge
The new framework operates on a principle: neural networks should be trained to recognise the statistical boundaries of their training data and assign lower confidence scores when encountering inputs that deviate substantially from that distribution. The developers describe this as making networks “aware” of what they do not know.
Rather than accepting high-confidence predictions on unfamiliar tumour subtypes, the system would flag uncertainty. This allows clinicians to treat AI outputs appropriately—as strong recommendations for routine cases and as preliminary assessments requiring expert review for novel presentations. The approach mirrors good clinical practice, where doctors recognise their own knowledge limits and seek specialist input when encountering unfamiliar presentations.
Clinical and regulatory implications
Implementation of uncertainty quantification frameworks addresses longstanding concerns about AI deployment in oncology. The FDA has emphasised the need for transparent uncertainty estimates in AI-based diagnostic devices, yet many current systems lack this capability. Clinical teams need to understand not just what an AI predicts, but how confident that prediction is.
Validation of such frameworks will require prospective testing in real clinical settings, comparing AI predictions with pathologist assessments across diverse patient populations and tumour subtypes. Quality assurance and safety monitoring during clinical trials is essential to establish whether uncertainty quantification actually reduces diagnostic errors in practice.
Neural networks trained on specific cancer datasets confidently misclassify unfamiliar tumour subtypes, but new uncertainty quantification frameworks can teach systems to recognise the statistical boundaries of their training data.
— Framework developers, 2026
What this means
Frequently asked questions
Why can’t current AI systems simply say “I don’t know”?
Neural networks are mathematical functions trained to optimise accuracy on their training dataset. Without explicit uncertainty quantification methods, they default to producing high-confidence predictions on any input, including data far outside their training distribution. The framework teaches networks to detect distributional shifts and report lower confidence scores accordingly.
How is this different from asking an AI for “confidence scores”?
Standard confidence scores reflect how sure a network is about choosing between known categories (e.g., cancer subtype A vs. B). Uncertainty quantification goes further, asking whether the input itself belongs to the domain the network was trained on. A cancer subtype never seen in training would generate both low categorical confidence and high “out-of-distribution” uncertainty.
Will this framework work for all types of cancer AI?
The framework is being developed for neural network-based image analysis and classification, particularly useful in histopathology and oncology imaging. Its applicability to other AI systems—such as genomics prediction models or treatment recommendation algorithms—depends on adapting the underlying statistical principles to those domains.
The adoption of uncertainty quantification in medical AI represents a maturation of the field toward greater trustworthiness and clinical utility. As more research on AI validation and uncertainty quantification emerges, future diagnostic systems may become more reliable collaborators in the oncology clinic, helping clinicians identify which cases warrant standard AI support and which require specialist judgment.
Source: New framework renders AI more trustworthy for cancer subtyping, Medical Xpress, 2026
Was this article helpful?
Disclaimer. This article is health journalism intended for general information and education. It is not medical advice and is not a substitute for professional diagnosis or treatment. Always consult a qualified healthcare provider about your individual circumstances. Full disclaimer →
Related Coverage
Editorial standards. This article was produced under the GMJ News editorial process, with oversight by the GMJ Editorial Board. Our editorial process. Spotted an error? Contact the editorial team.
