SEG4SEG: Identifying Systematic Failure Modes in Segmentation by Subgroup Discovery Methods
Abstract
Deep learning models for medical image segmentation can achieve high overall performance but fail systematically on critical subgroups. While Slice Discovery Methods (SDM) have shown promise in revealing classification failures, their effectiveness for segmentation remains unexplored. Moreover, although various systematic failures have been reported in segmentation tasks, no prior work has systematically categorized them. In this work, we address both gaps. First, we categorize potential sources of systematic errors in medical image segmentation. Second, we empirically investigate whether SDMs can identify problematic slices in each of those categories without manual annotations. Our evaluation covers four controlled failure types and two real-world failure cases, using medical imaging datasets and explicit success criteria for SDM evaluation. Our experiments show that SDMs adapted for segmentation can identify systematic errors, demonstrating their potential for failure analysis in medical imaging.
Taxonomy of Segmentation Failure Modes and Experiment Design
We systematically categorize segmentation failure modes and design representative experiments to evaluate whether the SDM can uncover them.
Failure Mode Taxonomy
More details and examples of each failure mode can be found in the paper.
Models rely on spurious correlations instead of meaningful visual patterns.
- Sample-level: global artifacts (e.g., calipers)
- Pixel-level: spatial bias (e.g., center cropping)
Inconsistencies in annotations that models may learn.
- Annotation Style: subjective boundary definitions
- Annotation Error: omission or incorrect labeling
Poor performance on rare cases, minority groups, or small structures due to limited data coverage.
Intrinsic difficulty caused by image quality, structural complexity, or high variability.
Experiment Design
Based on this taxonomy, we construct four representative experimental cases by introducing controlled perturbations corresponding to different failure modes. We then evaluate whether the SDM can successfully identify these failures.
Overview of Experimental Cases
| Case | Failure Mode | Real-world Case | Dataset | Failure Attribute (af) |
|---|---|---|---|---|
| A | Shortcut (sample level) | Calipers in ultrasound | HC18 | Non-caliper |
| B | Shortcut (pixel level) | Central cropping in skin lesions | ISIC2018 | Non-central masking |
| C | Annotation style | Boundary style in skin lesions | ISIC2018 | Polygon-style annotation |
| D | Difficult cases | Low-quality retinal images | FIVES | Low quality |
Table 1. Overview of failure modes and corresponding datasets.
Visual Overview
Figure 1. Illustration of failure modes across four experimental cases. Green backgrounds indicate samples that dominate the dataset, while red backgrounds highlight samples with systematic errors that we aim to identify and slice out using SDM methods. For demonstration purposes, we show the same sample with and without manipulation to amplify differences; in actual experiments, manipulated and unmanipulated versions of the same sample are never included in the same run. The added calipers in Case A are exaggerated for visual clarity.
Proposed Framework: SEG4SEG
A segmentation-aware slice discovery framework with representation and evaluation extensions.
SEG4SEG follows a three-step pipeline:
- Representation. Each sample is embedded using both image features and performance-related information. We use CLIP for image embeddings and metrics such as Dice and PPV for performance embeddings, followed by UMAP for dimensionality reduction.
- Clustering. A Gaussian Mixture Model (GMM) is applied to the joint representation, enabling flexible weighting between image and performance information.
- Inspection. The resulting clusters are analyzed to identify coherent slices associated with systematic segmentation failures.
SEG4SEG extends both image and performance representations to better capture segmentation-specific, pixel-level information.
- Image space: masked inputs using ground-truth or predicted masks
- Performance space: confusion-aware metrics such as PPV and NPV to distinguish different error types
We explicitly define when a discovered slice corresponds to a meaningful failure mode.
- High purity: enriched for the failure-related attribute
- Low performance: belongs to lower-performing samples
- Sufficient size: large enough to be reliable and inspectable
Key Findings
SEG4SEG effectively identifies problematic slices and highlights the importance of segmentation-aware representations and evaluation criteria.
Figure. SDM results across cases. (a) Detection rates under varying failure prevalence. (b) Case A: the proposed criterion is more sensitive to performance gaps than purity or performance alone. (c) Case C: segmentation-specific embeddings improve detection.
SEG4SEG reliably detects problematic slices.
High detection rates are achieved across most cases, even at low prevalence (5%). Pixel-level shortcuts (Case B) are more challenging due to sample-level grouping.
Purity and performance disparity alone are insufficient.
Neither metric reliably indicates meaningful slice discovery, and high purity does not necessarily correspond to the true source of performance degradation.
The proposed criterion better reflects failure impact.
Pass rates increase with the performance gap between failure-related and unaffected samples, indicating stronger alignment with real model failures.
Segmentation-specific embeddings are critical.
Default embeddings may fail to detect certain errors, while mask-aware inputs and confusion-aware metrics (e.g., PPV) significantly improve detection.
SEG4SEG on Real-world Datasets
SEG4SEG generalizes to real-world data and uncovers meaningful failure modes without synthetic perturbations.
Applying SEG4SEG to real-world datasets (ISIC2018 and FIVES) reveals failure modes such as annotation inconsistencies and low image quality. The discovered slices exhibit clear performance differences and pass the proposed criteria, confirming that they correspond to meaningful sources of model failure.
Figure 4. SEG4SEG uncovers real-world failure modes. (a) Annotation style differences in skin lesion segmentation. (b) Low image quality in retinal imaging. The proposed criteria align with performance disparities across subgroups.
Figure 5. Example SDM result on the FIVES dataset. Low-performing clusters contain a higher proportion of low-quality images, and visual inspection confirms clear differences between clusters.
Acknowledgements
NW, AF and SB were partially funded by DTU Compute, the Technical University of Denmark; the Pioneer Centre for AI (DNRF grant nr P1); and the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (MLLS, grant NNF20OC0062606). This work was conducted during NW’s external research stay, with the stay partially supported by Otto Mønsted foundation, IDAs og Berg-Nielsens Studie-og støttefond and travel scholarship from DTU. LMK and AB were supported by the Diabetes Center Berne. SS was supported by Deutsche Forschungsgemeinschaft (DFG) – EXC number 2064/1 – Project number 390727645, the Carl Zeiss Foundation in the project ``Certification and Foundations of Safe Machine Learning Systems in Healthcare". The funding agencies had no influence on the writing of the manuscript nor on the decision to submit it for publication.
BibTeX
@inproceedings{weng2026seg4seg,
title={SEG4SEG: Identifying Systematic Failure Modes in Segmentation by Subgroup Discovery Methods},
author={Weng, Nina and Petersen, Eike and Bissoto, Alceu and Sun, Susu and Koch, Lisa M and Feragen, Aasa and Bigdeli, Siavash and Baumgartner, Christian F},
booktitle={Medical Imaging with Deep Learning},
year={2026}
}