GutCheck
AI Research
A two-week comparison of SAM 3.1, DINOv3, and PraNet on colonoscopy polyp segmentation. The 2020 specialist still wins across datasets, and changing one prompt word gains 13.5 Dice points.
Key features
- PraNet, from 2020, scores 0.905 Dice on a second dataset. SAM 3.1 with LoRA scores 0.764 and DINOv3 with a trained decoder scores 0.778
- PraNet and DINOv3 run at 54 frames per second on an H100. SAM 3.1 runs at 16
- Changing the SAM 3.1 prompt from "polyp" to "growth" lifts cross-dataset Dice from 0.569 to 0.705
- No other word worked. "tumor" fell to 0.36 and "lesion" to 0.15
- Of SAM 3.1's 42 worst failures, only 2 are also bad for PraNet. Most come from the dark ring around the scope's view
- Trained on Kvasir-SEG and tested on Kvasir and CVC-ClinicDB, on one H100
The question was whether six years of progress in vision foundation models helps on a medical imaging task. I compared four approaches on finding polyps in colonoscopy images: SAM 3.1 with a text prompt, SAM 3.1 adapted with LoRA, DINOv3 with a small trained decoder, and PraNet, a specialist model from 2020.
The result
On the dataset they trained on, the new models get close. DINOv3 scores 0.895 Dice and PraNet 0.898. On a second dataset none of them had seen, PraNet scores 0.905 and nothing else passes 0.78. The specialist is also about three times faster than SAM.
Two things I didn’t expect
SAM 3.1 doesn’t fail on hard polyps. It fails on framing. Many colonoscopy images show a round view inside a black ring, and that ring is what predicts a bad result.
The other was the prompt. Asking SAM 3.1 for “growth” where I had asked for “polyp” gained 13.5 Dice points on the second dataset, with nothing else changed. It helped 12 images by a wide margin and hurt none by the same margin. My guess is that the model has seen the word “growth” far more often.