Detective SAM accepted at ICML 2025 DIG-BUGS Workshop
We are excited to announce that our paper "Detective SAM: Adapting SAM to Localize Diffusion-based Forgeries via Embedding Artifacts" has been accepted at the ICML 2025 DIG-BUGS Workshop.
The Problem
Image forgery localization in the diffusion era poses new challenges. Modern editing pipelines produce photorealistic, semantically coherent manipulations that bypass conventional detectors. While some recent methods leverage foundation model cues or handcrafted noise residuals, they still miss the subtle embedding artifacts introduced by modern diffusion pipelines.
Our Approach
Detective SAM extends the Segment Anything Model (SAM) with three key innovations:
- Blur-driven forensic embedding signals that capture the perturbation patterns unique to diffusion-based editing
- Hierarchical learnable prompts that guide SAM's decoder from coarse to fine forgery localization
- Lightweight adapters that enable efficient fine-tuning while keeping SAM's backbone frozen
By combining explicit forensic perturbation cues with foundation-model adaptation, Detective SAM achieves robust image forgery localization without requiring retraining on each new editing method.
Results
Detective SAM outperforms prior state-of-the-art methods on common benchmarks such as MagicBrush and CoCoGlide, demonstrating the power of this combined approach.
Links
This work is a collaboration with Chaoyi Zhu, Pin-Yu Chen, Robert Birke, and Lydia Y. Chen.