Abstract

The computer vision community has developed numerous techniques for digitally restoring true scene information from single-view degraded photographs, an important yet extremely ill-posed task. In this work, we tackle image restoration from a different perspective by jointly denoising multiple photographs of the same scene. Our core hypothesis is that degraded images capturing a shared scene contain complementary information that, when combined, better constrains the restoration problem. To this end, we implement a powerful multi-view diffusion model that jointly generates uncorrupted views by extracting rich information from multi-view relationships. Our experiments show that our multi-view approach outperforms existing single-view image and even video-based methods on image deblurring and super-resolution tasks. Critically, our model is trained to output 3D consistent images, making it a promising tool for applications requiring robust multi-view integration, such as 3D reconstruction or pose estimation.

Qualitative Results For Image Sets Restoration Demo

Image Sets Deblurring

Image Sets Super-Resolution

3D Reconstruction on Degraded Images Video Demo

Restormer

Ours

Sparse-View 3D Reconstruction Video Demo

Motion Deblurring 3D Reconstruction
3D reconstructions from motion-blurry images restored by single-view Restormer (middle) and our multi-view SIR-Diff (right).

Blurry Image

Restormer

Ours

Super-Resolution 3D Reconstruction
3D reconstructions from low-resolution images restored by single-view OSEDiff (middle) and multi-view SIR-Diff (right).

Low-Resolution Image			OSEDiff	Ours