OutCast: Outdoor Single Image Relighting with Cast Shadows
![](/images/pdf-icon.png)
![](/images/outcast/Teaser.png)
Abstract
We propose a relighting method for outdoor images. Our method mainly focuses on predicting cast shadows in arbitrary novel lighting directions from a single image while also accounting for shading and global effects such the sun light color and clouds. Previous solutions for this problem rely on reconstructing occluder geometry, e.g., using multi-view stereo, which requires many images of the scene. Instead, in this work we make use of a noisy off-the-shelf single-image depth map estimation as a source of geometry. Whilst this can be a good guide for some lighting effects, the resulting depth map quality is insufficient for directly ray-tracing the shadows. Addressing this, we propose a learned image space ray-marching layer that converts the approximate depth map into a deep 3D representation that is fused into occlusion queries using a learned traversal. Our proposed method achieves, for the first time, state-of-the-art relighting results, with only a single image as input.
Comparisons
If no comparisons are present please refresh your browser (CTRL/CMD R).![](/images/outcast/cristo_input.jpg)
![](/images/outcast/cristo_relit.jpg)
![](/images/outcast/lighthouse_input.jpg)
![](/images/outcast/lighthouse_relit.jpg)
![](/images/outcast/rothemburg_input.jpg)
![](/images/outcast/rothemburg_relit.jpg)
![](/images/outcast/mont_input.png)
![](/images/outcast/mont_relit.png)
Interactive Timelapse
Drag the slider beneath the relit image (right) to change sun position. All results are generated using our method.
![](/images/outcast/timelapse/caerfanon_castle/caerfanonCastle.jpeg)
![](/images/outcast/timelapse/cristo/cristo.jpeg)
![](/images/outcast/timelapse/temple_of_love/temple_of_love.jpeg)
![](/images/outcast/timelapse/peyrou/peyrou.jpg)
Video
Network Overview
![](/images/outcast/architecture.png)
Cite
@article{griffiths2022outcast,
title={OutCast: Single Image Relighting with Cast Shadows},
author={Griffiths, David and Ritschel, Tobias and Philip, Julien},
journal={Computer Graphics Forum},
volume={43},
year={2022},
organization={Wiley Online Library}
}