We propose a novel relighting approach that takes advantage of multiple color plus depth images acquired from a consumer camera. Assuming distant illumination and Lambertian reflectance, we model the reflected light field in terms of spherical harmonic coefficients of the bi-directional reflectance distribution function and lighting. We make use of the noisy depth information together with color images taken under different illumination conditions to refine surface normals inferred from depth. We first perform refinement on the surface normals using the first order spherical harmonics. We initialize this non-linear optimization with a linear approximation to greatly reduce computation time. With surface normals refined, we formulate the recovery of albedo and lighting in a matrix factorization setting, involving second order spherical harmonics. Albedo and lighting coefficients are recovered up to a global scaling ambiguity. We demonstrate our method on both simulated and real data, and show that it can successfully recover both illumination and albedo to produce realistic relighting results.
License type:
PublisherCopyrights
Funding Info:
Siying Liu was supported by A*STAR AGS Scholarship. Minh Do was supported by US National Science Foundation
under Grant CCF-1218682.
Description:
(c) 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.