High quality depth estimation for multi-view video

Abstract

This thesis presents new techniques to improve the quality of adaptive structured light based depth estimation for multi-view video. By interleaving structured light with ambient light on a frame-by-frame basis, the proposed methods can potentially capture both texture video under only ambient light for image rendering and video under both ambient light and structured light for depth estimation at video rates. By using structured light with adaptive colours which account for ambient light conditions and the colours of objects, the proposed methods not only avoid matching ambiguities in textureless areas and areas with repetitive textures but also perform robustly in areas with rich colours and textures under ambient light conditions. Firstly, a new structured light approach using adaptive colours is proposed. The adaptive colours are acquired using principal component analysis in the RGB colour space of the image of the scene under ambient light conditions. Based on a projected grid pattern of adaptive colours, a new depth estimation technique combining active and passive approaches is proposed. This technique shows the advantage of adaptive structured light over constant colours and the advantage of combining both active and passive approaches over active-only or passive-only techniques. A second new depth estimation technique using adaptive structured light and a shiftable window-based global optimization algorithm is proposed. This technique shows the advantage of adaptive structured light over random colors and generates a depth map with sharp depth discontinuities and half-pixel accuracy using the global optimization in an iterative way. A random noise pattern instead of a grid pattern is employed to improve the performance of depth estimation around object boundaries. A third new depth estimation technique using the dual-tree complex wavelet transform (DTCWT), graph cut and adaptive structured light is proposed. This technique employs the phase difference between coefficients from the DTCWT as a robust similarity measure and has the ability to keep clear depth discontinuities and detect clean and continuous occlusion areas. Finally, using the depth maps generated by the proposed techniques, image rendering is implemented through 3D warping to verify the feasibility of the proposed depth estimation techniques for multi-view video.