Conferences...
A Scalable End-to-End Optimized Image-Based Rendering Framework on Graphics Hardware
ABSTRACT: This paper presents the system-level overview of a real-time image-based rendering framework performing multiple intermediate view synthesis, completely on the Graphics Processing Unit (GPU). The software design achieves high-performance, yet maintains flexibility and ease of development through a hierarchical layered architecture. The framework implements the intermediate view synthesis by a chain of consecutive processing modules, as an extension to the Middlebury open software structure, allowing it to benchmark quality and execution time of individual modules for end-to-end system performance optimization. The modules can be flexibly coordinated, enabling scalability to run the multiple view synthesis in real-time on both powerful and weak GPUs.
- Type: Conference
- Date: Thursday, May 29th, 2008
- Location: Hotel Dedeman, Istanbul, Turkey
[Download Presentation (pdf, 1.80MB)]
Courses and Seminars...
Parallelization of Stereo to an MPSoC Platform with GPU Specific Optimization
- Type: Seminar
- Date: Friday, June 15th, 2007
- Location: IMEC Auditorium, Leuven, Belgium
[Download Presentation (pdf, 2.39MB)]
Student Promoting...
Advanced Real-Time Video Preprocessing on Graphics Hardware for Image-Based Rendering Systems
ABSTRACT: Free Viewpoint Video (FVV) is a revolutionary technique that allows a spectator to freely choose a custom camera viewpoint. Based on information of several fixed-point cameras, the requested intermediate viewpoint is synthesized without the need of a physical camera. These calculations heavily rely on stereo matching algorithms, which extract depth information out of a pair of available input cameras. However, these algorithms cannot process the live feed instantly and therefore require the feed to be preprocessed adequately. The preprocessing mainly involves correcting barrel distortion that occurs by the use of practical camera lenses, and rectifying the stereo vision, i.e. aligning object features to the same scan line between the two images. This enables image-based rendering systems to synthesize any viewpoint in real-time from a practical stereo camera setup.
The core topic of this thesis book focuses on advanced video preprocessing, implemented on graphics hardware. We have expanded an existing imaged-based rendering framework -- codenamed aLive --, that originally takes in ideal images with no regards to practical imperfections whatsoever. To correct these imperfections, the fixed-point cameras are also used in a self-calibration process, exposing the cameras' relative pose and specific camera properties. Using the obtained calibration data, an adequate amount of pincushion can be applied to the images, to restore the barrel distortion caused by practical camera lenses. The acquired geometrical information is then used to rectify the self-calibrated stereo pair. Since the rectification causes the object features to align on the same scan line of the images, the stereo matching process can efficiently compute the depth information needed to synthesize the requested viewpoint. Moreover, real-time performance is still maintained by offloading the high amount of computations from CPU to the Graphics Processing Unit (GPU).
The advanced preprocessing is implemented by developing additional modules in the existing framework that use the same environment, resulting in a stand-alone performance of 3691 fps for 450 x 375 image resolutions. The full functionality thereby only suffers from a minor frame rate drop of 1.32%, to a total speed of 43.9 fps.
- Student: Steven Schuermans
- Type: Master Thesis
- School: Hogeschool voor Wetenschap en Kunst -- Campus De Nayer
