Image transformation and synthesis methods

Abstract

In a telepresence system, a scene is captured by recording pixel data elements, each associated with a pixel ray vector having a direction and an intercept on an known locus in the frame of reference of the scene. Each pixel data element includes data representing the illumination along the pixel ray vector. For example, the pixel data elements may be captured by operating numerous video cameras pointing in different directions on a spherical locus. A virtual viewpoint image representing the image which would be seen from an arbitrary viewpoint, looking in an arbitrary direction, can be synthesized by determining the directions of synthetic pixel ray vectors from each pixel of the virtual viewpoint image through the virtual viewpoint and the intercepts of these vectors on the locus. Recorded pixel data elements having pixel ray vector directions and intercepts close to those of the synthetic pixel ray vector can be copied or interpolated to provide data representing illumination in the synthetic pixel.

1 . A method of providing data defining an image of a scene, the method comprising the steps of: 
 (a) providing a plurality of starting pixel data elements, each said starting pixel data element incorporating data corresponding to illumination seen along a starting pixel ray vector associated with that starting pixel data element, each said starting pixel ray vector having a direction and an intercept on a locus; and    (b) forming said pixel data elements into a transform image including a plurality of direction matrices, each said direction matrix including pixel data elements associated with pixel ray vectors having directions parallel within a preselected direction tolerance range to a common ray direction.