Compression of Depth Maps

Abstract

An apparatus comprises a receiver (201) receiving a first depth map for a first video frame of a plurality of videos frames. A divider (203) divides the first depth map into two dimensional geometric primitives/polygons larger than two pixels. A map generator (207) generates a vertex position map indicative of the vertex positions of the geometric primitives in the first depth map. A depth map generator (205) generates a second depth map comprising the depth values of the first depth map at the vertex positions and a bitstream generator (209) generates the video bitstream to include, for the first video frame, the second depth map, and the vertex position map. The division is dependent on the depth values of the depth map. A complementary apparatus renders images using a mesh generated from the bitstream by determining the original vertex positions based on the received vertex position map and projecting these to a 3D space.

1 . An apparatus comprising:
 a receiver circuit,
 wherein the receiver circuit is arranged to receive a first depth map, 
 wherein the first depth map is for at least a first video frame of a plurality of videos frames; 
   a divider circuit,
 wherein the divider circuit is arranged to divide at least a portion of the first depth map into a plurality of two dimensional geometric primitives, 
 wherein each of the plurality of two dimensional geometric primitives have an average area, 
 wherein the average area exceeds twice a pixel area for the first depth map, 
 wherein the divider circuit is arranged to determine vertex positions in the first depth map, 
 wherein vertex position are for vertices of the plurality of two dimensional geometric primitives, 
 wherein the divider circuit is arranged to generate the vertex positions based on the depth values of the first depth map, 
   a map generator circuit,
 wherein the map generator circuit is arranged to generate a vertex position map, 
 wherein the vertex position map indicates the vertex positions in the first depth map; 
   a depth map generator circuit,
 wherein the depth map generator circuit is arranged to generate a second depth map, 
 wherein the second depth map comprises the depth values of the first depth map at the vertex positions in the first depth map; 
   a bitstream generator circuit,
 wherein the bitstream generator circuit is arranged to generate a video bitstream, 
 wherein the video bitstream comprises the first video frame, the second depth map and the vertex position map. 
   
     
     
         2 . The apparatus of  claim 1 ,
 wherein the divider circuit is arranged to determine the vertex positions by offsetting reference vertex positions of a reference division of the first depth map,   wherein the offsetting for a first vertex point is based on values of the depth map for pixels within a geometric primitive of the first vertex point.   
     
     
         3 . The apparatus of  claim 2 , wherein the reference vertex positions of the reference division form a regular grid. 
     
     
         4 . The apparatus of  claim 2 , wherein the vertex position map indicates the vertex positions as relative positions with respect to the reference vertex positions of the reference division. 
     
     
         5 . The apparatus of  claim 2 ,
 wherein the offsetting comprises a first offset along a first predetermined direction,   wherein the first offset is determined as an offset for which a change of depth map values along the first predetermined direction meets a criterion.   
     
     
         6 . The apparatus of  claim 2 ,
 wherein the offsetting comprises a second offset along a second predetermined direction,   wherein the second offset is determined as an offset for which a change of depth map values along the second predetermined direction meets a criterion,   wherein the map generator circuit is arranged to generate the vertex position map so as to indicate offsets along the first predetermined direction,   wherein the map generator circuit is arranged to generate an additional vertex position map so as to indicate offsets along the second predetermined direction;   wherein the bitstream generator circuit is arranged to provide the additional vertex position map in the video bitstream.   
     
     
         7 . The apparatus of  claim 2 , the first predetermined direction is a horizontal direction. 
     
     
         8 . The apparatus of  claim 2 , where the first predetermined direction is a vertical direction. 
     
     
         9 . The apparatus of  claim 2 , wherein the offsetting is below an offset threshold. 
     
     
         10 . The apparatus of  claim 1 ,
 wherein the bitstream generator circuit comprises a video encoder,   wherein the video encoder is arranged to encode the vertex position map using video encoding.   
     
     
         11 . The apparatus of  claim 1 , wherein the divider circuit is arranged to determine the vertex positions so as to align the plurality of two dimensional geometric primitives with depth transitions in the depth map. 
     
     
         12 . An apparatus comprising:
 a receiver circuit,
 wherein the receiver circuit is arranged to receive a video bitstream, 
 wherein the video bitstream comprises:
 a first depth map for at least a first frame; and 
 a vertex position map for at least a first frame, 
 wherein the vertex position map indicates vertex positions for depth values of the first depth map in a second map, 
 wherein the second map has a higher resolution than the first depth map, 
 wherein the vertex positions define a plurality two dimensional geometric primitives, 
 wherein the plurality of two dimensional geometric primitives form a division of the second map; and 
 
   a renderer circuit, the renderer circuit comprising:
 a mesh generator circuit, wherein the mesh generator circuit is arranged to generate a mesh representation of a scene,
 wherein the mesh is formed by three dimensional geometric primitives, 
 wherein the three dimensional geometric primitives are formed by a projection of the plurality of two dimensional geometric primitives into a three dimensional space, 
 wherein the projection of a first vertex of the plurality of two dimensional geometric primitives depend on a vertex position of the first vertex in the second depth map and a depth value in the first depth map for the first vertex; and 
 
 a rendering circuit, wherein the rendering circuit is arranged to render an image based on the mesh representation. 
   
     
     
         13 . The apparatus of  claim 12 ,
 wherein the vertex position map indicates the vertex positions as relative vertex positions with respect to reference vertex positions of a reference division of the first depth map,   wherein the mesh generator circuit is arranged to determine absolute vertex positions in the second depth map based on the relative vertex positions and the reference vertex positions.   
     
     
         14 . The apparatus of  claim 13 ,
 wherein the video bitstream comprises an indication of the reference division,   wherein the mesh generator circuit is arranged to determine the reference vertex positions in response to the indication of the reference division.   
     
     
         15 . A method comprising:
 receiving a first depth map, wherein the first depth map is for at least a first video frame of a plurality of videos frames;   dividing at least a portion of the first depth map into a plurality two dimensional geometric primitives,
 wherein each of the plurality of two dimensional geometric primitives having an average area, 
 wherein the average area exceeds twice a pixel area for the first depth map, 
 wherein the divider circuit is arranged to determine vertex positions in the first depth map, 
 wherein vertex position are for vertices of the plurality of two dimensional geometric primitives, 
 wherein generating the vertex positions is bases on the depth values of the depth map; 
   generating a vertex position map, wherein the vertex position map indicates the vertex positions in the first depth map;   generating a second depth map,   wherein the second depth map comprises the depth values of the first depth map at the vertex positions in the first depth map; and   generating a video bitstream wherein the video bitstream comprises the first video frame, the second depth map and the vertex position map.   
     
     
         16 . A method comprising:
 receiving a video bitstream, wherein the video bitstream comprises:
 a first depth map for at least a first frame; and 
 a vertex position map for at least a first frame,
 wherein the vertex position map indicates vertex positions for depth values of the first depth map in a second map, 
 wherein the second map has a higher resolution than the first depth map, 
 wherein the vertex positions define a plurality two dimensional geometric primitives, 
 wherein the plurality of two dimensional geometric primitives form a division of the second map; and 
 
   rendering video images from the video bitstream, the rendering comprising:
 generating a mesh representation of a scene,
 wherein the mesh is formed by three dimensional geometric primitives, 
 wherein the three dimensional geometric primitives are formed by a projection of the plurality of two dimensional geometric primitives to a three dimensional space, 
 wherein the projection of a first vertex of the plurality of two dimensional geometric primitives depend on a vertex position of the first vertex in the second depth map and a depth value in the first depth map for the first vertex; and 
 
   rendering an image based on the mesh representation.   
     
     
         17 . A computer program on a non-transitory medium, wherein the computer program when executed on a processor performs the method as claimed in  claim 15 . 
     
     
         18 . (canceled) 
     
     
         19 . A computer program stored on a non-transitory medium, wherein the computer program when executed on a processor performs the method as claimed in  claim 16 . 
     
     
         20 . The method of  claim 15 , further comprising determining the vertex positions by offsetting reference vertex positions of a reference division of the first depth map, wherein the offsetting for a first vertex point is based on values of the depth map for pixels within a geometric primitive of the first vertex point. 
     
     
         21 . The method of  claim 20 , wherein the reference vertex positions of the reference division form a regular grid. 
     
     
         22 . The method of  claim 20 , wherein the vertex position map indicates the vertex positions as relative positions with respect to the reference vertex positions of the reference division. 
     
     
         23 . The method of  claim 20 ,
 wherein the offsetting comprises a first offset along a first predetermined direction,   wherein the first offset is determined as an offset for which a change of depth map values along the first predetermined direction meets a criterion.   
     
     
         24 . The method of  claim 20 ,
 wherein the offsetting comprises a second offset along a second predetermined direction,   wherein the second offset is determined as an offset for which a change of depth map values along the second predetermined direction meets a criterion,   wherein the map generator circuit is arranged to generate the vertex position map so as to indicate offsets along the first predetermined direction,   wherein the map generator circuit is arranged to generate an additional vertex position map so as to indicate offsets along the second predetermined direction;   wherein the bitstream generator circuit is arranged to provide the additional vertex position map in the video bitstream.   
     
     
         25 . The method of  claim 15 , further comprising determining the vertex positions so as to align the plurality of two dimensional geometric primitives with depth transitions in the depth map. 
     
     
         26 . The apparatus of  claim 16 ,
 wherein the vertex position map indicates the vertex positions as relative vertex positions with respect to reference vertex positions of a reference division of the first depth map,   wherein the mesh generator circuit is arranged to determine absolute vertex positions in the second depth map based on the relative vertex positions and the reference vertex positions.   
     
     
         27 . The apparatus of  claim 26 ,
 wherein the video bitstream comprises an indication of the reference division,   wherein the mesh generator circuit is arranged to determine the reference vertex positions in response to the indication of the reference division.