Arrangement and method for the recording and display of images of a scene and/or an object

Abstract

The invention relates to an arrangement and a method for capturing and displaying images of a scene and/or an object. Said arrangement and method are particularly suited to display the captured images in a three-dimensionally perceptible manner. The aim of the invention is to create a new possibility to take images of real scenes and/or objects with as little effort as possible and then autostereoscopically display the same in a three-dimensional fashion from two or more perspectives. Said aim is achieved by providing at least one main camera of a first camera type for recording images, at least one satellite camera of a second camera type for recording images, an image converting device which is mounted behind the cameras, and a 3D image display device, the two camera types differing from each other in at least one parameter. A total of at least three cameras is provided. Also disclosed is a method for transmitting 3D data.

1 - 39 . (canceled) 
   
   
       40 . The arrangement for the recording of images of a scene and/or an object and their display for spatial perception, comprising:
 at least one main camera of a first camera type for the recording of images;   at least two satellite cameras of a second camera type for the recording of images, with the camera types differing in at least one parameter;   an image conversion device, arranged downstream of the cameras, that receives and processes the initial image data, said image conversion device performing, among other processes, a depth or disparity recognition employing only those images recorded by cameras of the same camera type (by said at least two satellite cameras), but not the remaining images; and   a 3D image display device, connected to the image conversion device, that displays the provided image data for spatial perception without special aids, with the 3D image display device displaying at least two views.   
   
   
       41 . The arrangement as claimed in  claim 40 , wherein the two camera types differ at least in the resolution of the images to be recorded. 
   
   
       42 . The arrangement as claimed in  claim 40 , wherein the two camera types differ at least in the built-in imaging chip. 
   
   
       43 . The arrangement as claimed in  claim 40 , wherein exactly one main camera and two satellite cameras are provided. 
   
   
       44 . The arrangement as claimed in  claim 40 , wherein exactly one main camera and three satellite cameras are provided. 
   
   
       45 . The arrangement as claimed in  claim 40 , wherein exactly one main camera and five satellite cameras are provided. 
   
   
       46 . The arrangement as claimed in  claim 40 , wherein the second camera type has a lower resolution than the first camera type. 
   
   
       47 . The arrangement as claimed in  claim 43 , wherein the main camera is arranged between the satellite cameras. 
   
   
       48 . The arrangement as claimed in  claim 40 , wherein at least one partially transparent mirror is arranged in front of each of the objectives of the main camera and all satellite cameras. 
   
   
       49 . The arrangement as claimed in  claim 44 , wherein the center points of the objectives of the three satellite cameras form a triangle. 
   
   
       50 . The arrangement as claimed in  claim 49 , wherein the triangle is a isosceles triangle. 
   
   
       51 . The arrangement as claimed in  claim 49 , wherein the center point of the objective of the main camera is arranged inside said triangle, with the triangle to be understood to include its sides. 
   
   
       52 . The arrangement as claimed in  claim 44 , wherein one satellite camera and the main camera are optically arranged relative to each other in such a way that both record an image on essentially the same optical axis, for which purpose preferably at least one partially transparent mirror is arranged between the two cameras. 
   
   
       53 . The arrangement as claimed in  claim 52 , wherein the two other satellite cameras are arranged to form a straight line or a triangle together with the satellite camera associated to the main camera. 
   
   
       54 . The arrangement as claimed in  claim 40 , wherein the image conversion device generates at least two views of the scene or object recorded, and that, for generating these at least two views, the image conversion device employs, besides the depth or disparity data recognized, the image recorded by the at least one main camera and at least one more image recorded by the satellite cameras, but not necessarily the images of all cameras provided. 
   
   
       55 . The arrangement as claimed in  claim 54 , wherein one of the at least three views generated is still equal to one of the input images. 
   
   
       56 . The arrangement as claimed in  claim 40 , wherein the main camera or all main cameras, and all satellite cameras record with frame-accurate synchronization at a tolerance of maximally 100 frames per 24 hours. 
   
   
       57 . A method for the recording and display of images of a scene and/or an object, comprising the following steps:
 generating at least one n-tuple of images, with n>2, with at least two images of the n-tuple having different resolutions;   transferring the image data to an image conversion device, in which then a rectification, a color adjustment, a depth or disparity recognition and subsequent generation of further views from the n or less than n images of the said n-tuple and from the depth or disparity recognition data are carried out, with at least one view being generated that is not exactly equal to any of the n-tuple of images generated, and with the image conversion device employing, for depth or disparity recognition, only such images of the n-tuple that have the same resolution;   subsequently generating a combination of at least two different views or images in accordance with the parameter assignment of the 3D display of a 3D image display device, for spatial presentation without special aids; and   finally presenting the combined 3D image on the 3D display.   
   
   
       58 . The method as claimed in  claim 57 , wherein, for the depth or disparity recognition, those images of equal resolution are employed whose resolution has the lowest total number of pixels compared with all other resolutions provided. 
   
   
       59 . The method as claimed in  claim 58 , wherein, for depth recognition, a stack structure is established by means of a line-by-line comparison of the pre-processed initial image data of an n-tuple, precisely, of those images of the n-tuple only that have the same resolution, in such a way that first those lines of the different images of an n-tuple which have the same Y coordinate are placed in register on top of each other and then a first comparison is made, the result of the comparison being saved in one line in such a way that equal tonal values in register are saved, whereas different tonal values are deleted, which is followed by a displacement of the lines in opposite directions by specified increments of preferably ¼ to 2 pixels, the results after each increment being saved in further lines analogously to the first comparison; so that, as a result after the comparisons made for each pixel, the Z coordinate provides the information about the degree of displacement of the views relative to each other. 
   
   
       60 . The method as claimed in  claim 59 , wherein, after the establishment of the stack structure, an optimization is made in such a way that ambiguities are eliminated, and/or a reduction of the elements to an unambiguous height profile curve is carried out. 
   
   
       61 . The method as claimed in  claim 59 , wherein, after the establishment of the stack structure or after the steps described in  claim 60 , the depth is determined for at least three original images of the n-tuple, preferably in the form of depth maps. 
   
   
       62 . The method as claimed in  claim 61 , wherein, after transfer of the original images of the n-tuple and the respective depths appertaining to them, a reconstruction is carried out by inverse projection of the views of the n-tuple into the stack space by depth maps, so that die stack structure is reconstructed, and so that again different views can be subsequently generated therefrom by projection. 
   
   
       63 . The method as claimed in  claim 57 , wherein the images generated are transmitted to the image conversion device. 
   
   
       64 . The method as claimed in  claim 57 , wherein all views generated of each image by the image conversion device are transmitted to the 3D image display device. 
   
   
       65 . The method as claimed in  claim 61 , wherein the original images of the n-tuple with the respective depths appertaining to them are transmitted to the 3D image display device, after which first the reconstruction according to  claim 62  is carried out. 
   
   
       66 . The method as claimed in  claim 57 , wherein the images of the n-tuple are generated by a 3D camera system. 
   
   
       67 . The method as claimed in  claim 57 , wherein the images of the n-tuple are generated by a computer. 
   
   
       68 . The method as claimed in  claim 61 , wherein at least two depth maps differing in resolution are generated. 
   
   
       69 . A method for the transmission of 3D information for the purpose of later display for spatial perception without special aids, on the basis of at least two different views, comprising the steps of:
 proceeding from at least one n-tuple of images, with n>2, which characterize different angles of view of an object or a scene, with at least two images of the n-tuple having different resolutions;   determining the depth for at least three images; and   thereafter, at least three images of the n-tuple, together with the respective depth information, are transmitted in a transmission channel.   
   
   
       70 . The method as claimed in  claim 69 , wherein the depth information is in the form of depth maps. 
   
   
       71 . The method as claimed in  claim 69 , wherein the n-tuple of images is a quadruple of images (n=4), with three images preferably having the same resolution, whereas the fourth image has a higher resolution and preferably belongs to the images transmitted in the transmission channel. 
   
   
       72 . The method as claimed in  claim 69 , wherein at least two of the three depth maps have different resolutions. 
   
   
       73 . The method as claimed in  claim 69 , wherein the image data and the depth information are generated in the MPEG-4 format. 
   
   
       74 . The method as claimed in  claim 69 , wherein the depth information is determined only from such images of the n-tuple that have the same resolution. 
   
   
       75 . The method as claimed in  claim 74 , wherein, from the depth information determined, the depth also for at least one image of higher resolution is generated. 
   
   
       76 . The method as claimed in  claim 57 , wherein depth information determined from images of the n-tuple that have the lowest resolution provided are transformed into a higher resolution by way of edge recognitions in the at least one image of higher resolution. 
   
   
       77 . The method as claimed in  claim 57 , wherein a great number of n-tuples of images and appertaining depth information are processed in succession, so that a spatial display of moving images is made possible. 
   
   
       78 . The method as claimed in  claim 77 , wherein the great number of n-tuples of images is subjected to spatial and temporal filtering. 
   
   
       79 . A method of transmitting 3D information for the purpose of subsequent display for spatial perception without special aids, on the basis of at least two different views, comprising the steps of:
 proceeding from at least one n-tuple of images with n>2, which characterize different viewing angles of an object or a scene;   determining the depth for at least three images; and   thereafter at least three images of the n-tuple, together with the respective depth information are transmitted, in a transmission channel.   
   
   
       80 . The method of  claim 79 , wherein the depth information is in the form of depth maps. 
   
   
       81 . The method of  claim 79 , wherein the n-tuple of images is a triple of images (n=3), with the three images having the same resolution.