Virtual reality camera

Abstract

A method and apparatus for creating and rendering multiple-view images. A camera includes an image sensor to receive images, sampling logic to digitize the images and a processor programmed to combine the images based upon a spatial relationship between the images.

What is claimed is: 
     
       1. A hand-held camera comprising:
 a camera housing;   a camera lens mounted on said housing;   image acquisition circuitry located within said camera housing for acquiring images of fields of view via said camera lens at various orientations of said camera housing;   at least one user input panel for receiving a user request to select a panoramic or non-panoramic image capture mode; and   image processing circuitry located within said camera housing; responsive to the panoramic image capture mode selection, for at least partially combining each successively acquired image of a field of a view with previously acquired images of fields of view, on an image by image basis in real time, by determining spatial relationships between the images of fields of view, and by mapping the images of fields of view onto regions of a cylindrical surface, based on the spatial relationships.   
     
     
       2. The hand-held camera of  claim 1  wherein said image processing circuitry determines spatial relationships between the images based on at least one feature in images that at least partially overlap. 
     
     
       3. The hand-held camera of  claim 1  wherein said image processing circuitry determines spatial relationships between the images based on cross-correlations of images that at least partially overlap. 
     
     
       4. The hand-held camera of  claim 1  wherein said image processing circuitry determines spatial relationships between the images based on the orientations of said camera housing during image acquisition. 
     
     
       5. The hand-held cameral of  claim 4  further comprising a sensor for detecting the orientations of said camera housing. 
     
     
       6. The hand-held camera of  claim 5  wherein said image acquisition circuitry uses orientation information from said sensor to automatically determine fields of view for which to acquire images thereof. 
     
     
       7. The hand-held camera of  claim 1  wherein the camera is a video camera and wherein sampling logic digitizes the images at a predetermined rate. 
     
     
       8. A hand-held camera comprising:
 a camera housing;   a camera lens mounted on said housing;   a display mounted on said camera housing;   image acquisition circuitry located within said cameral housing for acquiring images of fields of view via said camera lens at various orientations of said camera housing;   image processing circuitry located within said camera housing for at least partially combining each successively acquired image of a field of view with previously acquired images of fields of view, on an image by image basis in real time, by determining spatial relationships between the images of fields of view, and by mapping the images of fields of view onto regions of a cylindrical surface, based on spatial relationships;   at least one user input panel to select a panoramic or non-panoramic image view mode, and to receive a user request to display a spatial region of the cylindrical panoramic image on said display; and   view control circuitry, located within said camera housing and responsive to the panoramic view mode, to display a spatial region of the cylindrical panoramic image on said display, wherein said view control circuitry selects the spatial region of the cylindrical panoramic image based upon the user request.   
     
     
       9. The hand-held camera of  claim 8  wherein said view control circuitry selects the spatial region of the cylindrical panoramic image to be displayed on said display based upon an orientation of said housing. 
     
     
       10. The hand-held camera of  claim 9  further comprising a sensor for detecting the orientation of said camera housing. 
     
     
       11. The hand-held camera of  claim 8  further comprising a sensor for detecting the orientation of said camera housing. 
     
     
       12. The hand-held camera of  claim 8  wherein said user input panel receives user requests to pan about a panoramic image. 
     
     
       13. The hand-held camera of  claim 12  wherein said user input panel comprises left, right, up and down buttons. 
     
     
       14. The hand-held camera of  claim 12  further comprising a sensor for detecting the orientation of said camera housing. 
     
     
       15. The hand-held camera of  claim 8  wherein said user input panel receives user requests to zoom in and out of a panoramic image. 
     
     
       16. The hand-held camera of  claim 15  wherein said user input panel comprises zoom in and zoom out buttons. 
     
     
       17. The hand-held camera of  claim 15  further comprising a sensor for detecting the orientation of said camera housing. 
     
     
       18. A method for providing cylindrical panoramic images comprising:
 selecting a panoramic or non-panoramic image capture mode;   acquiring images of fields of view at various orientations of a camera; and   when the panoramic image capture mode is selected, at least partially combining each successively acquired image of a field of view with previously acquired images of fields of view, on an image by image basis in real time, comprising:
 determining spatial relationships between the image of fields of view; and 
 mapping the images of fields of view onto regions of a cylindrical surface, based on the spatial relationships. 
   
     
     
       19. The method of  claim 18  wherein said determining is based on at least one feature in images that at least partially overlap. 
     
     
       20. The method of  claim 18  wherein said determining is based on cross-correlations of images that at least partially overlap. 
     
     
       21. The method of  claim 18  wherein said determining is based on the orientations of the cameral during image acquisitions. 
     
     
       22. The method of  claim 21  further comprising detecting the orientation of said camera housing. 
     
     
       23. The method of  claim 22  further comprising automatically determining fields of view for which to acquire images thereof, based on detected orientation information. 
     
     
       24. A method for providing cylindrical panoramic images comprising:
 acquiring images of fields of view at various orientations of a camera;   at least partially combining each successively acquired image of a fields of view with previously acquired images of fields of view, on an image by image basis in real time, comprising:
 determining spatial relationships between the images of fields of view; and 
 mapping the images of fields of view onto regions of a cylindrical surface, based on the spatial relationships; 
   selecting a panoramic or non-panoramic image view mode;   when the panoramic image view mode is selected,
 receiving a user request to display a spatial region of a cylindrical panoramic image; and 
 displaying the spatial region of the cylindrical panoramic image. 
   
     
     
       25. The method of  claim 24  further comprising selecting the spatial region of the cylindrical panoramic image to be displayed based upon an orientation of the camera. 
     
     
       26. The method of  claim 25  further comprising detecting the orientation of said camera housing. 
     
     
       27. A hand-held camera comprising:
 a camera housing;   a camera lens mounted on said housing;   image acquisition circuitry located within said camera housing for acquiring images of fields of view via said camera lens at various orientations of said camera housing;   at least one user input panel for receiving a user request to select a panoramic or non-panoramic image capture mode; and   image processing circuitry located within said camera housing, responsive to the panoramic image capture mode selection, for at least partially combining each successively acquired image of a field of view with previously acquired images of fields of view, on an image by image basis in real time, by determining spatial relationships between the images of fields of view, and by mapping the images of fields of view onto regions of a spherical surface, based on the spatial relationships.   
     
     
       28. The hand-held camera of  claim 27  herein the camera is a video camera and wherein sampling logic digitizes the images at a predetermined rate. 
     
     
       29. A hand-held camera comprising:
 a careen housing;   a camera lens mounted on said housing;   image acquisition circuitry located within said camera housing for acquiring images of fields of view via said camera lens at various orientations of said camera housing;   at least one user input panel for receiving a user request to select a panoramic or non-panoramic image capture mode; and   image processing circuitry located within said camera housing, responsive to the panoramic image capture mode selection, for at least partially combining each successively acquired images of a field of view with previously acquired images of fields of view, on an image by image basis in real time, by mapping the images of fields of view onto regions of a cylindrical surface, based on spatial relationships between the images of fields of view.   
     
     
       30. The hand-held camera of  claim 29  wherein the camera is a video camera and wherein sampling logic digitizes the images at a predetermined rate. 
     
     
       31. A hand-held camera comprising:
 a camera housing;   a camera lens mounted on said housing;   image acquisition circuitry located within said camera housing for acquiring images of fields of view via said camera lens at various orientations of said camera housing;   at least one user input panel for receiving a user request to select a panoramic or non-panoramic image capture mode; and   image processing circuitry located within said camera housing, responsive to the panoramic image capture mode selection, for at least partially combining each successively acquired image of a field of view with previously acquired fields of view, on an image by image basis in real time, by napping the images of fields of view onto regions of a spherical surface, based on spatial relationships between the images of fields of view.   
     
     
       32. The hand-held camera of  claim 31  wherein the camera is a video camera and wherein sampling logic digitizes the images at a predetermined rate. 
     
     
       33. A method for providing spherical panoramic images comprising:
 selecting a panoramic or non-panoramic image capture mode;   acquiring images of fields of view at various orientations of a camera; and   when the panoramic image capture mode is selected, at least partially combining each successively acquired image of a field of view with previously acquired images of fields of view, on an image by image basis in real time, comprising:
 determining spatial relationships between the images of fields of view; and 
 mapping the images of fields of view onto regions of a spherical surface, based on the spatial relationships. 
   
     
     
       34. A method for providing cylindrical panoramic images comprising:
 selecting a panoramic or non-panoramic image capture mode;   acquiring images of fields of view at various orientations of a camera; and   when the panoramic image capture mode is selected, at least partially combining each successively acquired image of a field of view with previously acquired images of fields of view, on an image by image basis in real time, comprising mapping the images of fields of view onto regions of a cylindrical surface, based on spatial relationships between the images of fields of view.   
     
     
       35. A method for providing spherical panoramic images comprising:
 selecting a panoramic or non-panoramic image capture mode;   acquiring images of fields of view at various orientations of a camera; and   when the panoramic image capture mode is selected, at least partially combining each successively acquired image of a field of view with previously acquired image of fields of view, on an image by image basis in real time, comprising mapping the images of fields of view onto regions of a spherical surface, based on spatial relationships between the images of fields of view.   
     
     
       36. A non-transitory computer readable medium storing instructions that, when executed by a computing device, cause the computing device to perform the steps of:
 sensing selection of a panoramic image capture mode;   acquiring images at various orientations of a camera; and   responsive to the selection of the panoramic image capture mode, at least partially combining at least one successively acquired image with one or more previously acquired images, on an image-by-image basis, comprising:
 determining spatial relationships between the images; and 
 mapping the images onto regions of a virtual three-dimensional surface, based on the spatial relationships. 
   
     
     
       37. A non-transitory computer readable medium of claim 36 further storing instructions that, when executed by the computing device, cause the computing device to perform the steps of sensing an orientation of the camera to determine when a next image in the panoramic image capture mode is to be acquired based at least in part on a camera orientation. 
     
     
       38. The non-transitory computer readable medium of claim 37, wherein the orientation of the camera includes at least one orientation selected from the group consisting of a pitch of the camera, a roll of the camera, and a yaw of the camera. 
     
     
       39. The non-transitory computer readable medium of claim 37 further storing instructions that, when executed by the computing device, further cause the computing device to perform the step of generating a signal to indicate that the next image is to be acquired. 
     
     
       40. The non-transitory computer readable medium of claim 37, wherein the sensing to determine when a next image is to be acquired is further based at least in part on an angle of view of the camera and a distance between the camera and a subject in successive images. 
     
     
       41. The non-transitory computer readable medium of claim 36 further storing instructions that, when executed by the computing device, cause the computing device to perform the steps of collecting image information for each acquired image, and associating the image information for each acquired image with that image, the image information including a spatial location of an acquired image at least relative to spatial locations of other acquired images. 
     
     
       42. A method for providing panoramic images performed by a camera having a processor, the method comprising:
 selecting a panoramic or non-panoramic image capture mode;   acquiring images of fields of view at various orientations of the camera; and   if the panoramic image capture mode is selected, at least partially combining by the processor of the camera each successively acquired image of a field of view with previously acquired images of fields of view, on an image-by-image basis, comprising:
 determining at least one spatial relationship between the images of fields of view; and 
 mapping the images of fields of view onto regions of a virtual three-dimensional surface based at least in part on the at least one spatial relationship. 
   
     
     
       43. The method of claim 42, wherein the virtual three-dimensional surface is a spherical surface. 
     
     
       44. The method of claim 42, wherein the virtual three-dimensional surface is a cylindrical surface. 
     
     
       45. The method of claim 42, wherein determining at least one spatial relationship comprises determining at least one feature in images that at least partially overlap. 
     
     
       46. The method of claim 42, wherein determining at least one spatial relationship comprises determining an orientation of the camera during each image acquisition. 
     
     
       47. The method according to claim 46, wherein the camera has a sensor capable of detecting the orientation of the camera based at least in part on a magnetic field of the earth. 
     
     
       48. A method performed by a camera having a processor, the method for providing panoramic images, comprising:
 enabling a selection of a panoramic image capture mode;   acquiring images of fields of view at various orientations of the camera; and   at least partially combining, by the processor of the camera, each successively acquired image of a field of view with a previously acquired image of a field of view on an image-by-image basis in the panoramic image capture mode, comprising:
 determining at least one spatial relationship between the images of fields of view; and 
 mapping the images of fields of view onto regions of a surface based at least in part on the at least one spatial relationship. 
   
     
     
       49. The method of claim 48, wherein determining at least one spatial relationship comprises determining based on at least one feature that at least partially overlaps in successively acquired images of fields of view. 
     
     
       50. The method of claim 48, wherein determining at least one spatial relationship comprises determining based at least in part on an orientation of the camera during image acquisition. 
     
     
       51. The method of claim 50, further comprising automatically determining fields of view for which to acquire images based at least in part on a determined orientation of the camera. 
     
     
       52. A non-transitory computer readable medium storing instructions that, when executed by a computing device, cause the computing device to perform the steps of:
 selecting a panoramic or non-panoramic image capture mode;   acquiring images of fields of view at various orientations of a camera; and   in response to selection of the panoramic image capture mode, at least partially combining each successively acquired image of a field of view with previously acquired images of fields of view, on an image by image basis, comprising mapping the images of fields of view onto regions of a smooth surface, based on spatial relationships between the images of fields of view.   
     
     
       53. The non-transitory computer readable medium of claim 52, wherein the smooth surface is a spherical surface. 
     
     
       54. The non-transitory computer readable medium of claim 52, wherein the smooth surface is a cylindrical surface. 
     
     
       55. The non-transitory computer readable medium of claim 52, wherein the camera is a video camera and wherein sampling logic digitizes the images at a predetermined rate. 
     
     
       56. The non-transitory computer readable medium of claim 36, wherein the three-dimensional surface is a cylinder. 
     
     
       57. The non-transitory computer readable medium of claim 36, wherein the spatial relationships comprise correlation between portions of the images. 
     
     
       58. The method of claim 42, wherein the at least one spatial relationship comprises correlation between portions of the images of fields of view. 
     
     
       59. The method of claim 48, wherein the surface is a cylinder. 
     
     
       60. The method of claim 48, wherein the at least one spatial relationship comprises correlation between portions of the images of fields of view. 
     
     
       61. The non-transitory computer readable medium of claim 52, wherein the spatial relationships comprise correlation between portions of the images of fields of view. 
     
     
       62. A camera for providing panoramic images, the camera comprising:
 a processor and a memory;   a mode component configured to enable a selection of a panoramic image capture mode;   an image acquisition component configured to acquire images of fields of view at various orientations of the camera; and   an image combining component configured to:
 determine the camera is in the panoramic image capture mode; and 
 at least partially combine, on an image-by-image basis using the processor, two or more successively acquired images from the image acquisition component, the combining performed by:
 determining at least one spatial relationship between the successively acquired images; and 
 mapping the successively acquired images onto regions of a surface based at least in part on the at least one spatial relationship. 
 
   
     
     
       63. The camera of claim 62, wherein determining at least one spatial relationship comprises determining based on at least one feature that at least partially overlaps in the successively acquired images. 
     
     
       64. The camera of claim 62, wherein determining at least one spatial relationship comprises determining based at least in part on an orientation of the camera during image acquisition. 
     
     
       65. The camera of claim 64, further comprising automatically determining fields of view for which to acquire images based at least in part on a determined orientation of the camera. 
     
     
       66. The camera of claim 62, wherein the surface is a cylinder. 
     
     
       67. The camera of claim 62, wherein the at least one spatial relationship comprises a correlation between portions of the successively acquired images.