Method for interactively viewing full-surround image data and apparatus therefor

Abstract

A method of modeling of the visible world using full-surround image data includes steps for selecting a view point within a p-surface, selecting a direction of view within the p-surface, texture mapping full-surround image data onto the p-surface such that the resultant texture map is substantially equivalent to projecting full-surround image data onto the p-surface from the view point to thereby generate a texture mapped p-surface, and displaying a predetermined portion of the texture mapped p-surface. An apparatus for implementing the method is also described.

1. A method of modeling of the visible world using full-surround image data, said method comprising:
 selecting a view point within a p-surface, wherein the p-surface comprises polygons approximating a partial sphere; 
 selecting a direction of view within the p-surface; 
 texture mapping full-surround image data onto said p-surface such that the resultant texture map is substantially equivalent to projecting full-surround image data onto the p-surface from said view point to thereby generate a texture mapped p-surface; and 
 displaying a predetermined portion of said texture mapped p-surface. 
 
   
   
     2. The method of  claim 1 , wherein the p-surface comprises one or more polygons such that there exists a half-space for each polygon, and wherein the intersection of all such half-spaces includes at least one point in common. 
   
   
     3. The method of  claim 2 , wherein a point is within the p-surface if it is included in the intersection. 
   
   
     4. The method of  claim 1 , wherein the p-surface comprises one or more polygons, and wherein a point is within the p-surface if it is included in the union of a given set of half-planes, wherein the set includes no more than one half-plane per polygon. 
   
   
     5. The method of  claim 1 , wherein the p-surface comprises one or more polygons, and wherein a point is within the p-surface if it is included in the intersection of a given set of half-planes, wherein the set includes no more than one half-plane per polygon. 
   
   
     6. The method of  claim 1 , wherein the full-surround image data is a sample of incoming image data. 
   
   
     7. A method of modeling a hemispheric view, said method comprising:
 capturing a first texture p-surface data set approximating a first hemisphere portion derived from a distorted view captured from a first wide-angle lens, said first texture p-surface data set comprising at least a portion of full-surround data, wherein the full-surround data includes a partial hemisphere; 
 selecting a view point within the p-surface; 
 selecting a direction of view within the p-surface; 
 texture mapping the full-surround data to a triangulation approximating the first hemisphere onto the p-surface substantially equivalent to projecting the full-surround data onto the p-surface from said view point; 
 generating a texture mapped p-surface corresponding to the selected view; and 
 displaying the selected view of the texture mapped p-surface. 
 
   
   
     8. The method of  claim 7 , wherein the full-surround data includes a full hemisphere. 
   
   
     9. The method of  claim 7 , further comprising steps of:
 capturing a second texture p-surface data set approximating a second hemisphere portion derived from a second wide-angle lens, said second texture p-surface data set comprising a portion of full-surround data; and 
 combining the first p-surface texture data set and the second p-surface texture data set to generate the full-surround data. 
 
   
   
     10. The method of  claim 9 , wherein the full-surround data includes a full sphere. 
   
   
     11. The method of  claim 9 , wherein the full-surround data includes a partial sphere. 
   
   
     12. The method of  claim 7 , further comprising steps of:
 capturing a second texture p-surface data set approximating a second hemisphere portion derived from the first wide-angle lens after movement to a second view, said second p-surface texture data set comprising a portion of full-surround data; and 
 combining the first texture p-surface data set and the second texture p-surface data set to generate the full-surround data. 
 
   
   
     13. A method of modeling an image from a wide-angle lens, said method comprising:
 texture mapping image data from the wide-angle lens onto a triangulation of at least a portion of a first hemisphere of full-surround data onto a p-surface wherein the full-surround data includes a partial hemisphere; 
 selecting a viewpoint within the p-surface; 
 selecting a direction of view within the p-surface; 
 selecting a perspective of view; 
 generating a texture mapped p-surface corresponding to the selected view from the selected perspective using the full-surround data; and 
 displaying the generated view of the texture mapped p-surface. 
 
   
   
     14. The method of  claim 13 , wherein the full-surround data includes a full hemisphere. 
   
   
     15. The method of  claim 14 , wherein the full-surround data includes a full sphere. 
   
   
     16. The method of  claim 14 , wherein the full-surround data includes a partial sphere. 
   
   
     17. The method of  claim 14 , wherein the full-surround data includes a full sphere. 
   
   
     18. The method of  claim 14 , wherein the full-surround data includes a partial sphere. 
   
   
     19. The method of  claim 13 , further comprising the steps of:
 texture mapping image data from the wide-angle lens onto a triangulation approximating at least a portion of a second hemisphere of full-surround data onto a p-surface; 
 combining the full- surround data onto a combined p-surface of the portions of the first hemisphere and the second hemisphere to provide a full-surround data set of at least a portion of a sphere including more than a hemisphere. 
 
   
   
     20. The method of  claim 13 , further comprising the steps of:
 texture mapping image data from a second wide-angle lens onto a triangulation approximating at least a portion of a second hemisphere of full-surround data onto a p-surface; 
 combining the full-surround data onto a combined p-surface of the first hemisphere and the second hemisphere to provide a full-surround data set of at least a portion of a sphere including more than a hemisphere. 
 
   
   
     21. The method of  claim 13 , further comprising the steps of:
 texture mapping image data from a nth wide-angle lens onto a triangulation approximating at least portion of a nth hemisphere of full-surround data onto a p-surface, wherein n designates one of a y number of wide-angle lens which collectively capture overlapping parts of at least a portion of the y hemispheres of image data; 
 combining the full-surround data onto a combined p-surface of the first hemisphere and the nth hemisphere to provide a full-surround data set of at least a portion of a sphere including more than a hemisphere. 
 
   
   
     22. A method of modeling of the visible world using full-surround image data, said method comprising:
 texture mapping full-surround image data onto a p-surface such that the resultant texture map is substantially equivalent to projecting the full-surround image data onto the p-surface from a point of projection to thereby generate a texture mapped p-surface; 
 selecting a direction of view from a view point; and 
 displaying a portion of said texture mapped p-surface based on the selecting; 
 wherein the p-surface comprises polygons approximating a partial sphere. 
 
   
   
     23. The method of  claim 22 , wherein the p-surface comprises one or more polygons such that there exists a half-space for each polygon, and wherein the intersection of all such half-spaces includes at least one point in common. 
   
   
     24. The method of  claim 23 , wherein a point is within the p-surface if it is included in the intersection. 
   
   
     25. The method of  claim 22 , wherein the p-surface comprises one or more polygons, and wherein a point is within the p-surface if it is included in the union of a given set of half-planes, wherein the set includes no more than one half-plane per polygon. 
   
   
     26. The method of  claim 22 , wherein the p-surface comprises one or more polygons, and wherein a point is within the p-surface if it is included in the intersection of a given set of half-planes, wherein the set includes no more than one half-plane per polygon. 
   
   
     27. The method of  claim 22 , wherein the full-surround image data is a sample of incoming image data. 
   
   
     28. The method of  claim 22 , wherein the point of projection comprises the view point. 
   
   
     29. The method of  claim 22 , wherein the point of projection is different from the view point. 
   
   
     30. The method of  claim 22 , wherein the selecting a direction of view is performed by a user. 
   
   
     31. The method of  claim 22 , wherein the selecting a direction of view is performed by a user using an interactive user interface. 
   
   
     32. The method of  claim 22 , wherein the view point is within the p-surface. 
   
   
     33. A method of modeling a hemispheric view, said method comprising:
 capturing a first texture p-surface data set approximating a first hemisphere portion derived from a distorted view captured from a first wide- angle lens, said first texture p-surface data set comprising at least a first portion of full-surround image data; 
 texture mapping the full-surround image data to a triangulation approximating the first hemisphere portion onto a p-surface in a manner substantially equivalent to projecting the full-surround image data onto the p-surface from a point of projection; 
 selecting a direction of view from a view point; and displaying a portion of the texture mapped p-surface based on the selecting 
 wherein the full-surround image data includes at least a partial hemisphere. 
 
   
   
     34. The method of  claim 33 , wherein the full-surround image data includes a full hemisphere. 
   
   
     35. The method of  claim 33 , wherein the point of projection comprises the view point. 
   
   
     36. The method of  claim 33 , wherein the point of projection is different from the view point. 
   
   
     37. The method of  claim 33 , wherein the selecting a direction of view is performed by a user. 
   
   
     38. The method of  claim 33 , wherein the selecting a direction of view is performed by a user using an interactive user interface. 
   
   
     39. The method of  claim 33 , wherein the view point is within the p-surface. 
   
   
     40. The method of  claim 33 , further comprising steps of:
 capturing a second texture p-surface data set approximating a second hemisphere portion derived from a second wide-angle lens, said second texture p-surface data set comprising at least a second portion of the full-surround image data; and 
 combining the first p-surface texture data set and the second p-surface texture data sets to generate the full-surround image data. 
 
   
   
     41. The method of  claim 40 , wherein the full-surround image data includes a full sphere. 
   
   
     42. The method of  claim 40 , wherein the full-surround image data includes a partial sphere. 
   
   
     43. The method of  claim 33 , further comprising steps of:
 capturing a second texture p-surface data set approximating a second hemisphere portion derived from the first wide-angle lens after movement from a first view to a second view, said second p-surface texture data set comprising at least a second portion of the full-surround image data; and 
 combining the first texture p-surface data set and the second texture p-surface data set to generate the full-surround image data. 
 
   
   
     44. A method of modeling an image from a wide-angle lens, said method comprising:
 texture mapping image data from the wide-angle lens onto a triangulation of at least a portion of a first hemisphere of full-surround image data onto a p-surface; 
 selecting a direction of view from a viewpoint within the p-surface; selecting a perspective of view; 
 generating a texture mapped p-surface corresponding to the selected direction of view from the selected perspective using the full-surround image data; and 
 displaying a portion of the texture mapped p-surface based on the selecting a direction of view and the selecting a perspective of view; 
 wherein the full-surround image data includes at least a partial hemisphere. 
 
   
   
     45. The method of  claim 44 , wherein the full-surround image data includes a full hemisphere. 
   
   
     46. The method of  claim 45 , wherein the full-surround image data set includes a partial sphere. 
   
   
     47. The method of  claim 45 , wherein the full-surround image data includes a full sphere. 
   
   
     48. The method of  claim 45 , wherein the full-surround image data includes a partial sphere. 
   
   
     49. The method of  claim 44 , further comprising the steps of:
 texture mapping image data from the wide-angle lens onto a triangulation approximating at least a portion of a second hemisphere of full-surround image data onto a p-surface to generate a combined p-surface comprised of the texture mapped image data approximating the at least portions of the first hemisphere and the second hemisphere, wherein the full-surround image data is comprised of at least a portion of a sphere including more than a hemisphere. 
 
   
   
     50. The method of  claim 45 , wherein the full-surround image data set includes a full sphere. 
   
   
     51. The method of  claim 44 , further comprising the steps of:
 texture mapping image data from a second wide-angle lens onto a triangulation approximating at least a portion of a second hemisphere of full-surround image data onto a p-surface to generate a combined p-surface comprised of the texture mapped image data approximating the at least a portion of the first hemisphere and the second hemisphere, wherein the full-surround image data of at least a portion of a sphere including more than a hemisphere. 
 
   
   
     52. The method of  claim 44 , further comprising the steps of:
 texture mapping image data from an nth wide-angle lens onto a triangulation approximating at least portion of an nth hemisphere of the full-surround image data onto a p-surface, wherein n designates one of a plurality y number of wide-angle lenses which collectively capture overlapping parts of at least a portion of the y hemispheres of the full-surround image data, wherein y>2; 
 combining the full-surround image data onto a combined p-surface of the first hemisphere and the nth hemisphere to provide a full-surround image data set is comprised of at least a portion of a sphere including more than a hemisphere. 
 
   
   
     53. A method, comprising:
 texture mapping full-surround image data onto a p-surface to generate a texture map that is substantially equivalent to projecting the image data onto the p-surface from a point of projection to thereby generate a texture mapped p-surface; 
 selecting a direction of view from a view point; and displaying a portion of the texture mapped p-surface based on the selecting; 
 wherein the p-surface comprises polygons approximating at least a portion of a sphere. 
 
   
   
     54. The method of  claim 53 , wherein the full-surround image data is derived from source image data generated from visible stimuli. 
   
   
     55. The method of  claim 54 , further comprising capturing the source image data using a single camera equipped with a fish-eye lens. 
   
   
     56. the method of  claim 55 , further comprising generating the full-surround image data by sampling points of the source image data defining the visible world. 
   
   
     57. The method of  claim 54 , further comprising capturing the source image data using n cameras, wherein n>2. 
   
   
     58. The method of  claim 57 , further comprising generating the full-surround image data by sampling points of the source image data defining the visible world. 
   
   
     59. The method of  claim 54 , wherein the source image data is captured using n cameras, wherein n>2. 
   
   
     60. The method of  claim 59 , wherein n is at least 6. 
   
   
     61. The method of  claim 59 , wherein the full-surround image data is derived by sampling points of the source image data defining the visible world. 
   
   
     62. The method of  claim 54 , wherein the source image data is captured using a single camera equipped with a fish-eye lens. 
   
   
     63. The method of  claim 62 , wherein the full-surround image data is derived by sampling points of the source image data defining the visible world. 
   
   
     64. The method of  claim 54 , wherein the p-surface comprises polygons approximating a cube. 
   
   
     65. The method of  claim 54 , wherein the p-surface comprises polygons approximating a tetrahedron. 
   
   
     66. The method of  claim 54 , wherein the p-surface comprises polygons approximating an ellipsoid. 
   
   
     67. The method of  claim 54 , wherein the p-surface comprises polygons approximating a dodecahedron. 
   
   
     68. The method of  claim 54 , wherein the p-surface comprises polygons approximating a cylinder. 
   
   
     69. The method of  claim 54 , wherein the full surround image data comprises an at least substantially spherical image data set. 
   
   
     70. The method of  claim 54 , wherein the full surround image data comprises an at least substantially cylindrical image data set. 
   
   
     71. The method of  claim 54 , wherein the full surround image data comprises an at least substantially hemispherical image data set. 
   
   
     72. The method of  claim 54 , wherein the displaying is performed using circular perspective. 
   
   
     73. The method of  claim 54 , wherein the method is performed using a 3D computer graphics system. 
   
   
     74. The method of  claim 54 , wherein the method is performed using a computer graphics system employing a conventional graphics library such as OpenGL. 
   
   
     75. The method of  claim 54 , further comprising a user selectively moving the view point toward and away from a surface area of the p-surface to thereby alter a perspective used for the displaying. 
   
   
     76. The method of  claim 54 , wherein the method is performed using a standard 3D computer graphics system. 
   
   
     77. The method of  claim 54 , wherein the displaying is performed using primitives of a 3D computer graphics system. 
   
   
     78. The method of  claim 54 , further comprising a user using an interactive viewing system to pan around imagery represented by the full-surround image data to give the user the effect of looking around at the imagery as if immersed within the p-surface, to thereby provide a virtual reality representation of the visible world. 
   
   
     79. The method of  claim 54 , further comprising a user using an input device to move the view point in and out along an axis relative to the p-surface to thereby effectively alter the user's view of the displayed portion of the texture-mapped p-surface. 
   
   
     80. The method of  claim 54 , further comprising a user using an input device to control rotation of the p-surface to thereby effectively allow the user to look around imagery represented by the full-surround image data of the texture-mapped p-surface. 
   
   
     81. The method of  claim 54 , further comprising:
 a user using an input device to move the view point in and out along an axis relative to the p-surface to thereby effectively alter the user's view of the displayed portion of the texture-mapped p-surface; and 
 the user using the input device to control rotation of the p-surface to thereby effectively allow the user to look around imagery represented by the full-surround image data of the texture-mapped p-surface. 
 
   
   
     82. The method of  claim 54 , further comprising providing a user an option to zoom in on imagery represented by the full-surround image data of the texture-mapped p-surface. 
   
   
     83. The method of  claim 54 , further comprising a user using an input device to control the direction of view to thereby effectively allow the user to look around imagery represented by the full-surround image data of the texture-mapped p-surface. 
   
   
     84. The method of  claim 54 , further comprising using the method to generate a plurality of p-surfaces using different respective sets of full-surround image data. 
   
   
     85. The method of  claim 54 , wherein the method is implemented in a multimedia entertainment system. 
   
   
     86. The method of  claim 54 , further comprising enabling multiple users using independent viewing systems to independently cause to be displayed and to view any selected portion of the texture-mapped p-surface. 
   
   
     87. The method of  claim 53 , wherein the point of projection comprises the view point. 
   
   
     88. The method of  claim 53 , wherein the point of projection is different from the view point. 
   
   
     89. The method of  claim 53 , wherein the view point is within the p-surface. 
   
   
     90. The method of  claim 53 , wherein the selecting a direction of view is performed by a user. 
   
   
     91. The method of  claim 53 , wherein the full-surround image data is derived by sampling points defining the visible world. 
   
   
     92. The method of  claim 53 , further comprising generating the full-surround image data by sampling points defining the visible world. 
   
   
     93. The method of  claim 53 , wherein the p-surface comprises polygons approximating a cube. 
   
   
     94. The method of  claim 53 , wherein the p-surface comprises polygons approximating a tetrahedron. 
   
   
     95. The method of  claim 53 , wherein the p-surface comprises polygons approximating an ellipsoid. 
   
   
     96. The method of  claim 53 , wherein the p-surface comprises polygons approximating a dodecahedron. 
   
   
     97. The method of  claim 53 , wherein the p-surface comprises polygons approximating a cylinder. 
   
   
     98. The method of  claim 53 , wherein the displaying comprises displaying a user-selected portion of the texture-mapped p-surface. 
   
   
     99. The method of  claim 53 , further comprising a user selecting the portion of the texture-mapped p-surface to be displayed during the displaying. 
   
   
     100. The method of  claim 53 , further comprising a user selecting the portion of the texture-mapped p-surface to be displayed during the displaying via an interactive user interface. 
   
   
     101. The method of  claim 100 , wherein the user selecting the portion of the texture-mapped p-surface to be displayed is performed using a zoom function via the interactive user interface. 
   
   
     102. The method of  claim 53 , wherein the full surround image data comprises an at least approximately spherical image data set. 
   
   
     103. The method of  claim 53 , wherein the full surround image data comprises an at least approximately cylindrical image data set. 
   
   
     104. The method of  claim 53 , wherein the full surround image data comprises an at least approximately hemispherical image data set. 
   
   
     105. The method of  claim 53 , wherein the displaying is performed using linear perspective. 
   
   
     106. The method of  claim 105 , wherein the user is provided an option of selecting any one of linear perspective and circular perspective for the displaying. 
   
   
     107. The method of  claim 105 , wherein the user is provided an option of selecting any one of linear perspective, elliptical perspective, and circular perspective for the displaying. 
   
   
     108. The method of  claim 53 , wherein the texture-mapping is performed using stereographic projection. 
   
   
     109. The method of  claim 53 , wherein the displaying is performed using any user-selected one of linear perspective and circular perspective, at the user's option. 
   
   
     110. The method of  claim 53 , wherein the displaying is performed using any user-selected one of linear perspective, elliptical perspective, and circular perspective, at the user's option. 
   
   
     111. The method of  claim 53 , further comprising a user selectively moving the view point toward and away from a surface area of the p-surface to thereby alter a perspective used for the displaying. 
   
   
     112. The method of  claim 53 , wherein the method is performed using a 3D computer graphics system native to a personal computer. 
   
   
     113. The method of  claim 53 , further comprising a user using an interactive viewing system to pan around imagery represented by the full- surround image data to give the user the effect of looking around at the imagery as if immersed within the p-surface, to thereby provide a virtual reality representation of the visible world. 
   
   
     114. The method of  claim 53 , further comprising a user using an input device to move the view point in and out along an axis relative to the p-surface to thereby effectively alter the user's view of the displayed portion of the texture-mapped p-surface. 
   
   
     115. The method of  claim 53 , further comprising a user using an input device to control rotation of the p-surface to thereby effectively allow the user to look around imagery represented by the full-surround image data of the texture-mapped p-surface. 
   
   
     116. The method of  claim 115 , further comprising linking the plurality of p-surfaces in such a manner as to enable a user to hop amongst the plurality of p-surfaces to simulate a tour thereof. 
   
   
     117. The method of  claim 53 , further comprising:
 a user using an input device to move the view point in and out along an axis relative to the p-surface to thereby effectively alter the user's view of the displayed portion of the texture-mapped p-surface; and 
 the user using the input device to control rotation of the p-surface to thereby effectively allow the user to look around imagery represented by the full-surround image data of the texture-mapped p-surface. 
 
   
   
     118. The method of  claim 53 , further comprising providing a user an option to zoom in on imagery represented by the full-surround image data of the texture-mapped p-surface. 
   
   
     119. The method of  claim 53 , further comprising a user using an input device to control the direction of view to thereby effectively allow the user to look around imagery represented by the full-surround image data of the texture-mapped p-surface. 
   
   
     120. The method of  claim 53 , further comprising using the method to generate a plurality of p-surfaces using different respective sets of full-surround image data. 
   
   
     121. The method of  claim 120 , further comprising linking the plurality of p-surfaces in such a manner as to enable a user to hop amongst the plurality of p-surfaces to simulate a tour thereof. 
   
   
     122. The method of  claim 53 , wherein the method is implemented in a multimedia entertainment system. 
   
   
     123. The method of  claim 53 , further comprising enabling multiple users using independent viewing systems to independently cause to be displayed and to view any selected portion of the texture-mapped p-surface.