Electronic system and method for generating panoramic light fields

Abstract

An electronic system for generating panoramic light fields is provided. The electronic system includes a camera, a depth estimation circuit, and a light field generation circuit. The camera is configured to capture a panoramic image of a scene. The depth estimation circuit is configured to estimate panoramic depth information of the scene based on the panoramic image captured by the camera. The light field generation circuit is configured to generate a panoramic light field based on the estimated panoramic depth information.

What is claimed is: 
     
         1 . An electronic system for generating panoramic light fields, comprising:
 a camera, configured to capture a panoramic image of a scene;   a depth estimation circuit, configured to estimate panoramic depth information of the scene based on the panoramic image captured by the camera;   a light field generation circuit, configured to generate a panoramic light field based on the estimated panoramic depth information.   
     
     
         2 . The electronic system as claimed in  claim 1 , wherein the camera is mounted on a rotational mechanism used for revolving the camera;
 wherein the camera is further configured to capture a sequence of image sets of the scene from a plurality of points on a first trajectory of the camera;   wherein the camera is further configured to stitch images in each image set to obtain a stitched image;   wherein the camera is further configured to transform the stitched image to the panoramic image using an equirectangular projection method; and   wherein the panoramic image is rectangular.   
     
     
         3 . The electronic system as claimed in  claim 2 , wherein the rotational mechanism is further used for lifting or lowering the camera;
 wherein the camera is further configured to capture another sequence of image sets of the scene from a plurality of points on a second trajectory of the camera.   
     
     
         4 . The electronic system as claimed in  claim 3 , wherein the first trajectory and the second trajectory are circles. 
     
     
         5 . The electronic system as claimed in  claim 4 , wherein the first trajectory and the second trajectory have different radiuses. 
     
     
         6 . The electronic system as claimed in  claim 2 , wherein the depth estimation circuit is further configured to estimate the panoramic depth information using a convolutional neural network-based model. 
     
     
         7 . The electronic system as claimed in  claim 6 , wherein the convolutional neural network uses a distance-based kernel. 
     
     
         8 . The electronic system as claimed in  claim 1 , wherein the light field generation circuit is further configured to generate the panoramic light field using a convolutional neural network-based model. 
     
     
         9 . The electronic system as claimed in  claim 1 , wherein each light ray in the panoramic light field is addressed by two sets of cylindrical coordinates. 
     
     
         10 . The electronic system as claimed in  claim 1 , wherein each light ray in the panoramic light field is addressed by two sets of spherical coordinates. 
     
     
         11 . An electronic system for generating panoramic light fields, comprising:
 a camera, configured to capture a panoramic image of a scene;   a processor, configured to estimate panoramic depth information of the scene based on the panoramic image captured by the camera, and to generate a panoramic light field based on the estimated panoramic depth information.   
     
     
         12 . The electronic system as claimed in  claim 11 , wherein the camera is mounted on a rotational mechanism used for revolving the camera;
 wherein the camera is further configured to capture a sequence of image sets of the scene from a plurality of points on a first trajectory of the camera;   wherein the camera is further configured to stitch images in each image set to obtain a stitched image;   wherein the camera is further configured to transform the stitched image to the panoramic image using an equirectangular projection method; and   wherein the panoramic image is rectangular.   
     
     
         13 . A method for generating panoramic light fields, for use in an electronic system, wherein the electronic system comprises a camera, the method comprising:
 capturing a panoramic image of a scene by the camera;   estimating panoramic depth information of the scene based on the panoramic image; and   generating a panoramic light field based on the panoramic depth information.   
     
     
         14 . The method as claimed in  claim 13 , wherein the step of capturing the panoramic image of the scene by the camera comprises:
 revolving the camera, and capturing a sequence of image sets of the scene from a plurality of points on a first trajectory of the camera;   stitching images in each image set to obtain a stitched image; and   transforming the stitched image in the sequence to the panoramic image using an equirectangular projection method;   wherein the panoramic image is rectangular.   
     
     
         15 . The method as claimed in  claim 14 , further comprising:
 lifting or lowering the camera; and   revolving the camera, and capturing another sequence of image sets of the scene from a plurality of points on a second trajectory of the camera.   
     
     
         16 . The method as claimed in  claim 14 , further comprising:
 estimating the panoramic depth information using a convolutional neural network-based model.   
     
     
         17 . The method as claimed in  claim 16 , wherein the convolutional neural network uses a distance-based kernel. 
     
     
         18 . The method as claimed in  claim 13 , further comprising:
 generating the panoramic light field using a convolutional neural network-based model.   
     
     
         19 . The method as claimed in  claim 13 , wherein each light ray in the panoramic light field is addressed by two sets of cylindrical coordinates. 
     
     
         20 . The method as claimed in  claim 13 , wherein each light ray in the panoramic light field is addressed by two sets of spherical coordinates.