Systems and methods for rolling shutter correction

Abstract

An image processing method includes obtaining an image frame through an imaging device over a period of time. The image frame includes a plurality of groups of pixels that are exposed to light at different time points within the period of time. The method further includes obtaining attitude information of the imaging device during the period of time, deriving positional state of an individual group of pixels in the plurality of groups of pixels based on the attitude information of the imaging device, and processing the image frame using the positional state.

What is claimed is: 
     
         1 . An image processing method comprising:
 obtaining, through an imaging device, an image frame over a period of time, wherein the image frame comprises a plurality of groups of pixels that are exposed to light at different time points within the period of time;   obtaining attitude information of the imaging device during the period of time;   deriving positional state of an individual group of pixels in the plurality of groups of pixels based on the attitude information of the imaging device; and   processing the image frame using the positional state.   
     
     
         2 . The method of  claim 1 , wherein the attitude information of the imaging device is measured by a motion capture system located remotely from the imaging device, and the motion capture system is selected from the group consisting of a vision sensor, a barometer, an ultrasonic based navigation system, an indoor positioning system, and a lidar navigation/position. 
     
     
         3 . The method of  claim 1 , wherein the imaging device is operably coupled to a movable object. 
     
     
         4 . The method of  claim 3 , wherein the movable object is an aerial vehicle, a land vehicle, a vehicle traversing water body, a mobile phone, a tablet, a laptop, or a wearable device. 
     
     
         5 . The method of  claim 3 , wherein the movable object is stationary relative to the imaging device, and wherein the attitude information are obtained from one or more sensors that are configured to measure attitude information of the movable object. 
     
     
         6 . The method of  claim 3 , wherein the movable object permits relative movement between the imaging device and the movable object, wherein a first IMU is operably coupled to the movable object and configured to measure attitude information of the movable object. 
     
     
         7 . The method of  claim 6 , wherein obtaining the attitude information of the imaging device comprises compensating the attitude information of the movable object based upon the relative movement between the movable object and the imaging device. 
     
     
         8 . The method of  claim 6 , wherein the imaging device is operably coupled to a second IMU, and the relative movement between the movable object and the imaging device is determined by data retrieved from the first IMU and the second IMU. 
     
     
         9 . The method of  claim 3 , wherein the imaging device is mounted on a stabilization system which is supported by the movable object, wherein the stabilization system permits the imaging device to rotate along one or more axes relative to the movable object, and wherein the movable object is operably coupled to a first IMU configured to measure attitude information of the movable object. 
     
     
         10 . The method of  claim 9 , wherein the attitude information of the imaging device are obtained by compensating the attitude information of the movable object along one or more rotational axes, and wherein rotation information along one or more rotational axes is provided by one or more motors of the stabilization system. 
     
     
         11 . The method of  claim 9 , wherein a second IMU is provided on a frame of the stabilization system which is rigidly connected to the imaging device, and wherein the attitude information are obtained by adjusting the attitude information from the first IMU using the attitude information from the second IMU. 
     
     
         12 . The method of  claim 3 , wherein the one or more processors are located on the movable object and/or the imaging device. 
     
     
         13 . The method of  claim 3 , wherein at least one processor for processing the image frame is located on the movable object, and at least one processor for adjusting apertures of the lens or zooming is located on the imaging device. 
     
     
         14 . The method of  claim 3 , wherein the one or more processors are located remotely from the movable object and on an external device (e.g., a remote terminal, a remote controller, a display device). 
     
     
         15 . The method of  claim 1 , wherein the imaging device comprises a video camera configured to shoot a video comprising a series of image frames with the aid of a rolling shutter. 
     
     
         16 . The method of  claim 15 , further comprising out-putting the video concurrent with shooting the video by the video camera. 
     
     
         17 . The method of  claim 15 , further comprising out-putting the video for display on an external device located remotely from the video camera. 
     
     
         18 . The method of  claim 15 , further comprising out-putting a processed image frame prior to or concurrent with completing obtaining one or more subsequent image frames of the video. 
     
     
         19 . The method of  claim 15 , wherein one or more groups of pixels of an image frame are processed prior to or concurrently with obtaining one or more subsequent image frames of the video. 
     
     
         20 . An image processing apparatus comprising:
 one or more processors; and   a non-transitory computer readable medium storing program instructions that, when executed by the one or more processors, cause the one or more processors to individually or collectively:   obtain, through an imaging device, an image frame over a period of time, wherein the image frame comprises a plurality of groups of pixels that are exposed to light at different time points within the period of time;   obtain attitude information of the imaging device during the period of time;   derive positional state of an individual group of pixels in the plurality of groups of pixels based on the attitude information of the imaging device; and   process the image frame using the positional state.