Image processing method and apparatus

Abstract

An image processing method includes acquiring an image frame; tracking a face region of a user based on first prior information obtained from at least one previous frame of the image frame; based on a determination that tracking of the face region based on the first prior information has failed, setting a scan region in the image frame based on second prior information obtained from the at least one previous frame; and detecting the face region in the image frame based on the scan region.

What is claimed is: 
     
         1 . An eye tracking method comprising:
 acquiring an image frame using a camera or an image sensor;   tracking a target region including eyes and a nose of a user based on first prior information obtained from at least one previous frame of the image frame;   based on a determination that tracking of the eyes and the nose based on the first prior information has failed, setting a scan region in the image frame based on second prior information obtained from the at least one previous frame; and   detecting the target region in the image frame based on the scan region.   
     
     
         2 . The eye tracking method of  claim 1 , wherein the second prior information comprises information related to at least one previous scan region based on which detection of the target region is performed in the at least one previous frame, and
 the setting comprises resetting the scan region to a region to which the at least one previous scan region is expanded.   
     
     
         3 . The eye tracking method of  claim 2 , wherein the resetting comprises resetting the scan region by sequentially expanding a size of a window for setting a previous scan region, based on whether tracking of the target region in the previous scan region has failed. 
     
     
         4 . The eye tracking method of  claim 3 , wherein the resetting of the scan region by sequentially expanding the size of the window comprises:
 sequentially expanding the size of the window for setting the previous scan region based on a number of times tracking of the target region in the previous scan region has failed; and   resetting the scan region based on the sequentially expanded size of the window.   
     
     
         5 . The eye tracking method of  claim 4 , wherein the sequentially expanding the size of the window based on the number of times the tracking of the target region has failed comprises at least one of:
 expanding the size of the window to a size of a first window to which the previous scan region is expanded upward, downward, leftward, and rightward, based on a determination that the tracking of the target region has failed one time;   expanding the size of the window to a size of a second window to which a scan region based on the first window is expanded leftward and rightward, based on a determination that the tracking of the target region has failed two times; and   expanding the size of the window to a size of a third window to which a scan region based on the second window is expanded upward and downward, based on a determination that the tracking of the target region has failed three times.   
     
     
         6 . The eye tracking method of  claim 3 , wherein the resetting of the scan region by sequentially expanding the size of the window comprises:
 resetting an entire image frame to the scan region, in response to the tracking of the target region has failed in the scan region reset through the sequential expanding.   
     
     
         7 . The eye tracking method of  claim 1 , further comprising:
 selecting an initial scan window corresponding to the scan region from among a plurality of candidate windows, using at least one of a statistical position coordinates of the user and a position of the camera used to capture the image frame, based on feature parts of a face of the user included in the image frame; and   setting an initial scan region based on the initial scan window.   
     
     
         8 . The eye tracking method of  claim 1 , wherein the first prior information comprises at least one of pupil center coordinates of the user accumulated in the at least one previous frame, position coordinates of feature points corresponding to a face of the user in the at least one previous frame, and position coordinates of feature points corresponding to the eyes and the nose of the user in the at least one previous frame. 
     
     
         9 . The eye tracking method of  claim 1 , further comprises:
 based on a determination that tracking of the target region succeeded, determining a moving range of the target region movable in the image frame, based on the first prior information; and   generating a tracking map corresponding to the target region, based on the moving range of the target region.   
     
     
         10 . The eye tracking method of  claim 9 , wherein the tracking the target region of the user comprises tracking the target region of the user based on the tracking map. 
     
     
         11 . The eye tracking method of  claim 1 , further comprising:
 outputting information related to the detected target region of the user,   wherein the information related to the detected target region comprises at least one of positions of a pupil and the nose included in the scan region, a viewpoint by the position of the pupil represented in the scan region.   
     
     
         12 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform an eye tracking method comprising:
 acquiring an image frame;   tracking a target region including eyes and a nose of a user based on first prior information obtained from at least one previous frame of the image frame;   based on a determination that tracking of the target region based on the first prior information has failed, setting a scan region in the image frame based on second prior information obtained from the at least one previous frame; and   detecting the target region in the image frame based on the scan region.   
     
     
         13 . An eye tracking apparatus comprising:
 a sensor configured to acquire an image frame;   a processor configured to:
 track a target region including eyes and a nose of a user based on first prior information obtained from at least one previous frame of the image frame; 
 based on a determination that tracking of the target region based on the first prior information has failed, set a scan region in the image frame based on second prior information obtained from the at least one previous frame; and 
 detect the target region in the image frame based on the scan region; and 
   a display configured to output information related to the detected target region of the user.   
     
     
         14 . The eye tracking apparatus of  claim 13 , wherein the second prior information comprises information related to at least one previous scan region based on which detection of the target region is performed in the at least one previous frame, and
 the processor is further configured to reset the scan region to a region to which the at least one previous scan region is expanded.   
     
     
         15 . The eye tracking apparatus of  claim 14 , wherein the processor is further configured to, based on whether tracking of the target region in a previous scan region has failed, reset the scan region by sequentially expanding a size of a window for setting the previous scan region. 
     
     
         16 . The eye tracking apparatus of  claim 15 , wherein the processor is further configured to:
 sequentially expand the size of the window for setting the previous scan region based on a number of times tracking of the target region in the previous scan region has failed; and   reset the scan region based on the sequentially expanded size of the window.   
     
     
         17 . The eye tracking apparatus of  claim 16 , wherein the processor is further configured to perform at least one of:
 expanding the size of the window to a size of a first window to which the previous scan region is expanded upward, downward, leftward, and rightward, based on a determination that the tracking of the target region has failed one time;   expanding the size of the window to a size of a second window to which a scan region based on the first window is expanded leftward and rightward, based on a determination that the tracking of the target region has failed two times; and   expanding the size of the window to a size of a third window to which a scan region based on the second window is expanded upward and downward, based on a determination that the tracking of the target region has failed three times.   
     
     
         18 . The eye tracking apparatus of  claim 15 , wherein the processor is further configured to reset an entire image frame to the scan region, in response to the tracking of the target region has failed in the scan region reset through the sequential expanding. 
     
     
         19 . The eye tracking apparatus of  claim 13 , wherein the processor is further configured to;
 select an initial scan window corresponding to the scan region from among a plurality of candidate windows, using at least one of a statistical position coordinates of the user and a position of a camera used to capture the image frame, based on feature parts of a face of the user included in the image frame; and   set an initial scan region based on the initial scan window.   
     
     
         20 . The eye tracking apparatus of  claim 13 , wherein the first prior information comprises at least one of pupil center coordinates of the user accumulated in the at least one previous frame, position coordinates of feature points corresponding to a face of the user in the at least one previous frame, and position coordinates of feature points corresponding to the eyes and the nose of the user in the at least one previous frame. 
     
     
         21 . The eye tracking apparatus of  claim 13 , wherein the processor is further configured to:
 based on a determination that tracking of the target region succeeded, determine a moving range of the target region movable in the image frame, based on the first prior information;   generate a tracking map corresponding to the target region, based on the moving range of the target region; and   track the target region of the user based on the tracking map.   
     
     
         22 . The eye tracking apparatus of  claim 13 , wherein the information related to the detected target region comprises at least one of positions of a pupil and the nose included in the scan region, a viewpoint by the position of the pupil represented in the scan region. 
     
     
         23 . The eye tracking apparatus of  claim 13 , wherein the sensor comprises at least one of an image sensor, a vision sensor, an IR camera, and an RGB camera. 
     
     
         24 . A sensor device comprising:
 a sensor configured to acquire an image frame; and   a processor configured to:   track a target region including eyes and a nose of a user based on first prior information obtained from at least one previous frame of the image frame;   based on a determination that tracking of the target region based on the first prior information has failed, set a scan region in the image frame based on second prior information obtained from the at least one previous frame; and   detect the target region in the image frame based on the scan region.