System and method for virtual touch sensing

Abstract

In view of existing mobile devices which have the limitation of relatively small area of the touch screen, the present invention describes a virtual touch sensing method based on computer vision technology. The method includes the steps of using more than one sensor to detect the coordinates of an indicator in a virtual touching area, and calculating the respective screen coordinates according to the coordinates of the indicator, where the area of the operation surface of the virtual touching area is independent to the area of the screen. The present invention also disclosed a corresponding virtual touch sensing system which provides a predictive control interface, where the area of the control interface is independent to the area of the actual screen.

What is claimed is: 
     
       1. A method for virtual touch sensing, comprising steps of:
 a) detecting coordinates of an indicator in a virtual sensing space by using more than one sensors; and 
 b) calculating corresponding screen coordinates on a screen according to said coordinates of said indicator; wherein an operation plane in said virtual sensing space having an area independent to that of said screen; 
 wherein said sensors further comprise a first camera and a second camera; said first camera being substantially located at a first edge corner of said display and said second camera being substantially located at a second edge corner of said display; a location of said indicator in a 2-dimensional image of said first camera being P L (x l , y l ); a location of said indicator in a 2-dimensional image of said second camera being P R (x r , y r ); a 3-dimensional coordinate P(x, y, z) of said indicator in said virtual sensing space being calculated using the following equations:
     x=b ( x   l   +x   r )/[2( x   l   −x   r )] 
     y=b ( y   l   +y   r )/[2( y   l   −y   r )] 
     z=bf /( x   l   −x   r ) 
 
 wherein b is a straight line distance between said first camera and said second camera, and f is a focal length of said first camera and said second camera. 
 
     
     
       2. The method of  claim 1  further comprises displaying a mouse cursor on said screen according to said screen coordinates of said indicator; said mouse cursor moving in said screen according to movement of said indicator when said indicator moves substantially on said operation plane in said virtual sensing space. 
     
     
       3. The method of  claim 2 , further comprises:
 a) dividing at least one region in said screen, said region being a clickable element range of a preset key; 
 b) determining if said mouse cursor has entered said clickable element range of one said preset key according to a location of said mouse cursor on said screen; and 
 c) sensing a click action of said indicator on a surface of said screen and treating said click action as an input operation to said preset key when said mouse cursor enters said clickable element range of said preset key; said click action of said indicator within said range of preset key in said virtual sensing space not being a command to move said cursor. 
 
     
     
       4. The method of  claim 2 , further comprises:
 a) dividing at least one region in said screen, said region being a clickable element range of a preset key; 
 b) determining if said mouse cursor has entered said clickable element range of one said preset key according to a location of said mouse cursor on said screen; 
 c) locking said preset key if said indicator moves on a predetermined direction in said virtual sensing space when said mouse cursor enters said clickable element range of said preset key; 
 d) invoking said preset key if said indicator substantially moves to a direction reverse to said predetermined direction within predetermined time; 
 e) unlocking said preset key if said indicator substantially does not move to said direction reverse to said predetermined direction within said predetermined time. 
 
     
     
       5. The method of  claim 1 , wherein said sensors are located at a back of said screen. 
     
     
       6. The method of  claim 5 , wherein said sensors capture a plurality of images of said indicator, said plurality of images being displayed in said screen in semi-transparent form. 
     
     
       7. The method of  claim 1 , wherein an angle between said operation plane of said virtual sensing space and a surface of said screen is arbitrary. 
     
     
       8. The method of  claim 1 , wherein said area of said operation plane of said virtual sensing space is dynamically adjusted according to a user preference. 
     
     
       9. The method of  claim 1 , wherein said indicator has a feature point on a surface of said indicator for said sensors to sense; said feature point being a matching point for images of said first camera and said second camera. 
     
     
       10. The method of  claim 9 , wherein said feature point is a straight line with three round dots thereon; the middle point of said three dots being said matching point for images of said first camera and said second camera. 
     
     
       11. A system for virtual touch sensing, comprising:
 a) a microprocessor; 
 b) a display screen; and 
 c) more than one sensors, said sensors detecting coordinates of an indicator in a virtual sensing space; said microprocessor calculating corresponding screen coordinates on a screen according to said coordinates of said indicator; wherein an operation plane in said virtual sensing space having an area independent to that of said screen; 
 wherein said sensors further comprise a first camera and a second camera; said first camera being substantially located at a first edge corner of said display and said second camera being substantially located at a second edge corner of said display; a location of said indicator in a 2-dimensional image of said first camera being P L (x l , y l ); a location of said indicator in a 2-dimensional image of said second camera being P R (x r , y r ); a 3-dimensional coordinate P(x, y, z) of said indicator in said virtual sensing space being calculated using the following equations:
     x=b ( x   l   +x   r )/[2( x   l   −x   r )] 
     y=b ( y   l   +y   r )/[2( y   l   −y   r )] 
     z=bf /( x   l   −x   r ) 
 
 wherein b is a straight line distance between said first camera and said second camera, and f is a focal length of said first camera and said second camera. 
 
     
     
       12. The system of  claim 11 , wherein said screen displays a mouse cursor, said mouse cursor moving on said screen according to a movement of said indicator when said indicator substantially moving on said operation plane in said virtual sensing space. 
     
     
       13. The system of  claim 12 , wherein at least one region is divided in said screen; said region being a clickable element range of a preset key; said microprocessor determining if said mouse cursor has entered said clickable element range of one said preset key according to a location of said mouse cursor on said screen; when said mouse cursor enters said clickable element range of said preset key, said click action of said indicator within said range of preset key in said virtual sensing space not being a command to move said cursor; a click action of said indicator on a surface of said screen being sensed by said screen and treated as an input operation to said preset key. 
     
     
       14. The system of  claim 12 , wherein at least one region is divided in said screen; said region being a clickable element range of a preset key; said microprocessor determining if said mouse cursor has entered said clickable element range of one said preset key according to a location of said mouse cursor on said screen; when said mouse cursor enters said clickable element range of said preset key, a movement of said indicator on a predetermined direction in said virtual sensing space resulting in said preset key being locked; if said indicator substantially moving to a direction reverse to said predetermined direction within predetermined time, said preset key being invoked; if said indicator not substantially moving to said direction reverse to said predetermined direction within said predetermined time, said preset key being unlocked. 
     
     
       15. The system of  claim 11 , wherein said area of said operation plane of said virtual sensing space is dynamically adjusted according to a user preference. 
     
     
       16. The system of  claim 11 , wherein said indicator has a feature point on a surface of said indicator for said sensors to sense; said feature point being a matching point for images of said first camera and said second camera. 
     
     
       17. The system of  claim 16 , wherein said feature point is a straight line with three round dots thereon; the middle point of said three dots being said matching point for images of said first camera and said second camera. 
     
     
       18. The system of  claim 11  further comprises a mobile device, said microprocessor, said screen and said more than one sensors being installed in said mobile device.