Three-dimensional interaction system

Abstract

Among other things, embodiments of the present disclosure improve the functionality of computer imaging software and systems by facilitating the manipulation of virtual content displayed in conjunction with images of real-world objects and environments. Embodiments of the present disclosure allow virtual objects to be moved relative to a real-world environment and manipulated in other ways.

What is claimed is: 
     
         1 . A system comprising:
 a processor;   a camera coupled to the processor;   a user interface coupled to the processor, the user interface comprising a touch screen; and   memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising:
 displaying a two-dimensional image of a three-dimensional real-world scene captured via the camera on the touch screen; 
 mapping a virtual object to the three-dimensional real-world scene; 
 displaying the virtual object within the image; 
 receiving, via the touch screen, an input from the user; and 
 modifying a visual characteristic of the virtual object within the image in response to the input from the user. 
   
     
     
         2 . The system of  claim 1 , wherein the system further includes an inertial measurement sensor and wherein the input from the user further includes a movement of the system by the user measured by the inertial measurement sensor. 
     
     
         3 . The system of  claim 2 , wherein the movement of the system by the user includes one or more of: rotation about an axis, and repositioning the system from a first physical location to a second physical location. 
     
     
         4 . The system of  claim 3 , wherein displaying the virtual object within the image includes displaying the virtual object in a fixed position relative to the real-world scene as the user moves the system. 
     
     
         5 . The system of  claim 3 , wherein displaying the virtual object within the image includes modifying a displayed perspective of the virtual object as the user moves the system. 
     
     
         6 . The system of  claim 2 , wherein mapping the virtual object to the three-dimensional real-world scene includes generating a matrix containing image data from the camera and movement data from the inertial measurement sensor. 
     
     
         7 . The system of  claim 1 , wherein the input from the user includes the user pressing a finger of the user on the touchscreen to select the virtual object. 
     
     
         8 . The system of  claim 7 , wherein modifying the visual characteristic of the virtual object includes determining a touch position of the user's finger on the touchscreen, and determining an intersection between the touch position and the virtual object. 
     
     
         9 . The system of  claim 8 , wherein determining the intersection between the touch position and the virtual object includes raycasting the touch position to a bounding box surrounding the virtual object. 
     
     
         10 . The system of  claim 7 , wherein the input from the user further includes the user sliding a finger of the user across the touchscreen, and wherein modifying the visual characteristic of the virtual object includes moving the virtual object from a first position to a second position in response to the sliding of the user's finger across the touchscreen. 
     
     
         11 . The system of  claim 10 , wherein moving the virtual object from the first position to the second position includes:
 identifying a spatial limit in the three-dimensional real-world scene; and   restricting movement of the virtual object based on the spatial limit.   
     
     
         12 . The system of  claim 10 , wherein moving the virtual object from the first position to the second position includes determining a frustum-relative orientation for the virtual object and maintaining the frustum-relative orientation for the object between the first position and the second position. 
     
     
         13 . The system of  claim 1 , further comprising modifying a second visual characteristic of the virtual object in response to the input from the user. 
     
     
         14 . The system of  claim 13 , wherein modifying the second visual characteristic includes modifying a scale of at least a portion of the virtual object. 
     
     
         15 . The system of  claim 13 , wherein modifying the second visual characteristic includes rotating the virtual object. 
     
     
         16 . The system of  claim 13 , wherein modifying the second visual characteristic includes modifying a shape of at least a portion of the virtual object. 
     
     
         17 . The system of  claim 13 , wherein modifying the second visual characteristic includes modifying one or more of: a color, a shading, and a texture. 
     
     
         18 . The system of  claim 13 , wherein modifying the second visual characteristic includes displaying one or more of: text, video, and an image. 
     
     
         19 . A computer-implemented method comprising:
 displaying, by a computer system on a touch screen coupled to the computer system, a two-dimensional image of a three-dimensional real-world scene captured via a camera coupled to the computer system;   mapping, by the computer system, a virtual object to the three-dimensional real-world scene;   displaying, by the computer system, the virtual object within the image;   receiving, by the computer system via the touch screen, an input from the user; and   modifying, by the computer system, a visual characteristic of the virtual object within the image in response to the input from the user.   
     
     
         20 . A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:
 displaying, on a touch screen coupled to the computer system, a two-dimensional image of a three-dimensional real-world scene captured via a camera coupled to the computer system;   mapping a virtual object to the three-dimensional real-world scene;   displaying the virtual object within the image;   receiving, via the touch screen, an input from the user; and   modifying a visual characteristic of the virtual object within the image in response to the input from the user.