Porting physical object into virtual reality

Abstract

Disclosed herein are related to a system and a method for porting a physical object in a physical space into a virtual reality. In one approach, the method includes detecting an input device in a physical space relative to a user of the input device. In one approach, the method includes presenting, by a display device to the user, a virtual model of the detected input device in a virtual space at a location and an orientation. The location and the orientation of the virtual model in the virtual space may correspond to a location and an orientation of the input device in the physical space relative to the user. In one approach, the method includes visually providing relative to the virtual model in the virtual space, through the display device, spatial feedback on the user's interaction with the input device in the physical space.

What is claimed is: 
     
         1 . A method comprising:
 detecting a first contour or a first layout of an input device in a physical space;   identifying, from a plurality of candidate models having various levels of similarity to the first contour or the first layout of the input device and corresponding to a same device type as the input device, a candidate model having a second contour or a second layout in a virtual space, that is closest to the first contour or the first layout of the input device in the physical space, as a virtual model of the input device; and   presenting, by a display device, the virtual model of the input device in the virtual space.   
     
     
         2 . The method of  claim 1 , further comprising:
 determining a first location and a first orientation of the virtual model in the virtual space according to a second location and a second orientation of the input device in the physical space; and   rendering an image of the virtual space including the virtual model of the input device, according to the determined first location and the first orientation of the virtual model in the virtual space.   
     
     
         3 . The method of  claim 2 , further comprising:
 detecting a movement of a user in the physical space;   determining a change in the first location and the first orientation of the virtual model according to the detected movement of the user; and   rendering another image of the virtual space including the virtual model of the input device, according to the change in the first location and the first orientation of the virtual model.   
     
     
         4 . The method of  claim 1 , wherein detecting the first contour or the first layout of the input device in the physical space includes:
 capturing an image of the input device, and   detecting the first contour or the first layout of the input device in the captured image.   
     
     
         5 . The method of  claim 4 , wherein detecting the first contour or the first layout of the input device in the captured image includes:
 determining an arrangement of keys on the input device.   
     
     
         6 . The method of  claim 1 , further comprising:
 detecting a product identification of the input device; and   determining the candidate model corresponding to the detected product identification.   
     
     
         7 . The method of  claim 1 , wherein the input device comprises a keyboard. 
     
     
         8 . A system comprising:
 a display device;   a processor; and   a non-transitory computer readable medium storing instructions when executed by the processor cause the processor to:
 detect a first contour or a first layout of an input device in a physical space, 
 identify, from a plurality of candidate models having various levels of similarity to the first contour or the first layout of the input device and corresponding to a same device type as the input device, a candidate model having a second contour or a second layout in a virtual space, that is closest to the first contour or the first layout of the input device in the physical space, as a virtual model of the input device, and 
 present the virtual model of the input device in the virtual space. 
   
     
     
         9 . The system of  claim 8 , wherein the non-transitory computer readable medium stores instructions when executed by the processor cause the processor to:
 determine a first location and a first orientation of the virtual model in the virtual space according to a second location and a second orientation of the input device in the physical space; and   render an image of the virtual space including the virtual model of the input device, according to the determined first location and the first orientation of the virtual model in the virtual space.   
     
     
         10 . The system of  claim 9 , wherein the non-transitory computer readable medium stores instructions when executed by the processor cause the processor to:
 detect a movement of a user in the physical space;   determine a change in the first location and the first orientation of the virtual model according to the detected movement of the user; and   render another image of the virtual space including the virtual model of the input device, according to the change in the first location and the first orientation of the virtual model.   
     
     
         11 . The system of  claim 8 , wherein the processor is configured to detect the input device in the physical space by:
 capturing an image of the input device, and   detecting the first contour or the first layout of the input device in the captured image.   
     
     
         12 . The system of  claim 11 , wherein the processor is configured to detect the first contour or the first layout of the input device in the captured image by:
 determining an arrangement of keys on the input device.   
     
     
         13 . The system of  claim 8 , wherein the non-transitory computer readable medium stores instructions when executed by the processor cause the processor to:
 detect a product identification of the input device; and   determine the candidate model corresponding to the detected product identification.   
     
     
         14 . The system of  claim 8 , wherein the input device comprises a keyboard. 
     
     
         15 . A head mounted display comprising:
 an imaging sensor configured to capture an image of an input device in a physical space;   a processor configured to:
 detect from the captured image, a first contour or a first layout of the input device in the physical space, and 
 identify, from a plurality of candidate models having various levels of similarity to the first contour or the first layout of the input device and corresponding to a same device type as the input device, a candidate model having a second contour or a second layout in a virtual space, that is closest to the first contour or the first layout of the input device in the physical space, as a virtual model of the input device; and 
   a display device configured to:
 present the virtual model of the input device in the virtual space. 
   
     
     
         16 . The head mounted display of  claim 15 , wherein:
 the processor is configured to determine a first location and a first orientation of the virtual model in the virtual space according to a second location and a second orientation of the input device in the physical space; and   the display device is configured to render an image of the virtual space including the virtual model of the input device, according to the determined first location and the first orientation of the virtual model in the virtual space.   
     
     
         17 . The head mounted display of  claim 16 , wherein:
 the processor is configured to detect a movement of a user in the physical space and determine a change in the first location and the first orientation of the virtual model according to the detected movement of the user, and   the display device is configured to render another image of the virtual space including the virtual model of the input device, according to the change in the first location and the first orientation of the virtual model.   
     
     
         18 . The head mounted display of  claim 15 , wherein the processor is configured to detect the first contour or the first layout of the input device in the captured image by:
 determining an arrangement of keys on the input device.   
     
     
         19 . The head mounted display of  claim 15 , wherein the processor is configured to:
 detect a product identification of the input device; and   determine the candidate model corresponding to the detected product identification.   
     
     
         20 . The head mounted display of  claim 15 , wherein the input device is a keyboard.