Performing operations using a mirror in an artificial reality environment

Abstract

This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

What is claimed is: 
     
         1 . A system comprising:
 an image capture system configured to capture image data representative of a physical environment that includes a reflective surface and a plurality of objects, wherein the image capture system includes a head-mounted display (HMD) worn by a first user;   a mapping engine configured to determine, based on the image data, a map of the physical environment including position information about the HMD, the reflective surface, and each of the plurality of physical objects; and   an application engine configured to:
 enable communication over a network between the first user and a second user that is in a second physical environment, 
 determine, based on the position of the HMD and the reflective surface, whether the first user is located in front of the reflective surface, 
 responsive to determining that the first user is located in front of the reflective surface, output, over the network, a plurality of images of the first user, and 
 responsive to determining that the first user is not located in front of the reflective surface, refrain from outputting images of the first user. 
   
     
     
         2 . The system of  claim 1 , wherein the physical environment is a first physical environment, wherein the HMD worn by the first user is a first HMD, wherein the reflective surface is a first reflective surface, and wherein the application engine is further configured to:
 receive image data captured by a second HMD worn by the second user in the second physical environment, wherein the image data captured by the second HMD includes images reflected by a second reflective surface in the second physical environment, and wherein the second physical environment is located a remote distance from the first physical environment.   
     
     
         3 . The system of  claim 2 , wherein the application engine is further configured to:
 generate, based on the received image data, artificial reality content that includes an image representing the second user.   
     
     
         4 . The system of  claim 3 , wherein to output the plurality of images of the first user, the application engine is further configured to:
 enable the second HMD worn by the second user in the second physical environment to present, to the second user, image data representative of the first physical environment.   
     
     
         5 . The system of  claim 1 , further comprising:
 a rendering engine configured to render, for display at the HMD, the artificial reality content overlaid on an image of the reflective surface.   
     
     
         6 . The system of  claim 1 , wherein to output the plurality of images of the first user, the application engine is further configured to:
 output images that include image data representative of the first physical environment.   
     
     
         7 . The system of  claim 6 , wherein the image capture system includes:
 a plurality of cameras positioned within the physical environment configured to capture images of the physical environment.   
     
     
         8 . The system of  claim 7 , wherein to output images that include image data representative of the first physical environment, the application engine is further configured to:
 output images captured by the plurality of cameras.   
     
     
         9 . The system of  claim 7 , wherein to output the plurality of images of the first user, the application engine is further configured to:
 output images of the first user captured by the plurality of cameras.   
     
     
         10 . The system of  claim 1 , wherein to enable communication between the first user and the second user, the application engine is further configured to:
 enable two-way audio and video communications between the first user and the second user.   
     
     
         11 . A method comprising:
 capturing, by a computing system that includes a head-mounted display (HMD) worn by a first user, image data representative of a physical environment that includes a reflective surface, a plurality of objects, and the HMD;   determining, by the computing system and based on the image data, a map of the physical environment including position information about the HMD, the reflective surface, and each of the plurality of physical objects;   enabling, by the computing system, communication over a network between the first user and a second user, wherein the second user is located in a second physical environment;   determining, by the computing system and based on the position of the HMD and the reflective surface, whether the first user is located in front of the reflective surface;   responsive to determining that the first user is located in front of the reflective surface, outputting, by the computing system and over the network, a plurality of images of the first user; and   responsive to determining that the first user is not located in front of the reflective surface, refraining from outputting images of the first user.   
     
     
         12 . The method of  claim 11 , wherein the physical environment is a first physical environment, wherein the HMD worn by the first user is a first HMD, wherein the reflective surface is a first reflective surface, and wherein the method further comprises:
 receiving, by the computing system, image data captured by a second HMD worn by the second user in the second physical environment, wherein the image data captured by the second HMD includes images reflected by a second reflective surface in the second physical environment, and wherein the second physical environment is located a remote distance from the first physical environment.   
     
     
         13 . The method of  claim 12 , further comprising:
 generating, by the computing system and based on the received image data, artificial reality content that includes an image representing the second user.   
     
     
         14 . The method of  claim 13 , wherein outputting the plurality of images of the first user includes:
 enabling the second HMD worn by the second user in the second physical environment to present, to the second user, image data representative of the first physical environment.   
     
     
         15 . The method of  claim 11 , further comprising:
 rendering, by the computing system and for display at the HMD, the artificial reality content overlaid on an image of the reflective surface.   
     
     
         16 . The method of  claim 11 , wherein outputting the plurality of images of the first user includes:
 outputting images that include image data representative of the first physical environment.   
     
     
         17 . The method of  claim 16 , wherein the computing system includes a plurality of cameras positioned within the physical environment, and wherein capturing the image data includes:
 capturing, by the plurality of cameras, images of the physical environment.   
     
     
         18 . The method of  claim 17 , wherein outputting images that include image data representative of the first physical environment includes:
 outputting images captured by the plurality of cameras.   
     
     
         19 . The method of  claim 17 , wherein outputting the plurality of images of the first user includes:
 outputting images of the first user captured by the plurality of cameras.   
     
     
         20 . A non-transitory computer-readable medium comprising instructions for causing processing circuitry of an artificial reality system having an image capture system that includes a head-mounted display (HMD) worn by a first user to perform operations comprising:
 capturing image data representative of a physical environment that includes a reflective surface, a plurality of objects, and the HMD;   determining, based on the image data, a map of the physical environment including position information about the HMD, the reflective surface, and each of the plurality of physical objects;   enabling communication over a network between the first user and a second user, wherein the second user is located in a second physical environment;   determining, based on the position of the HMD and the reflective surface, whether the first user is located in front of the reflective surface;   responsive to determining that the first user is located in front of the reflective surface, outputting, over the network, a plurality of images of the first user; and   responsive to determining that the first user is not located in front of the reflective surface, refraining from outputting images of the first user.