Integrating Artificial Reality and Other Computing Devices

Abstract

Methods and systems for integrating functionality of XR devices and non-XR devices are disclosed herein. An XR device can identify and obtain one or more content items being displayed on non-XR devices and create virtual objects associated with the content item in an artificial reality environment of the XR device. A user of the XR device can interact with the virtual objects, such as by moving the virtual objects to new locations within the artificial reality environment or interact with the virtual objects to view the content items. The user of the XR device can also provide one or more gestures or commands to send content items or identifiers of content items associated with the virtual objects to non-XR devices for display at the non-XR devices.

I/We claim: 
     
         1 . A method comprising:
 creating a registration between an XR device and one or more non-XR devices;   identifying a virtual object, displayed in an artificial reality environment of the XR device, associated with a content item;   identifying a gesture of a user directed at a non-XR device of the one or more non-XR devices, wherein the identifying includes identifying an endpoint of the gesture, within the artificial reality environment, in relation to the non-XR device; and   using a communication connection, set up based on the registration between the XR device and one or more non-XR devices, to provide, to the non-XR device, A) the content item or an identifier of the content item associated with the selected virtual object and B) an indication of the gesture endpoint in relation to the non-XR device;   wherein the non-XR device, in response to the providing of the content item or identifier of the content item:
 obtains the content item; 
 accesses one or more rules in relation to the obtained content item; and 
 evaluates the one or more accessed rules against the gesture endpoint to take a corresponding action for the obtained content item. 
   
     
     
         2 . The method of  claim 1 , wherein the identifying the endpoint of the gesture, within the artificial reality environment, in relation to the non-XR device includes identifying a point on a screen of the non-XR device associated with a location where the gesture was completed. 
     
     
         3 . The method of  claim 1 , wherein the identifying the endpoint of the gesture, within the artificial reality environment, in relation to the non-XR device includes the XR device determining offsets from an identified corner of a display of the non-XR device. 
     
     
         4 . The method of  claim 1 ,
 wherein the virtual object is a representation of a video call; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to access a data source of the video call and then begin participating in the video call at the non-XR device.   
     
     
         5 . The method of  claim 1 ,
 wherein the virtual object is a representation of a social media post; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to open a social media site associated with the social media post.   
     
     
         6 . The method of  claim 1 ,
 wherein the virtual object is a representation of a user contact; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to initiate a communication session with a user represented by the user contact.   
     
     
         7 . The method of  claim 1 , wherein the non-XR device evaluating the one or more accessed rules includes determining that the gesture endpoint corresponds to a graphical representation, on a display of the non-XR device, of a particular software application and, in response, opening the content item with the software application. 
     
     
         8 . The method of  claim 1 , wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to output the content item based on a mapping of a determined type of the content item to an output method. 
     
     
         9 . The method of  claim 1 ,
 wherein the content item is a first content item; and   wherein the non-XR device evaluating the one or more accessed rules includes determining that the gesture endpoint corresponds to a graphical representation, on a display of the non-XR device, of a second content item and, in response, embedding the first content item in the second content item.   
     
     
         10 . The method of  claim 1 ,
 wherein the virtual object is a representation of a video that is playing in the artificial reality environment;   wherein the providing further includes providing C) a playback frame or timestamp of playback of the video; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to start playback of the video, on the non-XR device, at the playback frame or timestamp.   
     
     
         11 . A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:
 creating a registration between an XR device and one or more non-XR devices;   identifying a virtual object, displayed in an artificial reality environment of the XR device, associated with a content item;   identifying a gesture of a user directed at a non-XR device of the one or more non-XR devices, wherein the identifying includes identifying an endpoint of the gesture, within the artificial reality environment, in relation to the non-XR device; and   using a communication connection, set up based on the registration between the XR device and one or more non-XR devices, to provide, to the non-XR device, A) the content item or an identifier of the content item associated with the selected virtual object and B) an indication of the gesture endpoint in relation to the non-XR device;   wherein the non-XR device, in response to the providing of the content item or identifier of the content item:
 obtains the content item; 
 accesses one or more rules in relation to the obtained content item; and 
 evaluates the one or more accessed rules against the gesture endpoint to take a corresponding action for the obtained content item. 
   
     
     
         12 . The computer-readable storage medium of  claim 11 , wherein the identifying the endpoint of the gesture, within the artificial reality environment, in relation to the non-XR device includes the XR device determining offsets from an identified corner of a display of the non-XR device. 
     
     
         13 . The computer-readable storage medium of  claim 11 ,
 wherein the virtual object is a representation of a video call; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to access a data source of the video call and then begin participating in the video call at the non-XR device.   
     
     
         14 . The computer-readable storage medium of  claim 11 ,
 wherein the virtual object is a representation of a video that is playing in the artificial reality environment;   wherein the providing further includes providing C) a playback frame or timestamp of playback of the video; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to start playback of the video, on the non-XR device, at the playback frame or timestamp.   
     
     
         15 . The computer-readable storage medium of  claim 11 ,
 wherein the virtual object is a representation of a user contact; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to initiate a communication session with a user represented by the user contact.   
     
     
         16 . The computer-readable storage medium of  claim 11 , wherein the non-XR device evaluating the one or more accessed rules includes determining that the gesture endpoint corresponds to a graphical representation, on a display of the non-XR device, of a particular software application and, in response, opening the content item with the software application. 
     
     
         17 . The computer-readable storage medium of  claim 11 , wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to output the content item based on a mapping of a determined type of the content item to an output method. 
     
     
         18 . A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:
 creating a registration between an XR device and one or more non-XR devices;   identifying a virtual object, displayed in an artificial reality environment of the XR device, associated with a content item;   identifying a gesture of a user directed at a non-XR device of the one or more non-XR devices, wherein the identifying includes identifying an endpoint of the gesture, within the artificial reality environment, in relation to the non-XR device; and   using a communication connection, set up based on the registration between the XR device and one or more non-XR devices, to provide, to the non-XR device, A) the content item or an identifier of the content item associated with the selected virtual object and B) an indication of the gesture endpoint in relation to the non-XR device;   wherein the non-XR device, in response to the providing of the content item or identifier of the content item:
 obtains the content item; 
 accesses one or more rules in relation to the obtained content item; and 
 evaluates the one or more accessed rules against the gesture endpoint to take a corresponding action for the obtained content item. 
   
     
     
         19 . The computing system of  claim 18 ,
 wherein the content item is a first content item; and   wherein the non-XR device evaluating the one or more accessed rules includes determining that the gesture endpoint corresponds to a graphical representation, on a display of the non-XR device, of a second content item and, in response, embedding the first content item in the second content item.   
     
     
         20 . The computing system of  claim 18 ,
 wherein the virtual object is a representation of a social media post; and   wherein the non-XR device evaluating the one or more accessed rules causes the non-XR device to open a social media site associated with the social media post.