System for multi-presence interaction with extended reality objects

Abstract

Aspects of the subject disclosure may include, for example, scanning, by an augmented reality system, a local environment which includes an object. The scanning includes identifying one or more characteristics of the object such as its size or shape. The subject disclosure may further include providing, to a display device of a local user in the local environment, local image information of the object from a viewing perspective of the local user and providing, to a display device of a remote user in a remote environment, remote image information of the object from a viewing perspective of the remote user. The subject disclosure may further include detecting a local user interaction with the object, detecting a release of control of the object by the local user to control of the object by the remote user, and providing updated local image information of the object to the display device of the local user and providing updated remote image information to the display device of the remote user based on the release of control of the object. Other embodiments are disclosed.

What is claimed is: 
     
         1 . A method, comprising:
 detecting, by a processing system including a processor, an object in an augmented reality system, the object located in a local environment of a local user of the augmented reality system;   displaying, by the processing system, the object in the local environment in a local display on a local display device of the local user, wherein the displaying comprises displaying the object from perspective of the local user;   providing, by the processing system, a remote display of the object on a remote display device of a remote user of the augmented reality system, wherein the displaying the object on the remote display comprises displaying a remote virtual image of the object in the local environment from perspective of the remote user;   detecting, by the processing system, a movement of the object by the local user;   updating, by the processing system, the local display of the object on the local display device to reflect the movement of the object; and   updating, by the processing system, the remote display of the object on the remote display device.   
     
     
         2 . The method of  claim 1 , wherein the updating the local display comprises:
 in-painting, by the processing system, a portion of the local display at a first location and providing a local virtual image of the object at a second location; and   providing, by the processing system, the remote virtual image of the object in the second location of the local environment to update the remote display of the object.   
     
     
         3 . The method of  claim 2 , comprising:
 detecting, by the processing system, a remote user interaction with the object by the remote user at the remote display device; and   updating, by the processing system, the local virtual image of the object and updating the remote virtual image responsive to the remote user interaction.   
     
     
         4 . The method of  claim 2 , further comprising:
 identifying, by the processing system, a location of the object in the local environment;   detecting, by the processing system, a release of the object by the local user;   determining, by the processing system, a trajectory and a speed of motion of the object responsive to the detecting the release of the object;   tracking, by the processing system, motion of the object;   updating, by the processing system, the local virtual image of the object responsive to the trajectory and the speed of motion of the object; and   updating, by the processing system, the remote virtual image responsive to the trajectory and the speed of motion of the object.   
     
     
         5 . The method of  claim 4 , further comprising:
 identifying, by the processing system, a catching interaction with the object by the remote user; and   updating, by the processing system, the local virtual image of the object and the remote virtual image responsive to the catching interaction.   
     
     
         6 . The method of  claim 2 , further comprising:
 recording, by the processing system, the local virtual image and the remote virtual image, forming recorded information;   recording, by the processing system, associated time stamp information; and   subsequently, providing a playback of the recorded information.   
     
     
         7 . The method of  claim 1 , further comprising:
 displaying, by the processing system, an initial local approximate appearance of the object in the local environment in the local display;   displaying, by the processing system, the object in the local environment in the local display with additional visual details of the object as the additional visual details of the object are detected by the processing system;   providing, by the processing system, an initial remote approximate appearance of the object at the remote display of the object; and   providing, by the processing system, the remote display of the object on the remote display device with additional visual details of the object as the additional visual details of the object are detected by the processing system.   
     
     
         8 . The method of  claim 7 , further comprising:
 scanning, by the processing system, the object in the local environment, wherein the scanning comprises identifying one or more characteristics of the object;   developing, by the processing system, a model of the object based on the scanning; and   producing, by the processing system, the initial local approximate appearance of the object and the initial remote approximate appearance of the object based on the model of the object.   
     
     
         9 . A device, comprising:
 a processing system including a processor; and   a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising:   identifying one or more characteristics of an object located in a local environment;   providing, to a display device of a local user in the local environment, local image information of the object, the local image information from a viewing perspective of the local user, the local image information causing the display device of the local user to produce a local image of the object;   providing, to the display device of a remote user in a remote environment, remote image information of the object, the remote image information from a viewing perspective of the remote user, the remote image information causing the display device of the remote user to produce a remote image of the object;   detecting a remote user interaction with the object by the remote user; and   providing updated local image information of the object to the display device of the local user and providing updated remote image information to the display device of the remote user based on the remote user interaction with the object.   
     
     
         10 . The device of  claim 9 , wherein the operations further comprise:
 detecting a local user interaction with the object;   further updating the local image information of the object and the remote image information based on the local user interaction; and   providing the further updated local image information to the display device of the local user and providing the further updated remote image information to the display device of the remote user.   
     
     
         11 . The device of  claim 9 , further comprising:
 identifying a hand of the local user proximate the object;   identifying a machine learning model for the object based on the one or more characteristics of the object, wherein the machine learning model is accessed from a library of pre-existing models;   scanning the object to identify additional characteristics of the object; and   updating the machine learning model based on the additional characteristics.   
     
     
         12 . The device of  claim 11 , wherein the operations further comprise:
 based on the machine learning model, providing as the local image information an initial approximate local appearance of the object and providing as the remote image information an initial approximate remote appearance of the object;   during the scanning of the object to identify additional characteristics of the object; updating the machine learning model to form an updated model of the object; and   updating the local image information and the remote image information using the updated model of the object.   
     
     
         13 . The device of  claim 9 , wherein the operations further comprise:
 identifying a shape and an orientation of the object, wherein the object is an arbitrary, unknown object;   identifying a machine learning model for the object based on the shape and the orientation of the object;   identifying additional characteristics of the object, wherein the additional characteristics include a texture, a color or a deformability of the object, or a combination of these; and   updating the machine learning model for the object based on the additional characteristics.   
     
     
         14 . The device of  claim 9 , wherein the providing updated local image information of the object to the display device of the local user comprises:
 providing real object image information of the object to the display device, the real object image information forming an image of the object prior to the remote user interaction with the object by the remote user; and   after the remote user interaction with the object by the remote user, in-painting the image of the object based on the remote user interaction with the object.   
     
     
         15 . The device of  claim 9 , wherein the operations further comprise:
 detecting a release of control of the object by the remote user to control of the object by the local user; and   after the detecting the release of control of the object by the remote user, maintaining physics of the object while providing the local image information and the remote image information.   
     
     
         16 . The device of  claim 15 , wherein maintaining physics of the object comprises:
 detecting a trajectory and a speed of motion of the object after the detecting the release of control of the object; and   providing the local image information and the remote image information according to the trajectory and the speed of motion of the object.   
     
     
         17 . The device of  claim 9 , wherein the operations further comprise:
 recording the local image information and the remote image information, forming recorded information;   storing the recorded information with associated time stamp information; and   subsequently, providing a playback of the recorded information according to the associated time stamp information.   
     
     
         18 . A non-transitory, computer-readable storage medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, the operations comprising:
 detecting visual characteristics of one or more objects in a local environment and detecting a local user in the local environment;   providing, to a local display device of the local user, a local image of an object of the one or more objects in the local environment, wherein the providing the local image of the object comprises providing an initial approximate image of the object from perspective of the local user based on visual characteristics of the object;   providing to a remote display device of a remote user, a remote virtual image of the object, wherein providing the remote virtual image of the object comprises providing a three-dimensional remote virtual image showing the object in the local environment, and wherein providing the remote virtual image of the object comprises providing an initial approximate image of the object viewed from perspective of the remote user based on the visual characteristics of the object;   identifying additional visual characteristics of the object; and   updating the local image of the object at the local display device and the remote virtual image of the object at the remote display device responsive to the identifying the additional visual characteristics of the object.   
     
     
         19 . The non-transitory, computer-readable storage medium of  claim 18 , wherein the operations further comprise:
 detecting a local user action with the object by the local user, wherein the local user action comprises a movement of the object from a first location to a second location in the local environment; and   updating the local image of the object, wherein the updating the local image of the object comprises in-painting the local image at the first location and providing a local virtual image of the object at the second location in the local environment; and   updating the remote virtual image of the object at the remote display device responsive to the detecting the local user action, wherein updating the remote virtual image of the object comprises providing the remote virtual image of the object at the second location in the local environment.   
     
     
         20 . The non-transitory, computer-readable storage medium of  claim 19 , wherein the operations further comprise:
 detecting a release of control of the object by the local user;   determining a trajectory and a speed of motion of the object responsive to the detecting the release of control of the object;   predicting an interaction of the object with another object, wherein the predicting the interaction is responsive to the trajectory and the speed of motion of the object; and   updating the local virtual image of the object and updating the remote virtual image responsive to the trajectory and the speed of motion of the object.