Optimal assistance for object-rearrangement tasks in augmented reality

Abstract

The disclosed computer-implemented method may include identifying, via an artificial reality system, a current state of a physical object in a real-world environment of a user and determining a desired end state for the physical object. The method may also include determining a sequence of action steps for manipulating the object to place the object in the desired end state, and presenting, via the artificial reality system, a notification to the user indicative of the sequence of action steps. Various other methods, systems, and computer-readable media are also disclosed.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 identifying, via an artificial reality system, a current state of a physical object in a real-world environment of a user;   determining a desired end state for the physical object;   determining a sequence of action steps for manipulating the object to place the object in the desired end state; and   presenting, via the artificial reality system, a notification to the user indicative of the sequence of action steps.   
     
     
         2 . The method of  claim 1 , wherein determining the desired end state further comprises inferring, based on identifying the current state of the physical object, a user goal with respect to the physical object. 
     
     
         3 . The method of  claim 2 , wherein inferring the user goal is further based on contextual signals. 
     
     
         4 . The method of  claim 3 , wherein the contextual signals include at least one of a time of day, a user input, or a view of the user. 
     
     
         5 . The method of  claim 1 , wherein the current state of the physical object includes an uncertain state for the physical object. 
     
     
         6 . The method of  claim 5 , wherein the sequence of action steps includes an action step that is expected to resolve the uncertain state. 
     
     
         7 . The method of  claim 1 , wherein the physical object comprises an unknown object. 
     
     
         8 . The method of  claim 1 , further comprising:
 monitoring, via the artificial reality system, a progress of the sequence of action steps;   detecting, while monitoring the progress, a deviation from the sequence of action steps; and   updating the sequence of action steps based on the detected deviation.   
     
     
         9 . The method of  claim 1 , wherein the notification includes at least one of a visual notification, a haptic notification, or an audible notification. 
     
     
         10 . A system comprising:
 at least one physical processor;   physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the physical processor to:
 identify, via an artificial reality system, a current state of a physical object in a real-world environment of a user; 
 determine a desired end state for the physical object; 
 determine a sequence of action steps for manipulating the object to place the object in the desired end state; and 
 present, via the artificial reality system, a notification to the user indicative of the sequence of action steps. 
   
     
     
         11 . The system of  claim 10 , wherein the instructions for determining the desired end state further comprises instructions for inferring, based on identifying the current state of the physical object, a user goal with respect to the physical object. 
     
     
         12 . The system of  claim 11 , wherein inferring the user goal is further based on contextual signals. 
     
     
         13 . The system of  claim 12 , wherein the contextual signals include at least one of a time of day, a user input, or a view of the user. 
     
     
         14 . The system of  claim 10 , wherein the current state of the physical object includes an uncertain state for the physical object and the sequence of action steps includes an action step that is expected to resolve the uncertain state. 
     
     
         15 . The system of  claim 10 , wherein the physical object comprises an unknown object. 
     
     
         16 . A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
 identify, via an artificial reality system, a current state of a physical object in a real-world environment of a user;   determine a desired end state for the physical object;   determine a sequence of action steps for manipulating the object to place the object in the desired end state; and   present, via the artificial reality system, a notification to the user indicative of the sequence of action steps.   
     
     
         17 . The non-transitory computer-readable medium of  claim 16 , wherein the instructions for determining the desired end state further comprises instructions for inferring, based on identifying the current state of the physical object, a user goal with respect to the physical object. 
     
     
         18 . The non-transitory computer-readable medium of  claim 17 , wherein inferring the user goal is further based on contextual signals and the contextual signals include at least one of a time of day, a user input, or a view of the user. 
     
     
         19 . The non-transitory computer-readable medium of  claim 16 , wherein the current state of the physical object includes an uncertain state for the physical object and the sequence of action steps includes an action step that is expected to resolve the uncertain state. 
     
     
         20 . The non-transitory computer-readable medium of  claim 16 , further comprising instructions for:
 monitoring, via the artificial reality system, a progress of the sequence of action steps;   detecting, while monitoring the progress, a deviation from the sequence of action steps; and   updating the sequence of action steps based on the detected deviation.