Method and Device for Presenting Synthesized Reality Content in Association with Recognized Objects

Abstract

In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.

What is claimed is: 
     
         1 . A method comprising:
 at a computing system including non-transitory memory and one or more processors, wherein the computing system is communicatively coupled to a display device, one or more input devices, and an image sensor:
 obtaining, via the image sensor, image data associated with a physical setting; 
 recognizing a first physical object and a second physical object within the image data associated with the physical setting; 
 determining a first camera pose associated with the image data relative to the first and second physical objects; 
 obtaining a first synthesized reality (SR) object associated with the first physical object based on the first camera pose relative to the first physical object; 
 obtaining a second SR object associated with the second physical object based on the first camera pose relative to the second physical object, wherein the second SR object is different from the first SR object 
 causing presentation of the first SR object in association with the first physical object and the second SR object in association with the second physical object via the display device. 
   
     
     
         2 . The method of  claim 1 , wherein obtaining the first SR object associated with the first physical object based on the first camera pose relative to the first physical object includes:
 obtaining a third SR object associated with the first physical object in accordance with a determination that the first camera pose relative to the first physical object correspond to a first point-of-view (POV) of the first physical object; and   obtaining a fourth SR object associated with the first physical object, wherein the fourth SR object is different from the third SR object, in accordance with a determination that the first camera pose relative to the first physical object correspond to a second POV of the first physical object different from the first POV of the first physical object.   
     
     
         3 . The method of  claim 1 , further comprising:
 detecting a change from the first camera pose to a second camera pose associated with the image data relative to the first and second physical objects; and   in response to detecting the change to the second camera pose:
 obtaining a third SR object associated with the first physical object based on the second camera pose relative to the first physical object; 
 obtaining a fourth SR object associated with the second physical object based on the second camera pose relative to the second physical object, wherein the fourth SR object is different from the first third object 
 causing presentation of the third SR object in association with the first physical object and the fourth SR object in association with the second physical object via the display device. 
   
     
     
         4 . The method of  claim 1 , wherein the first SR object is overlaid or superimposed on a portion of the first physical object, and wherein the second SR object is overlaid or superimposed on a portion of the second physical object. 
     
     
         5 . The method of  claim 1 , wherein the first SR object is displayed around or adjacent to a portion of the first physical object, and wherein the second SR object is displayed around or adjacent to a portion of the second physical object. 
     
     
         6 . The method of  claim 1 , wherein the first and second SR objects are selected from a library of pre-existing SR content associated with the first and second physical objects. 
     
     
         7 . The method of  claim 1 , wherein the first and second SR objects are generated based on sensor data associated with the first and second physical objects. 
     
     
         8 . The method of  claim 1 , wherein the first and second SR objects are generated based on video content associated with the first and second physical objects. 
     
     
         9 . The method of  claim 1 , further comprising:
 displaying a first affordance provided to initiate display of the first SR object in response to obtaining the first SR object associated with the first physical object based on the first camera pose relative to the first physical object, wherein the first SR object is displayed in association with the first physical object in response to detecting selection of the first affordance; and   displaying a second affordance provided to initiate display of the second SR object in response to obtaining the second SR object associated with the second physical object based on the first camera pose relative to the second physical object, wherein the second SR object is displayed in association with the second physical object in response to detecting selection of the second affordance.   
     
     
         10 . A computing system comprising:
 one or more processors;   a communication interface for communicating with a display device, one or more input devices, and an image sensor;   a non-transitory memory; and   one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the computing system to:
 obtain, via the image sensor, image data associated with a physical setting; 
 recognize a first physical object and a second physical object within the image data associated with the physical setting; 
 determine a first camera pose associated with the image data relative to the first and second physical objects; 
 obtain a first synthesized reality (SR) object associated with the first physical object based on the first camera pose relative to the first physical object; 
 obtain a second SR object associated with the second physical object based on the first camera pose relative to the second physical object, wherein the second SR object is different from the first SR object 
 cause presentation of the first SR object in association with the first physical object and the second SR object in association with the second physical object via the display device. 
   
     
     
         11 . The computing system of  claim 10 , wherein obtaining the first SR object associated with the first physical object based on the first camera pose relative to the first physical object includes:
 obtaining a third SR object associated with the first physical object in accordance with a determination that the first camera pose relative to the first physical object correspond to a first point-of-view (POV) of the first physical object; and   obtaining a fourth SR object associated with the first physical object, wherein the fourth SR object is different from the third SR object, in accordance with a determination that the first camera pose relative to the first physical object correspond to a second POV of the first physical object different from the first POV of the first physical object.   
     
     
         12 . The computing system of  claim 10 , wherein the one or more programs further cause the computing system to:
 detect a change from the first camera pose to a second camera pose associated with the image data relative to the first and second physical objects; and   in response to detecting the change to the second camera pose:
 obtain a third SR object associated with the first physical object based on the second camera pose relative to the first physical object; 
 obtain a fourth SR object associated with the second physical object based on the second camera pose relative to the second physical object, wherein the fourth SR object is different from the first third object 
 cause presentation of the third SR object in association with the first physical object and the fourth SR object in association with the second physical object via the display device. 
   
     
     
         13 . The computing system of  claim 10 , wherein the first SR object is overlaid or superimposed on a portion of the first physical object, and wherein the second SR object is overlaid or superimposed on a portion of the second physical object. 
     
     
         14 . The computing system of  claim 10 , wherein the first SR object is displayed around or adjacent to a portion of the first physical object, and wherein the second SR object is displayed around or adjacent to a portion of the second physical object. 
     
     
         15 . The computing system of  claim 10 , wherein the one or more programs further cause the computing system to:
 display, via the display device, a first affordance provided to initiate display of the first SR object in response to obtaining the first SR object associated with the first physical object based on the first camera pose relative to the first physical object, wherein the first SR object is displayed in association with the first physical object in response to detecting selection of the first affordance; and   display, via the display device, a second affordance provided to initiate display of the second SR object in response to obtaining the second SR object associated with the second physical object based on the first camera pose relative to the second physical object, wherein the second SR object is displayed in association with the second physical object in response to detecting selection of the second affordance.   
     
     
         16 . A non-transitory memory storing one or more programs, which, when executed by one or more processors of a computing system with a communication interface for communicating with a display device, one or more input devices, and an image sensor, cause the computing system to:
 obtain, via the image sensor, image data associated with a physical setting;   recognize a first physical object and a second physical object within the image data associated with the physical setting;   determine a first camera pose associated with the image data relative to the first and second physical objects;   obtain a first synthesized reality (SR) object associated with the first physical object based on the first camera pose relative to the first physical object;   obtain a second SR object associated with the second physical object based on the first camera pose relative to the second physical object, wherein the second SR object is different from the first SR object   cause presentation of the first SR object in association with the first physical object and the second SR object in association with the second physical object via the display device.   
     
     
         17 . The non-transitory memory of  claim 16 , wherein obtaining the first SR object associated with the first physical object based on the first camera pose relative to the first physical object includes:
 obtaining a third SR object associated with the first physical object in accordance with a determination that the first camera pose relative to the first physical object correspond to a first point-of-view (POV) of the first physical object; and   obtaining a fourth SR object associated with the first physical object, wherein the fourth SR object is different from the third SR object, in accordance with a determination that the first camera pose relative to the first physical object correspond to a second POV of the first physical object different from the first POV of the first physical object.   
     
     
         18 . The non-transitory memory of  claim 16 , wherein the one or more programs further cause the computing system to:
 detect a change from the first camera pose to a second camera pose associated with the image data relative to the first and second physical objects; and   in response to detecting the change to the second camera pose:
 obtain a third SR object associated with the first physical object based on the second camera pose relative to the first physical object; 
 obtain a fourth SR object associated with the second physical object based on the second camera pose relative to the second physical object, wherein the fourth SR object is different from the first third object 
 cause presentation of the third SR object in association with the first physical object and the fourth SR object in association with the second physical object via the display device. 
   
     
     
         19 . The non-transitory memory of  claim 16 , wherein the first SR object is overlaid or superimposed on a portion of the first physical object, and wherein the second SR object is overlaid or superimposed on a portion of the second physical object. 
     
     
         20 . The non-transitory memory of  claim 16 , wherein the first SR object is displayed around or adjacent to a portion of the first physical object, and wherein the second SR object is displayed around or adjacent to a portion of the second physical object. 
     
     
         21 . The non-transitory memory of  claim 16 , wherein the one or more programs further cause the computing system to:
 display, via the display device, a first affordance provided to initiate display of the first SR object in response to obtaining the first SR object associated with the first physical object based on the first camera pose relative to the first physical object, wherein the first SR object is displayed in association with the first physical object in response to detecting selection of the first affordance; and   display, via the display device, a second affordance provided to initiate display of the second SR object in response to obtaining the second SR object associated with the second physical object based on the first camera pose relative to the second physical object, wherein the second SR object is displayed in association with the second physical object in response to detecting selection of the second affordance.