Temporal segmentation

Abstract

A method includes obtaining a plurality of sequential images. The method includes identifying a first plurality of pixel sets respectively associated with the plurality of sequential images. Each of the first plurality of pixel sets locates a first object within a corresponding sequential image on a per-pixel basis. The method includes determining a characterization vector of the first object by extracting a corresponding plurality of displayable characteristic values based on the first plurality of pixel sets. Each of the plurality of displayable characteristic values is associated with a corresponding pixel set and a corresponding sequential image. The characterization vector includes a plurality of distinct temporal values respectively associated with the corresponding plurality of displayable characteristic values. The method includes generating a first behavioral value for the first object based on a function of the corresponding plurality of displayable characteristic values and the plurality of distinct temporal values.

1 . A method comprising:
 at an electronic device including one or more processors, a non-transitory memory, and a display:
 determining, for each of a plurality of sequential images, a corresponding pixel set corresponding to a first object; 
 determining a plurality of displayable characteristic values respectively associated with the plurality of sequential images, wherein each of the plurality of displayable characteristic values is based on a corresponding pixel set of the plurality of corresponding pixel sets; 
 generating a first behavioral value for the first object based on the plurality of displayable characteristic values and a plurality of distinct temporal values respectively associated with the plurality of sequential images; and 
 displaying, on the display, an objective-effectuator based on the first behavioral value, wherein the objective-effectuator is configured to perform an action. 
   
     
     
         2 . The method of  claim 1 , wherein at least one of the plurality of displayable characteristic values corresponds to a feature of the first object. 
     
     
         3 . The method of  claim 2 , wherein the feature corresponds to a material feature of the first object. 
     
     
         4 . The method of  claim 2 , wherein the feature corresponds to a shape or form of the first object. 
     
     
         5 . The method of  claim 2 , wherein the feature corresponds to an amount of deformation of the first object. 
     
     
         6 . The method of  claim 1 , wherein at least one of the plurality of displayable characteristic values corresponds to a instance label value. 
     
     
         7 . The method of  claim 1 , wherein at least one of the plurality of displayable characteristic values corresponds to a semantic label value, and wherein the semantic label value provides an understand or meaning regarding the first object. 
     
     
         8 . The method of  claim 1 , wherein each of the plurality of corresponding pixel sets locates the first object within a corresponding one of the plurality of sequential images on a pixel-by-pixel basis. 
     
     
         9 . The method of  claim 1 , wherein the action satisfies one or more objectives. 
     
     
         10 . The method of  claim 1 , further comprising:
 receiving an input to instantiate the objective-effectuator in an emergent content container; and   in response to receiving the input:
 instantiating the objective-effectuator in the emergent content container; and 
 determining the action. 
   
     
     
         11 . The method of  claim 10 , further comprising displaying the emergent content container on the display, wherein the input corresponds to dragging the objective-effectuator to the emergent content container. 
     
     
         12 . The method of  claim 10 , wherein the emergent content container determines the action based on the action satisfying a set of predefined objectives. 
     
     
         13 . The method of  claim 1 , wherein the electronic device includes an image sensor, and wherein the image sensor captures the plurality of sequential images. 
     
     
         14 . The method of  claim 1 , wherein determining a particular one of the plurality of corresponding pixel sets includes identifying, within a corresponding one of the plurality of sequential images, a respective region including the first object in accordance with a determination that the respective region satisfies an object confidence threshold for the first object. 
     
     
         15 . The method of  claim 1 , wherein generating the first behavioral value includes comparing each of the plurality of displayable characteristic values against a corresponding one of the plurality of distinct temporal values. 
     
     
         16 . An electronic device comprising:
 one or more processors;   a non-transitory memory;   a display; and   one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing operations comprising:
 determining, for each of a plurality of sequential images, a corresponding pixel set corresponding to a first object; 
 determining a plurality of displayable characteristic values respectively associated with the plurality of sequential images, wherein each of the plurality of displayable characteristic values is based on a corresponding pixel set of the plurality of corresponding pixel sets; 
 generating a first behavioral value for the first object based on the plurality of displayable characteristic values and a plurality of distinct temporal values respectively associated with the plurality of sequential images; and 
 displaying, on the display, an objective-effectuator based on the first behavioral value, wherein the objective-effectuator is configured to perform an action. 
   
     
     
         17 . The electronic device of  claim 16 , wherein at least one of the plurality of displayable characteristic values corresponds to a feature of the first object. 
     
     
         18 . The electronic device of  claim 16 , wherein the action satisfies one or more objectives. 
     
     
         19 . The electronic device of  claim 16 , the one or more programs including further instructions for performing operations comprising:
 receiving an input to instantiate the objective-effectuator in an emergent content container; and   in response to receiving the input:
 instantiating the objective-effectuator in the emergent content container; and 
 determining the action. 
   
     
     
         20 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device including a display, cause the electronic device to:
 determine, for each of a plurality of sequential images, a corresponding pixel set corresponding to a first object;   determine a plurality of displayable characteristic values respectively associated with the plurality of sequential images, wherein each of the plurality of displayable characteristic values is based on a corresponding pixel set of the plurality of corresponding pixel sets;   generate a first behavioral value for the first object based on the plurality of displayable characteristic values and a plurality of distinct temporal values respectively associated with the plurality of sequential images; and   display, on the display, an objective-effectuator based on the first behavioral value, wherein the objective-effectuator is configured to perform an action.