Systems and methods for dynamically augmenting videos via in-video insertion on mobile devices

Abstract

Disclosed are systems and methods for rendering augmented videos on mobile devices and computing environment with limited computational resources. The disclosed systems and methods provide a novel framework for performing automatic detection of surfaces in video frames resulting in the creation of a seamless in-video augmentation object experience for viewing users. The disclosed framework operates by leveraging available surfaces in digital content to show augmentation objects in compliance with various pre-established contextual and technical constraints. The disclosed framework evidences a streamlined, automatic and computationally efficient process(es) that modifies digital content at the surface level within the frames of the digital content based on the contextual and technical constraints, and the computational resources of the device augmented digital content is rendered on.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method for dynamically augmenting content, comprising:
 receiving digital content and corresponding surface constraints;   analyzing the digital content to identify one or more surfaces;   comparing the identified surfaces to the corresponding surface constraints to determine a manifest of surfaces that satisfy the corresponding surface constraints;   receiving a set of endorser constraints for one or more surfaces;   comparing the manifest of surfaces to the endorser constraints to determine a subset of surfaces that satisfy the endorser constraints;   identifying digital objects that satisfy the subset of surfaces; and   rendering an augmented digital content such that the identified digital objects are displayed on at least one surface of the subset of surfaces.   
     
     
         22 . The computer-implemented method of  claim 21 , further comprising:
 analyzing the digital content in order to identify known features; and   generating a companion video based on the identified known features.   
     
     
         23 . The computer-implemented method of  claim 21 , further comprising:
 calculating a current frame of the digital content; and   determining if augmentation slots are available for the current frame.   
     
     
         24 . The computer-implemented method of  claim 21 , wherein determining a subset of surfaces that satisfy the endorser constraints further comprises:
 scoring the subset of surfaces based on a comparison of the manifest of surfaces and the subset of surfaces satisfying the endorser constraints.   
     
     
         25 . The computer-implemented method of  claim 21 , further comprises generating homography transformation data for a graphical processing unit in order to render the augmented digital content. 
     
     
         26 . The computer-implemented method of  claim 25 , wherein generating homography transformation data for a graphical processing unit further comprises:
 implementing one or more of: a direct linear transformation algorithm utilizing a homography matrix, or automatic image mosaicking.   
     
     
         27 . The computer-implemented method of  claim 21 , further comprising:
 evaluating a current cache state by calculating past and future buffer size; and cancelling current cache fill operations and destroying cache contents that exceed the calculated past and future buffer size.   
     
     
         28 . The computer-implemented method of  claim 21 , wherein the digital content and one or more augmentation objects is one or more of a video, text, audio, and/or images. 
     
     
         29 . A system for dynamically augmenting content, comprising:
 a processor and storage medium storing instructions, the processor configured to execute the instructions to perform a method including:   receiving digital content and corresponding surface constraints;   analyzing the digital content to identify one or more surfaces;   comparing the identified surfaces to the corresponding surface constraints to determine a manifest of surfaces that satisfy the corresponding surface constraints;   receiving a set of endorser constraints for one or more surfaces;   comparing the manifest of surfaces to the endorser constraints to determine a subset of surfaces that satisfy the endorser constraints;   identifying digital objects that satisfy the subset of surfaces; and   rendering an augmented digital content such that the identified digital objects are displayed on at least one surface of the subset of surfaces.   
     
     
         30 . The system of  claim 29 , further comprising:
 analyzing the digital content in order to identify known features; and   generating a companion video based on the identified known features.   
     
     
         31 . The system of  claim 29 , further comprising:
 calculating a current frame of the digital content; and   determining if augmentation slots are available for the current frame.   
     
     
         32 . The system of  claim 29 , wherein determining a subset of surfaces that satisfy the endorser constraints further comprises:
 scoring the subset of surfaces based on a comparison of the manifest of surfaces and the subset of surfaces satisfying the endorser constraints.   
     
     
         33 . The system of  claim 29 , further comprises generating homography transformation data for a graphical processing unit in order to render the augmented digital content. 
     
     
         34 . The system of  claim 33 , wherein generating homography transformation data for a graphical processing unit further comprises:
 implementing one or more of: a direct linear transformation algorithm utilizing a homography matrix, or automatic image mosaicking.   
     
     
         35 . The system of  claim 29 , further comprising:
 evaluating a current cache state by calculating past and future buffer size; and cancelling current cache fill operations and destroying cache contents that exceed the calculated past and future buffer size.   
     
     
         36 . The system of  claim 29 , wherein the digital content and one or more augmentation objects is one or more of a video, text, audio, and/or images. 
     
     
         37 . A non-transitory computer readable medium for dynamically augmenting content, tangibly encoded with computer-executable instructions, that when executed by a processor performs a method comprising:
 receiving digital content and corresponding surface constraints;   analyzing the digital content to identify one or more surfaces;   comparing the identified surfaces to the corresponding surface constraints to determine a manifest of surfaces that satisfy the corresponding surface constraints;   receiving a set of endorser constraints for one or more surfaces;   comparing the manifest of surfaces to the endorser constraints to determine a subset of surfaces that satisfy the endorser constraints;   identifying digital objects that satisfy the subset of surfaces; and   rendering an augmented digital content such that the identified digital objects are displayed on at least one surface of the subset of surfaces.   
     
     
         38 . The non-transitory computer readable medium of  claim 37 , further comprising:
 analyzing the digital content in order to identify known features; and   generating a companion video based on the identified known features.   
     
     
         39 . The non-transitory computer readable medium of  claim 37 , further comprising:
 calculating a current frame of the digital content; and   determining if augmentation slots are available for the current frame.   
     
     
         40 . The non-transitory computer readable medium of  claim 37 , wherein determining a subset of surfaces that satisfy the endorser constraints further comprises:
 scoring the subset of surfaces based on a comparison of the manifest of surfaces and the subset of surfaces satisfying the endorser constraints.