Method and system for an automatic sensing, analysis, composition and direction of a 3d space, scene, object, and equipment

Abstract

Method and system for automatic composition and orchestration of a 3D space or scene using networked devices and computer vision to bring ease of use and autonomy to a range of compositions. A scene, its objects, subjects and background are identified and classified, and relationships and behaviors are deduced through analysis. Compositional theories are applied, and context attributes (for example location, external data, camera metadata, and the relative positions of subjects and objects in the scene) are considered automatically to produce optimal composition and allow for direction of networked equipment and devices. Events inform the capture process, for example, a video recording initiated when a rock climber waves her hand, an autonomous camera automatically adjusting to keep her body in frame throughout the sequence of moves. Model analysis allows for direction, including audio tones to indicate proper form for the subject and instructions sent to equipment ensure optimal scene orchestration.

What is claimed is: 
     
         1 . A method, comprising:
 Capturing a 2D image in a specific format of an object, subject, and scene using a device;   Sensing an object, subject, and scene automatically and continuously using the device;   Analyzing the 2D image of the object, subject, and scene captured to determine the most relevant composition and direction model;   Transforming an object, subject, and scene into a 3D model using existing reference composition/architecture model; and   Storing the 3D model of the scene in a database for use and maintaining it in a feedback loop.   
     
     
         2 . The method of  claim 1 , further comprising:
 Performing continuous contextual analysis of an image and its resulting 3D model to provide an update to subsequent 3D modeling processes; and   Dynamically updating and responding to contextual analytics performed.   
     
     
         3 . The method of  claim 2 , further comprising:
 Coordinating accurate tracking of objects and subjects in a scene by orchestrating autonomous equipment movements using a feedback loop.   
     
     
         4 . The method of  claim 3 , further comprising:
 Controlling the direction of a scene and its subjects via devices using a feedback loop   
     
     
         5 . The method of  claim 4 , further comprising:
 Creating and dynamically modifying in real-time the 2D or 3D model for the subject, object, scene, and equipment in any spatial orientation and Providing immediate feedback in a user interface.   
     
     
         6 . The method of  claim 1 , wherein the device is at least one of a camera, wearable device, desktop computer, laptop computer, phone, tablet, and other mobile computer. 
     
     
         7 . A system, comprising:
 A processing unit that can exist on a user device, on-premise, or as an off-premise service to house the following modules;   A sensing module that can understand the subjects and context of a scene over time via models;   An analytics module that can analyze sensed scenes and subjects to determine the most relevant composition and direction models or create them if necessary;   A composition/architecture module that can simultaneously store the direction of multiple subjects or objects of a scene according to one or more composition models;   A direction/control module that can provide direction and control to each subject, object, and equipment individually and relative to a scene model; and   A database that can store models for use and maintain them in a feedback loop with the above modules.