Device, computer program and method

Abstract

A method of detecting the real-world position of an object in a real-world scene includes obtaining, from a plurality of cameras each having a different field of view of the object, a plurality of images of the object in the real-world scene over a first time window, each of the plurality of images being provided by a different one of the cameras; determining the position of the object in each image captured by each of the plurality of cameras over the first time window; building a 3D model of the continuous real-world position of the object in the real-world scene over the first time window based upon the determined position in each image captured by the plurality of cameras; and establishing, from the 3D model, the 3D position of the object at a particular time in the real-world scene.

What is claimed is: 
     
         1 . A method of detecting the real-world position of an object in a real-world scene, comprising:
 obtaining, from a plurality of cameras each having a different field of view of the object, a plurality of images of the object in the real-world scene over a first time window, each of the plurality of images being provided by a different one of the cameras;   determining the position of the object in each image captured by each of the plurality of cameras over the first time window;   building a 3D model of the continuous real-world position of the object in the real-world scene over the first time window based upon the determined position in each image captured by the plurality of cameras; and   establishing, from the 3D model, the 3D position of the object at a particular time in the real-world scene.   
     
     
         2 . The method according to  claim 1 , further comprising:
 building a 3D model of the continuous real-world position of the object in the real-world scene over a second time window based upon the determined position in each image captured by the plurality of cameras, wherein the final real-world position of the object in the 3D model of the first time window is the initial real-world position of the object in the 3D model of the second time window.   
     
     
         3 . The method according to  claim 1 , further comprising:
 deleting the 3D model when the object is not detected for a period of time.   
     
     
         4 . The method according to  claim 1 , further comprising:
 establishing the 3D position of the object over a periodic time interval.   
     
     
         5 . The method according to  claim 4 , wherein the periodic time interval is set based upon a frame rate of video being viewed. 
     
     
         6 . The method according to  claim 1 , further comprising:
 checking the detected object in a first of the plurality of images against a list of previously detected objects and, in the event that the object is not in the list of previously detected objects, the method further comprises:   identifying the detected object in a second of the plurality of images, the second image having a different field of view of the detected object to the first image;   determining the position of the object detected in the second image at a synchronised time to the first image;   determining, using the time synchronised first and second image, the real-world position of the object; and   providing the determined real-world position as an initial real-world position of the object in the 3D model.   
     
     
         7 . The method according to  claim 1 , further comprising:
 determining the position of the object in 2D image space using the pixel position of the object in one of the images captured by a respective one of the plurality of cameras and camera information associated with the respective camera, wherein the camera information includes the focal length of the camera and camera heading information.   
     
     
         8 . The method according to  claim 1 , wherein the object is one or more of at least part of a sporting projectile, at least part of an implement used by a player, a player bounding box, player keypoints, key locations on the body, projectile/ball bounding boxes, projectile/ball keypoints, racket/stick/bat bounding boxes, racket/stick/bat key points, team identification information or player identification information. 
     
     
         9 . A device for detecting the real-world position of an object in a real-world scene, comprising circuitry configured to:
 obtain, from a plurality of cameras each having a different field of view of the object, a plurality of images of the object in the real-world scene over a first time window, each of the plurality of images being provided by a different one of the cameras;   determine the position of the object in each image captured by each of the plurality of cameras over the first time window;   build a 3D model of the continuous real-world position of the object in the real-world scene over the first time window based upon the determined position in each image captured by the plurality of cameras; and   establish, from the 3D model, the 3D position of the object at a particular time in the real-world scene.   
     
     
         10 . The device according to  claim 9 , wherein the circuitry is configured to:
 build a 3D model of the continuous real-world position of the object in the real-world scene over a second time window based upon the determined position in each image captured by the plurality of cameras, wherein the final real-world position of the object in the 3D model of the first time window is the initial real-world position of the object in the 3D model of the second time window.   
     
     
         11 . The device according to  claim 9 , wherein the circuitry is further configured to:
 delete the 3D model when the object is not detected for a period of time.   
     
     
         12 . The device according to  claim 9 , wherein the circuitry is further configured to:
 establish the 3D position of the object over a periodic time interval.   
     
     
         13 . The device according to  claim 12 , wherein the periodic time interval is set based upon a frame rate of video being viewed. 
     
     
         14 . The device according to  claim 9 , wherein the circuitry is further configured to:
 check the detected object in a first of the plurality of images against a list of previously detected objects and, in the event that the object is not in the list of previously detected objects, the method further comprises:   identify the detected object in a second of the plurality of images, the second image having a different field of view of the detected object to the first image;   determine the position of the object detected in the second image at a synchronised time to the first image;   determine, using the time synchronised first and second image, the real-world position of the object; and   provide the determined real-world position as an initial real-world position of the object in the 3D model.   
     
     
         15 . The device according to  claim 9 , wherein the circuitry is further configured to:
 determine the position of the object in 2D image space using the pixel position of the object in one of the images captured by a respective one of the plurality of cameras and camera information associated with the respective camera, wherein the camera information includes the focal length of the camera and camera heading information.   
     
     
         16 . The device according to  claim 9 , wherein the object is one or more of at least part of a sporting projectile, at least part of an implement used by a player, a player bounding box, player keypoints, key locations on the body, projectile/ball bounding boxes, projectile/ball keypoints, racket/stick/bat bounding boxes, racket/stick/bat key points, team identification information or player identification information. 
     
     
         17 . A non-transitory computer readable medium storing a computer program comprising computer readable instructions which, when loaded onto a computer, configures the computer to perform the method according to  claim 1 .