Viewability measurement in video game stream

Abstract

A computer-implemented method includes obtaining a video stream comprising footage of video game play, detecting an instance of the object within an image frame of the video stream using an object detector trained to detect instances of an object within footage of video game play, and comparing the detected instance of the object with a specimen instance of the object to determine a value of a viewability characteristic for the detected instance of the object.

What is claimed is: 
     
         1 . A system comprising at least one processor and at least one memory storing instructions which, when executed by the at least one processor, cause the at least one processor to carry out operations comprising:
 obtaining a video stream comprising footage of video game play;   detecting, using an object detector trained to detect instances of an object within footage of video game play, an instance of the object within an image frame of the video stream; and   comparing the detected instance of the object with a specimen instance of the object to determine a value of a viewability characteristic for the detected instance of the object.   
     
     
         2 . The system of  claim 1 , wherein:
 the viewability characteristic is a proportion of the detected instance of the object visible within the image frame; and   determining the value of the proportion of the detected instance of the object visible within the image frame comprises:
 determining a transformation relating a geometry of the detected instance of the object to a geometry of the specimen instance of the object; 
 determining a set of pixel positions occupied by the specimen instance of the object when transformed in accordance with the determined transformation and overlaid on the detected instance of the object; and 
 calculating a proportion of pixels of the image frame at the determined set of pixel positions that are occupied by the detected instance of the object. 
   
     
     
         3 . The system of  claim 2 , wherein calculating the proportion of pixels of the image frame at the determined set of pixel positions that are occupied by the detected instance of the object comprises:
 mapping pixels of the specimen instance of the object to pixels of the image frame at the determined set of pixel positions using the determined transformation; and   calculating a proportion of the determined set of pixel positions for which a deviation between color values of the pixels of the image frame and color values of the pixels of the specimen instance of the object is less than a threshold value.   
     
     
         4 . The system of  claim 2 , wherein the transformation is a first transformation, and determining the first transformation comprises:
 transforming the specimen instance of the object in accordance with a plurality of candidate transformations; and   identifying the first transformation as one of the plurality of candidate transformations resulting in a best match between pixels of the transformed specimen instance of the object and pixels of the image frame at matching pixel positions when the transformed specimen instance of the object is overlaid on the detected instance of the object.   
     
     
         5 . The system of  claim 2 , wherein determining the transformation comprises:
 estimating characteristics of a bounding polygon for the detected instance of the object; and   determining the transformation by comparing the estimated characteristics of the bounding polygon with characteristics of a bounding polygon for the specimen instance of the object.   
     
     
         6 . The system of  claim 2 , wherein the operations comprise estimating characteristics of a bounding polygon for the detected instance of the object,
 wherein determining the proportion of the detected instance of the object visible in the image frame comprises estimating a proportion of the detected instance of the object appearing within a coordinate range of the image frame by comparing the estimated characteristics of the bounding polygon with characteristics of a bounding polygon for the specimen instance of the object when transformed in accordance with the determined transformation.   
     
     
         7 . The system of  claim 1 , wherein:
 the viewability characteristic is a size of the detected instance of the object in relation to a size of the image frame; and   determining the size of the detected instance of the object in relation to the size of the image frame comprises:
 estimating characteristics of a bounding polygon for the detected instance of the object; and 
 determining the size of the detected instance of the object in relation to the size of the image frame based on the estimated characteristics of the bounding polygon. 
   
     
     
         8 . The system of  claim 1 , wherein:
 the viewability characteristic is a size of the detected instance of the object in relation to a size of the image frame; and   determining the size of the detected instance of the object in relation to the size of the image frame comprises:
 determining a transformation relating a geometry of the detected instance of the object to a geometry of the specimen instance of the object; and 
 determining a number of pixels occupied by the specimen instance of the object transformed in accordance with the determined transformation. 
   
     
     
         9 . The system of  claim 1 , wherein:
 the image frame depicts a three-dimensional environment viewed from a perspective of a virtual camera;   the object has a substantially planar surface; and   the viewability characteristic is a viewing angle between a normal vector to the substantially planar surface and a vector between the virtual camera and a point on the substantially planar surface.   
     
     
         10 . The system of  claim 9 , wherein determining the viewing angle comprises:
 transforming the specimen instance of the object in accordance with a plurality of candidate transformations, each of the candidate transformations including a respective rotation; and   identifying the value of the viewing angle from the respective rotation of one of the plurality of candidate transformations resulting in a best match between pixels of the transformed specimen instance of the object and pixels of the image frame at matching pixel positions when the transformed specimen instance of the object is overlaid on the detected instance of the object.   
     
     
         11 . The system of  claim 9 , wherein determining the value of the viewing angle comprises:
 estimating characteristics of a bounding polygon for the detected instance of the object; and   determining the viewing angle by comparing the estimated characteristics of the bounding polygon with characteristics of a bounding polygon for the specimen instance of the object.   
     
     
         12 . The system of  claim 1 , wherein the operations further comprise training the object detector using steps comprising:
 for a plurality of input image frames:
 applying a transformation to the specimen instance of the object to generate a transformed instance of the object; 
 inserting the transformed instance of the object into the input image frame to generate a training image frame; and 
 determining associated label data for the training image frame indicating at least a location of the transformed instance of the object within the training image frame; and 
   training the object detector using supervised learning with the generated training image frames and associated label data.   
     
     
         13 . The system of  claim 12 , wherein for at least one of the plurality of input image frames, generating the training image frame comprises overlaying an occluding object on the transformed instance of the object and/or applying a distortion effect to the transformed instance of the object. 
     
     
         14 . The system of  claim 1 , wherein the image frame is a first image frame, the operations further comprising:
 tracking the instance of the object over a plurality of image frames including the first image frame; and   aggregating values of the viewability characteristic over the plurality of image frames to determine an aggregated value of the viewability characteristic for the instance of the object.   
     
     
         15 . The system of  claim 1 , wherein:
 the object changes appearance in a predetermined manner between image frames; and   comparing the detected instance of the object with the specimen instance of the object comprises synchronizing a timing of the detected instance of the object with a timing of the specimen instance of the object.   
     
     
         16 . The system of  claim 1 , wherein:
 the object is an in-game advert; and   the operations include counting an impression of the in-game advert in dependence on the determined value of the viewability characteristic.   
     
     
         17 . The system of  claim 16 , wherein the operations include:
 determining a number of views of the video stream by users of a streaming service; and   determining a total number of impressions of the in-game advert in proportion to the number of views of the video stream by the users of the streaming service and a number of impressions counted in the video stream.   
     
     
         18 . The system of  claim 16 , wherein the image frame is a first image frame, the operations comprising:
 tracking the instance of the in-game advert over a plurality of image frames including the first image frame; and   counting the impression in dependence on determining that the values of the viewability characteristic satisfy a set of criteria for at least a predetermined number of frames.   
     
     
         19 . A computer-implemented method comprising, using one or more processors:
 obtaining a video stream comprising footage of video game play;   detecting, using an object detector trained to detect instances of an object within footage of video game play, an instance of the object within an image frame of the video stream; and   comparing the detected instance of the object with a specimen instance of the object to determine a value of a viewability characteristic for the detected instance of the object.   
     
     
         20 . One or more non-transient storage media comprising instructions which, when executed by the one or more processors, cause the one or more processors to carry out operations comprising:
 obtaining a video stream comprising footage of video game play;   detecting, using an object detector trained to detect instances of an object within footage of video game play, an instance of the object within an image frame of the video stream; and   comparing the detected instance of the object with a specimen instance of the object to determine a value of a viewability characteristic for the detected instance of the object.