Method and apparatus for measuring stream availability, quality and performance

Abstract

A streaming measurement agent designed to experience, measure, and report on a media stream as an actual end user would experience the stream. Preferably, agent resides transparently within a streaming media player itself so that it can monitor stream packet flows within the player as the measured streams are being played. In an illustrative embodiment, the agent comprises a performance monitoring module (PMM), which is software that resides in an interface between an existing core module and a renderer of a media player. The agent PMM intercepts each useful packet as it goes from the core to the renderer and, as a result, it is able to compute quality metrics about the playback. The agent functions “transparently” to the media player by presenting the core with an application programming interface (API) that emulates the API that the renderer normally presents to the core. Thus, when the core believes it is calling the renderer, it is actually calling the agent PMM, which can then receive all the packets from the core and process them. After computing relevant performance metrics using the packets it receives, the agent PMM calls the renderer. A set of performance agents can be managed by a service provider to enable a content provider to determine how a stream is perceived by end users.

1 . A monitoring system, comprising:
 one or more monitoring computers including a first sub-system to provision a quality test;   a set of monitoring agents positioned at given network locations to execute the quality test;   where each of the monitoring agents is associated with a media player, each monitoring agent comprising (i) first code to receive test data identifying at least one quality test and to initiate the at least one quality test at a given time, (ii) second code executed in association with the media player to capture a quality metric from one or more useful packets of a stream, the useful packet has a given set of properties, wherein the properties are that the packet has not been thinned by a server, that the packet was not lost in an unrecoverable manner during transport, and that the packet arrived at the renderer in time to be useful for rendering, and (iii) third code to output data associated with the quality metric;   the one or more monitoring computers also including a second sub-system to collect the data from the set of monitoring agents.   
   
   
       2 . The system as described in  claim 1  wherein the test suite includes a set of parameters selected from a set that includes: a start time, and at least one test. 
   
   
       3 . The system as described in  claim 1  wherein the quality test includes a set of parameters selected from a set that includes: a URL, a maximum startup time, a maximum stream time, and a maximum play time. 
   
   
       4 . The system as described in  claim 1  wherein the quality metric is actual playback bandwidth. 
   
   
       5 . The system as described in  claim 1  wherein the quality metric is useful packet rate. 
   
   
       6 . The system as described in  claim 1  wherein the quality metric is generated using a media player callback function and is selected from a set of metrics that include: startup time, startup time components, and interruption and rebuffer statistics. 
   
   
       7 . The system as described in  claim 1  further including a machine that enables a view of the collected data.