Apparatuses, computer program products, and computer-implemented methods for hardware-accelerated video stream synthesizing

Abstract

Embodiments of the present disclosure efficiently create a single video stream from a plurality of individual video streams. The single video stream may embody a grid-based presentation of the individual video streams, without requiring separate objects for each stream. Such embodiments utilize particular hardware-accelerated multiplexing image pipelines executed via one or more graphics processing unit for efficient processing. Embodiments include a multiplexer component configured to perform the hardware-accelerated image pipelines executed on one or more GPUs. Some such embodiments output a common output video that may be outputted for a variety of use cases, including but without limitation outputting to a display wall, outputting to one or more web-based client devices, outputting locally to a client device, and outputting locally to a multimonitor display. Some embodiments generate utilize an image primitive frame that is sharable between multiplexers for use in different use cases without need to repeat processing steps.

What is claimed is: 
     
         1 . An apparatus for hardware-accelerated video multiplexing, the apparatus comprising:
 at least one graphics processing unit; and   at least one processor and at least one non-transitory memory including computer-coded instructions thereon, the computer coded instructions, with the at least one processor, cause the apparatus to:   receive a plurality of input video streams from a plurality of video input sources;   process each input video stream of the plurality of input video streams via the at least one graphics processing unit, each input video stream associated with a particular multiplexing image pipeline of a plurality of multiplexing image pipelines, wherein each multiplexing image pipeline is executed via the at least one graphical processing unit and is configured to:   parse a frame from the particular input video stream;   copy the frame to the at least one graphics processing unit;   process, via the particular multiplexing image pipeline executed by the at least one graphics processing unit, the frame to generate a processed frame, wherein the processing of the frame includes:   generating decoded frame data by decoding the frame based at least in part on an input encoding corresponding to the particular input video stream;   generating resized frame data by resizing the decoded frame data to an output frame size corresponding to the particular input video stream;   generating reformatted frame data by reformatting the resized frame data to a base format; and   copying the reformatted frame data to the portion of the output frame in an output buffer; and   output, via the at least one graphics processing unit, the processed frame to a portion of an output frame, the portion of the output frame corresponding to the particular input video stream; and   output, via the at least one graphics processing unit, the output frame to a common output video representing a transmissible stream of data or a data file.   
     
     
         2 . The apparatus of  claim 1 , wherein to output each output frame to the common output video, the apparatus is configured to:
 write, via the at least one graphics processing unit, the output frame to a file comprising the common output video.   
     
     
         3 . The apparatus of  claim 1 , wherein to output each output frame to a single output video, the apparatus is configured to:
 generate, via the at least one graphics processing unit, a packet comprising at least a portion of the common output video.   
     
     
         4 . The apparatus of  claim 1 , wherein the output frame comprises a grid comprising a plurality of rows and a plurality of columns, wherein the portion of the output frame corresponding to the particular input video stream comprises a defined range of rows and columns in the grid. 
     
     
         5 . The apparatus of  claim 1 , the at least one graphics processing unit further caused to receive output configuration data comprising:
 first data representing a plurality of URIs corresponding to the plurality of input video streams selected from a plurality of candidate input streams,   second data representing an output frame rate of the common output video,   third data representing an output resolution of the common output video, and/or   fourth data representing a configuration of the common output video.   
     
     
         6 . The apparatus of  claim 1 , wherein the plurality of input video streams comprises a plurality of prerecorded video data, and wherein the at least one graphics processing unit is configured to output the output frame to the common output video at a first framerate that is faster than a second framerate, the second framerate comprising a playback framerate of the common output. 
     
     
         7 . The apparatus of  claim 1 , wherein the base format comprises an RGBa format. 
     
     
         8 . The apparatus of  claim 1 , wherein the multiplexing image pipeline comprises an output pipeline configured to
 copy the output buffer to an encoder; and   generate the output frame by encoding the output buffer via the encoder.   
     
     
         9 . The apparatus of  claim 1 , the apparatus further caused to:
 output the common output video to a plurality of client devices.   
     
     
         10 . The apparatus of  claim 1 , wherein the processed frame comprises an image primitive, the apparatus further caused to:
 receive an indication of a selected output type; and   output the common output video based at least in part on the selected output type.   
     
     
         11 . The apparatus of  claim 1 , the apparatus further caused to:
 receive an inputs modification request to remove a selected input stream from the plurality of input video streams, replace the selected stream from the plurality of input video streams with a replacement input stream, or add a new input stream to the plurality of input video streams; and   alter the plurality of input video streams based at least in accordance with the inputs modification request.   
     
     
         12 . The apparatus of  claim 1 , wherein at least a first multiplexing image pipeline of the plurality of multiplexing image pipelines is associated with a first framerate and a second multiplexing image pipeline of the plurality of multiplexing image pipelines is associated with a second framerate, and wherein to output the output frame the apparatus is caused to:
 synchronize a plurality of portions of the output frame based at least in part on a synchronization frame rate.   
     
     
         13 . The apparatus of  claim 1 , the apparatus further caused to:
 encode the common video output to a target video format.   
     
     
         14 . The apparatus of  claim 1 , wherein the output frame comprises compressed image data, raw image data, or packetized image data. 
     
     
         15 . The apparatus of  claim 1 , the apparatus further caused to:
 decompress each input video stream of the plurality of video streams before processing.   
     
     
         16 . The apparatus of  claim 1 , wherein the common output video represents one selected from:
 the transmissible stream of data communicable in real-time to an external video wall;   the transmissible stream of data communicable in real-time to an external client device;   the transmissible stream of data communicable in real-time to at least one local display;   the data file storable to a local storage; or   the data file playable via a local rendering server.   
     
     
         17 . The apparatus of  claim 1 , the apparatus further caused to:
 receive at least one data-driven alert associated with the plurality of input video streams; and   adjust at least one input video stream of the plurality of input video streams based at least in part on the at least one data-driven alert, wherein the at least one data-driven alert indicates an occurrence of an event requiring a display of the at least one input video stream.   
     
     
         18 . A computer-implemented method comprising:
 receiving, at a system comprising at least one graphics processing unit, a plurality of input video streams from a plurality of video input sources;   processing each input video stream of the plurality of input video streams via the at least one graphics processing unit, each input video stream processed using a particular multiplexing image pipeline of a plurality of multiplexing image pipelines, wherein each multiplexing image pipeline is executed via the at least one graphical processing unit and is configured for:
 parsing, a frame from the particular input video stream; 
 copying, the frame to the at least one graphics processing unit; 
 processing, via the particular multiplexing image pipeline, the frame to generate a processed frame, wherein the processing of the frame includes:
 generating decoded frame data by decoding the frame based at least in part on an input encoding corresponding to the particular input video stream; 
 generating resized frame data by resizing the decoded frame data to an output frame size corresponding to the particular input video stream; 
 generating reformatted frame data by reformatting the resized frame data to a base format; and 
 copying the reformatted frame data to the portion of the output frame in an output buffer; and 
 
 outputting, via the at least one graphics processing unit, the processed frame to a portion of an output frame, the portion of the output frame corresponding to the particular input video stream; and 
   outputting, via the at least one graphics processing unit, the output frame to a common output video.   
     
     
         19 . A computer program product comprising at least one non-transitory computer-readable storage medium having computer program code stored thereon that, in execution with at least one processor, configures the at least one processor for:
 receiving a plurality of input video streams from a plurality of video input sources;   processing each input video stream of the plurality of input video streams via at least one graphics processing unit, each input video stream processed using a particular multiplexing image pipeline of a plurality of multiplexing image pipelines, wherein each multiplexing image pipeline is executed via the at least one graphical processing unit and is configured for:
 parsing, a frame from the particular input video stream; 
 copying, the frame to the at least one graphics processing unit; 
 processing, via the particular multiplexing image pipeline, the frame to generate a processed frame, wherein the processing of the frame includes:
 generating decoded frame data by decoding the frame based at least in part on an input encoding corresponding to the particular input video stream; 
 generating resized frame data by resizing the decoded frame data to an output frame size corresponding to the particular input video stream; 
 generating reformatted frame data by reformatting the resized frame data to a base format; and 
 copying the reformatted frame data to the portion of the output frame in an output buffer; and 
 
 outputting, via the at least one graphics processing unit, the processed frame to a portion of an output frame, the portion of the output frame corresponding to the particular input video stream; and 
   outputting, via the at least one graphics processing unit, the output frame to a common output video.