Pipeline systems and method for transferring data in a network environment

Abstract

A communications system having a data transfer pipeline apparatus for transferring data in a sequence of N stages from an origination device to a destination device. The apparatus comprises dedicated memory having buffers dedicated for carrying data and a master control for registering and controlling processes associated with the apparatus for participation in the N stage data transfer sequence. The processes include a first stage process for initiating the data transfer and a last Nth stage process for completing data transfer. The first stage process allocates a buffer from a predetermined number of buffers available within the memory for collection, processing, and sending of the data from the origination device to a next stage process. The Nth stage process receives a buffer allocated to the first stage process from the (N−1)th stage and to free the buffer upon processing completion to permit reallocation of the buffer.

What is claimed is: 
     
         1 . A pipeline system for providing data transfer between multiple computing devices, the pipeline system comprising:
 a datapipe that spans multiple computing devices, the datapipe comprising a sequence of stages for transferring data from an origination computing device to a destination computing device, wherein the datapipe is identified on the origination computing device and the destination computing device with a data pipeline identifier;   one or more control modules configured to control at least a first stage of the sequence of stages on the origination computing device;   a first dedicated memory comprising a first pool of buffers, wherein the one or more control modules allocate at least a first buffer from the first pool of buffers to the first stage, and wherein the first buffer is associated with the data pipeline identifier until freed by the one or more control modules;   one or more of the control modules further configured to control at least a second stage of the sequence of stages on the destination computing device; and   a second dedicated memory comprising a second pool of buffers, wherein the one or more control modules allocate at least a second buffer from the second pool of buffers to the second stage, and wherein the second buffer is associated with the data pipeline identifier until freed by the one or more control modules.