Method and apparatus for preconditioning data to be transferred on a switched underlay network

Abstract

Data may be preconditioned to be transferred on a switched underlay network to alleviate the data access and transfer rate mismatch, so that large files may be effectively transferred on the network at optical networking speeds. A data meta-manager service may be provided on the network to interface a data source and/or data target to prepare a data file for transmission, such as by dividing a large file into multiple pieces and causing those pieces to be stored on multiple storage subsystems. The file may then be read from the multiple storage subsystems simultaneously and multiplexed onto scheduled resources on the network. This enables the high bandwidth transfer resource to be filled by a data transfer without requiring the storage subsystem to be augmented to output the data at the network transfer rate. The file may be de-multiplexed at the data target to one or more storage subsystems.

1 . A method of preconditioning data to be transferred on a switched underlay network, the method comprising the steps of: 
 causing data to be moved from a first storage subsystem having a first data read rate to a plurality of second storage subsystems having a collective read rate of greater magnitude than the first data read rate; and    causing the data to be read out of the plurality of second storage subsystems at the collective read rate.    
     
     
         2 . The method of  claim 1 , wherein the first read data rate is lower than a data transfer rate on the switched underlay network.  
     
     
         3 . The method of  claim 1 , wherein the plurality of second storage subsystems comprises the first storage subsystem and additional storage subsystems.  
     
     
         4 . The method of  claim 1 , wherein the data is provided to a network element configured to multiplex the data from the plurality of second storage subsystems onto the switched underlay network.  
     
     
         5 . The method of  claim 4 , wherein the second storage subsystems are geographically closer to the network element than the first storage subsystem.  
     
     
         6 . The method of  claim 4 , wherein the second storage subsystems are connected to the network element over links having a higher bandwidth than the first storage subsystem.  
     
     
         7 . The method of  claim 1 , wherein the step of causing the data to be moved from the first storage subsystem comprises dividing the data into sections, and moving each of the sections to at least one of the second storage subsystems.  
     
     
         8 . The method of  claim 1 , wherein the collective read rate is based on individual read rates of each of the second storage subsystems.  
     
     
         9 . The method of  claim 1 , further comprising the step of defining a pattern for reading of the data from the plurality of second storage subsystems, and causing the pattern to be communicated to a target storage subsystem.  
     
     
         10 . An apparatus for preconditioning data to be transferred on a switched underlay network, the apparatus comprising: 
 an interface to a storage subsystem containing a file to be transferred, the storage subsystem having a data output interface having a first data read rate;    control logic configured to generate instructions to the storage subsystem to cause the storage subsystem to transfer portions of the file to a plurality of second storage subsystems from which the data may be read at a collective data read rate greater than the first data read rate.    
     
     
         11 . The apparatus of  claim 10 , wherein the control logic is further configured to generate instructions to define a pattern at which the data may be read from the second storage subsystems.  
     
     
         12 . The apparatus of  claim 10 , wherein the instructions generated by the control logic cause the file to be divided into sections, each section of which comprises a portion of the file.  
     
     
         13 . The apparatus of  claim 10 , wherein the portions of the file are copies of the file.  
     
     
         14 . The apparatus of  claim 10 , wherein the control logic is further configured to generate instructions to a network element configured to transfer the data over reserved resources on the switched underlay network, the instructions comprising a multiplexing pattern relating to the data to be read from the second storage subsystems.  
     
     
         15 . The apparatus of  claim 14 , wherein the instructions to the network element comprise buffering instructions as to how the network element should buffer the data prior to transmission on the reserved resources, and instructions as to the identity of the second storage subsystems that will provide data to be transferred on the reserved resources.