Scalable data storage architecture and methods of eliminating I/O traffic bottlenecks

Abstract

A Storage Area Network (SAN) system has host computers, front-end SAN controllers (FE_SAN) connected via a bus or network interconnect to back-end SAN controllers (BE_SAN), and physical disk drives connected via network interconnect to the BE_SANs to provide distributed high performance centrally managed storage. Described are hardware and software architectural solutions designed to eliminate I/O traffic bottlenecks, improve scalability, and reduce the overall cost of SAN systems. In an embodiment, the BE_SAN has firmware to recognize when, in order to support a multidisc volume, such as a RAID volume, it is configured to support, it requires access to a physical disk attached to a second BE_SAN; when such a reference is recognized it passes assess commands to the second BE_SAN. Buffer memory of each FE_SAN is mapped into application memory space to increase access speed, where multiple hosts share an LBA the BE_SAN tracks writes and invalidates the unwritten buffers.

What is claimed is: 
     
       1. A data processing system having a storage area network (SAN), the system comprising:
 at least a first and a second host processor central units (CPUs), the first CPU coupled to a first FE_SAN (Front End SAN) controller and the second CPU coupled to a second FE_SAN controller, the first CPU configured to execute first FE_SAN virtualization firmware, and the second CPU configured to execute second FE_SAN virtualization firmware; 
 at least a first and a second Back End SAN controller (BE_SAN), each BE_SAN being interconnected via at least one back-end network to a plurality of storage devices; 
 a network interconnecting the FE_SANs with the BE_SANs; and 
 BE_SAN virtualization firmware executing on the BE_SANs; 
 wherein the FE_SAN virtualization firmware and the BE_SAN virtualization firmware are together configured to request data from a particular BE_SAN of the at least a first and a second BE_SANs when data is required by a particular requesting host CPU of the host CPUs; and 
 wherein an operating system (OS) executing on a first CPU maps a portion of physical memory as a first buffer of a first logical block address (LBA) into an application's address space allowing the application to directly access that memory; 
 wherein a copy-on-write method is used to automatically copy data from the first buffer to at least one corresponding buffer allocated on the first BE_SAN; 
 wherein, if an operating system executing a host maps a portion of the second FE_SAN controller's memory as a second buffer of the first LBA into an application's address space, and if data of the first buffer is modified by a host coupled to the first FE_SAN, the first BE_SAN controller notifies the second FE_SAN to invalidate data of the second buffer. 
 
     
     
       2. The system of  claim 1  wherein the FE_SAN and BE_SAN firmware is configured for writeback caching without mirroring. 
     
     
       3. A data processing system having a storage area network (SAN), the system comprising:
 at least a first and a second host processor central units (CPUs), the first CPU coupled to a first FE_SAN (Front End SAN) controller and the second CPU coupled to a second FE_SAN controller, the first FE_SAN controller and first CPU configured to execute first FE_SAN virtualization firmware, and the second FE_SAN controller and second CPU configured to execute second FE_SAN virtualization firmware; 
 at least a first and a second Back End SAN controller (BE_SAN), each BE_SAN being interconnected via at least one back-end network to a plurality of storage devices; 
 a network interconnecting the FE_SANs with the BE_SANs; and 
 BE_SAN virtualization firmware executing on the BE_SANs; 
 wherein the FE_SAN virtualization firmware and the BE_SAN virtualization firmware are together configured to request data from a particular BE_SAN of the at least a first and a second BE_SANs when data is required by a particular requesting host CPU of the host CPUs; and 
 wherein an operating system (OS) executing on a first CPU maps a portion of physical memory as a first buffer of a first logical block address (LBA) into an application's address space allowing the application to directly access that memory; 
 wherein a copy-on-write method is used to automatically copy data from the first buffer to at least one corresponding buffer allocated on the first BE_SAN; 
 wherein, if an operating system executing a host maps a portion of the second FE_SAN controller's physical memory as a second buffer of the first LBA into an application's address space, and if data of the first buffer is modified by a host coupled to the first FE_SAN, the first BE_SAN controller notifies the second FE_SAN to invalidate data of the second buffer; and 
 wherein the FE_SAN and BE_SAN firmware is configured for writeback caching with mirroring. 
 
     
     
       4. A data processing system having a storage area network (SAN), the system comprising:
 at least a first and a second host processor central units (CPUs), the first CPU coupled to a first FE_SAN (Front End SAN) controller and the second CPU coupled to a second FE_SAN controller, the first FE_SAN controller and first CPU configured to execute first FE_SAN virtualization firmware, and the second FE_SAN controller and second CPU configured to execute second FE_SAN virtualization firmware; 
 at least a first and a second Back End SAN controller (BE_SAN), each BE_SAN being interconnected via at least one back-end network to a plurality of storage devices; 
 a network interconnecting the FE_SANs with the BE_SANs; and 
 BE_SAN virtualization firmware executing on the BE_SANs; 
 wherein the FE_SAN virtualization firmware and the BE_SAN virtualization firmware are together configured to request data from a particular BE_SAN of the at least a first and a second BE_SANs when data is required by a particular requesting host CPU of the host CPUs; and 
 wherein an operating system (OS) executing on a first CPU maps a portion of physical memory as a first buffer of a first logical block address (LBA) into an application's address space allowing the application to directly access that memory; 
 wherein a copy-on-write method is used to automatically copy data from the first buffer to at least one corresponding buffer allocated on the first BE_SAN; 
 wherein, if an operating system executing a host maps a portion of the second FE_SAN controller's physical memory as a second buffer of the first LBA into an application's address space, and if data of the first buffer is modified by a host coupled to the first FE_SAN, the first BE_SAN controller notifies the second FE_SAN to invalidate data of the second buffer; and 
 wherein the FE_SAN and BE_SAN firmware is configured for writeback caching with and without mirroring. 
 
     
     
       5. The system of  claim 4  wherein the FE_SAN and BE_SAN firmware is configured for writeback caching without mirroring.