Hardware-Assisted Memory Disaggregation with Recovery from Network Failures Using Non-Volatile Memory

Abstract

Techniques for implementing hardware-assisted memory disaggregation with recovery from network failures/problems are provided. In one set of embodiments, a hardware controller of a computer system can maintain a copy of a “remote memory” of the computer system (i.e., a section of the physical memory address space of the computer system that maps to a portion of the physical system memory of a remote computer system) in a local backup memory. The backup memory may be implemented using a non-volatile memory that is slower, but also less expensive, than conventional dynamic random-access memory (DRAM). Then, if the hardware controller is unable to retrieve data in the remote memory from the remote computer system within a specified time window due to, e.g., a network failure or other problem, the hardware controller can retrieve the data from the backup memory, thereby avoiding a hardware error condition (and potential application/system crash).

What is claimed is: 
     
         1 . A method comprising:
 receiving, by a hardware controller of a computer system, a memory load instruction directed to a physical memory address in a remote memory of the computer system, the remote memory being a section of a physical memory address space of the computer system that is mapped to a portion of a physical system memory of another computer system;   transmitting, by the hardware controller, a network packet including the memory load instruction to said another computer system;   waiting, by the hardware controller, for a response network packet from said another computer system that includes data requested via the memory load instruction; and   upon determining that the response network packet has not been received from said another computer system within a threshold time window, retrieving, by the hardware controller, the data from a backup memory attached to the hardware controller.   
     
     
         2 . The method of  claim 1  wherein the portion of the physical system memory of said another computer system is a borrowed memory portion that is inaccessible by other computer systems while it is mapped to the remote memory of the computer system. 
     
     
         3 . The method of  claim 1  wherein the memory load instruction is received from a central processing unit (CPU) of the computer system over a cache-coherent interface, and wherein upon retrieving the data from the backup memory, the hardware controller provides the data to the CPU over the cache-coherent interface. 
     
     
         4 . The method of  claim 3  wherein the threshold time window is based on a property of the cache-coherent interface. 
     
     
         5 . The method of  claim 1  wherein the backup memory is implemented using a non-volatile memory that is configured to hold a copy of all data in the remote memory. 
     
     
         6 . The method of  claim 1  further comprising:
 upon determining that the response network packet has been received from said another computer system within the threshold time window, extracting the data from the response network packet. 
 
     
     
         7 . The method of  claim 1  wherein the backup memory is organized using a log-structured data layout. 
     
     
         8 . A hardware controller implemented in a computer system, the hardware controller being configured to:
 receive a memory load instruction directed to a physical memory address in a remote memory of the computer system, the remote memory being a section of a physical memory address space of the computer system that is mapped to a portion of a physical system memory of another computer system;   transmit a network packet including the memory load instruction to said another computer system;   wait for a response network packet from said another computer system that includes data requested via the memory load instruction; and   upon determining that the response network packet has not been received from said another computer system within a threshold time window, retrieve the data from a backup memory attached to the hardware controller.   
     
     
         9 . The hardware controller of  claim 8  wherein the portion of the physical system memory of said another computer system is a borrowed memory portion that is inaccessible by other computer systems while it is mapped to the remote memory of the computer system. 
     
     
         10 . The hardware controller of  claim 8  wherein the memory load instruction is received from a central processing unit (CPU) of the computer system over a cache-coherent interface, and wherein upon retrieving the data from the backup memory, the hardware controller provides the data to the CPU over the cache-coherent interface. 
     
     
         11 . The hardware controller of  claim 10  wherein the threshold time window is based on a property of the cache-coherent interface. 
     
     
         12 . The hardware controller of  claim 8  wherein the backup memory is implemented using a non-volatile memory that is configured to hold a copy of all data in the remote memory. 
     
     
         13 . The hardware controller of  claim 8  wherein the method further comprises:
 upon determining that the response network packet has been received from said another computer system within the threshold time window, extracting the data from the response network packet. 
 
     
     
         14 . The hardware controller of  claim 8  wherein the backup memory is organized using a log-structured data layout. 
     
     
         15 . A computer system comprising:
 a central processing unit (CPU);   a remote memory that is mapped to a portion of a physical system memory of another computer system;   a hardware controller communicatively coupled with the CPU; and   a backup memory communicatively coupled with the hardware controller,   wherein the hardware controller is configured to:
 receive, from the CPU, a memory load instruction directed to a physical memory address in the remote memory; 
 transmit a network packet including the memory load instruction to said another computer system; 
 wait for a response network packet from said another computer system that includes data requested via the memory load instruction; and 
 upon determining that the response network packet has not been received from said another computer system within a threshold time window, retrieve the data from the backup memory. 
   
     
     
         16 . The computer system of  claim 15  wherein the portion of the physical system memory of said another computer system is a borrowed memory portion that is inaccessible by other computer systems while it is mapped to the remote memory of the computer system. 
     
     
         17 . The computer system of  claim 15  wherein the memory load instruction is received from the CPU over a cache-coherent interface, and wherein upon retrieving the data from the backup memory, the hardware controller provides the data to the CPU over the cache-coherent interface. 
     
     
         18 . The computer system of  claim 17  wherein the threshold time window is based on a property of the cache-coherent interface. 
     
     
         19 . The computer system of  claim 15  wherein the backup memory is implemented using a non-volatile memory that is configured to hold a copy of all data in the remote memory. 
     
     
         20 . The computer system of  claim 15  wherein the hardware controller is further configured to:
 upon determining that the response network packet has been received from said another computer system within the threshold time window, extract the data from the response network packet. 
 
     
     
         21 . The computer system of  claim 15  wherein the backup memory is organized using a log-structured data layout.