Preventing stack buffer overflow attacks

Abstract

Improved buffer overflow protection for a computer function call stack is provided by placing a predetermined ShadowKEY value on a function's call stack frame and copying the ShadowKEY, a caller EBP, and a return pointer are pushed onto a duplicate stack. The prologue of the function may be modified for this purpose. The function epilogue is modified to compare the current values of the ShadowKEY, caller EBP, and the return pointer on the function stack to the copies stored on the duplicate stack. If they are not identical, an overflow is detected. The preserved copies of these values may be copied back to the function stack frame thereby enabling execution of the process to continue. A function prologue and epilogue may be modified during compilation of the program.

What is claimed is: 
     
         1 . A method of protecting a function stack frame on a computer call stack, the function stack frame corresponding to a function, the method comprising:
 modifying a prologue of the function wherein, when executed, the prologue performs the steps of:
 creating in the function stack frame a starting ShadowKEY value, a starting extended base pointer (EBP) value, and a starting return pointer value; and 
 creating a duplicate stack frame on a duplicate stack different from the computer call stack, the duplicate stack frame comprising the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; and 
   modifying an epilogue of the function wherein, when executed, the epilogue performs the steps of:
 fetching from the function stack frame a finishing ShadowKEY value, a finishing EBP value, and a finishing return pointer value; 
 fetching from the duplicate stack frame the starting ShadowKEY value, the starting EBP value, and the starting return pointer value; 
 comparing the starting ShadowKEY value, the starting EBP value, and the starting return pointer value to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively; and 
 if any one of the starting ShadowKEY value, the starting EBP value, and the starting return pointer value is unequal to the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value, respectively, replacing the finishing ShadowKEY value, the finishing EBP value, and the finishing return pointer value in the function stack frame with the starting ShadowKEY value, the starting EBP value, and the starting return pointer value, respectively. 
   
     
     
         2 . The method according to  claim 1 , wherein the function prologue is modified during compilation of the function. 
     
     
         3 . The method according to  claim 1 , wherein the function epilogue is modified during compilation of the function. 
     
     
         4 . The method according to  claim 1 , wherein the starting ShadowKEY value is placed on the function stack frame adjacent a local variables buffer and on a side of the local variables buffer in a direction of growth in memory of strings on the computer call stack. 
     
     
         5 . The method according to  claim 1 , wherein the duplicate stack frame is created on a side of the function stack frame opposite a direction of growth in memory of strings on the computer call stack. 
     
     
         6 . The method according to  claim 1 , wherein the starting ShadowKEY value, the starting EBP value, and the starting return pointer value are respective DWORD values. 
     
     
         7 . The method according to  claim 1  wherein the starting ShadowKEY value is specified by a caller of the function. 
     
     
         8 . The method according to  claim 1 , wherein the comparing comprises:
 comparing the starting ShadowKEY value to the finishing ShadowKEY value; and   only if the starting ShadowKEY value is equal to the finishing ShadowKEY value:
 comparing the starting EBP value to the finishing EBP value; and 
 only if the EBP value is equal to the finishing EBP value: 
   comparing the starting return pointer value to the finishing return pointer value.   
     
     
         9 . The method according to  claim 1 , wherein the starting EBP value is placed on the function stack frame on a side of the starting ShadowKEY value in a direction in memory of growth of strings on the computer call stack. 
     
     
         10 . The method according to  claim 1 , wherein the starting return pointer value is placed on the function stack frame on a side of the starting ShadowKEY value in a direction in memory of growth of strings on the computer call stack. 
     
     
         11 . A method of protecting a computer call stack from a buffer overflow vulnerability in a computer application, the method comprising performing the method according to  claim 1  for each function in the computer application. 
     
     
         12 . A computer program product comprising a computer-readable memory storing instructions executable by a computer to perform the method according to  claim 1 . 
     
     
         13 . The computer program product according to  claim 12  wherein the instructions are included in a compiler, and wherein the method is performed when the compiler is executed to compile a program containing the function. 
     
     
         14 . The computer program product according to  claim 13 , wherein the method is further performed for each one of a plurality of further functions contained in the program when the compiler is executed. 
     
     
         15 . The computer program product according to  claim 12 , wherein the starting ShadowKEY value is placed on the function stack frame adjacent a local variables buffer and on a side of the local variables buffer in a direction of growth in memory of strings on the computer call stack. 
     
     
         16 . The computer program product according to  claim 12 , wherein the duplicate stack frame is created on a side of the function stack frame opposite a direction of growth in memory of strings on the computer call stack. 
     
     
         17 . The computer program product according to  claim 12 , wherein the comparing comprises:
 comparing the starting ShadowKEY value to the finishing ShadowKEY value; and   only if the starting ShadowKEY value is equal to the finishing ShadowKEY value:
 comparing the starting EBP value to the finishing EBP value; and 
 only if the EBP value is equal to the finishing EBP value:
 comparing the starting return pointer value to the finishing return pointer value. 
 
   
     
     
         18 . The computer program product according to  claim 12 , wherein the starting EBP value is placed on the function stack frame on a side of the starting ShadowKEY value in a direction in memory of growth of strings on the computer call stack. 
     
     
         19 . The computer program product according to  claim 12 , wherein the starting return pointer value is placed on the function stack frame on a side of the starting ShadowKEY value in a direction in memory of growth of strings on the computer call stack.