Concurrent execution of critical sections by eliding ownership of locks

Abstract

Critical sections of multi-threaded programs, normally protected by locks providing access by only one thread, are speculatively executed concurrently by multiple threads with elision of the lock acquisition and release. Upon a completion of the speculative execution without actual conflict as may be identified using standard cache protocols, the speculative execution is committed, otherwise the speculative execution is squashed. Speculative execution with elision of the lock acquisition, allows a greater degree of parallel execution in multi-threaded programs with aggressive lock usage.

We claim: 
     
         1 . A method of coordinating access to common memory by multiple program threads comprising the steps of:
 in each given program thread,   (a) detecting the beginning of a critical section of the given program thread in which conflicts to access of the common memory could occur resulting from execution of other program threads;   (b) speculatively executing the critical section; and   (c) committing the speculative execution of the critical section if there has been no conflict and squashing the speculative execution of the critical section if there has been a conflict.   
     
     
         2 . The method of  claim 1 , wherein the conflict is:
 (a) another thread writing data read by the given program thread in the critical section, or   (b) another thread reading or writing data written by the given program thread.   
     
     
         3 . The method of  claim 2  wherein the conflict is detected by an invalidation of a cache block holding data of the critical section. 
     
     
         4 . The method of  claim 1  wherein the speculative execution is committed at the end of the critical section. 
     
     
         5 . The method of  claim 4  wherein the end of the critical section is detected by a pattern of instructions typically associated with a lock release. 
     
     
         6 . The method of  claim 5  wherein the pattern of instructions is a store instruction to a deduced lock variable address. 
     
     
         7 . The method of  claim 1  wherein he speculative execution is committed at a resource boundary limiting further speculation. 
     
     
         8 . The method of  claim 7  including the step of:
 (d) if at step (c) there was no conflict from the execution of another thread acquiring a lock variable allowing the given thread to have exclusive access to the critical section and continuing execution from the commitment point to the conclusion of the critical section. 
 
     
     
         9 . The method of  claim 1  wherein the speculative execution is committed upon the occurrence of a non cacheable operation limiting further speculation. 
     
     
         10 . The method of  claim 9  including the step of
 (d) if at step (c) there was no conflict from the execution of another thread acquiring a lock variable allowing the given thread to have exclusive access to the critical section and continuing execution from the commitment point to the conclusion of the critical section. 
 
     
     
         11 . The method of  claim 1  wherein step (a) includes reading of a lock variable and wherein step (b) is performed only when the lock variable is not held by another program thread. 
     
     
         12 . The method of  claim 1  wherein step (a) includes reading a prediction table holding historical data indicating past successes in speculatively executing the critical section and wherein step (b) is performed only when the prediction table indicates a likelihood of successful speculative execution of the critical section of above a predetermined threshold. 
     
     
         13 . The method of  claim 1  wherein step (a) deduces the beginning of a critical section by detecting patterns of instructions typically associated with a lock acquisitions. 
     
     
         14 . The method of  claim 13  wherein pattern includes an atomic read/modify/write sequence. 
     
     
         15 . The method of  claim 1  wherein the critical section is preceded by a lock acquisition section and including the step of eliding the lock acquisition before step (b). 
     
     
         16 . The method of  claim 1  wherein the critical section ends with a lock release section and including the step of eliding the lock release section after step (c) when at step (c) upon reaching the end of the critical section, no conflict from the execution of another thread occurred. 
     
     
         17 . The method of  claim 1  including the further step of:
 (d) after squashing the speculative execution of the critical section if there has been conflict, re-executing the critical section speculatively. 
 
     
     
         18 . The method of  claim 17  wherein the speculative re-execution of the critical section is repeated up to a predetermined number of times until there is not a conflict. 
     
     
         19 . The method of  claim 17  wherein (d) if after the predetermined number of tries there remains a conflict from the execution of another thread, acquiring a lock variable allowing the given thread to have exclusive access to the critical section and continuing execution of the critical section from its beginning. 
     
     
         20 . The method of  claim 1  wherein the speculation executes the critical section using a cache memory to record the speculative execution without visibility to other processing units. 
     
     
         21 . The method of  claim 1  wherein the speculation executes the critical section eliding write instructions that do not change a value of memory location being written to. 
     
     
         22 . A lock elision circuit for a computer architecture allowing the access of common memory by multiple program threads, the circuit comprising:
 means for controlling the execution of each given program thread to:   (a) detect the beginning of a critical section of the given program thread in which conflicts to access of the common memory could occur resulting from execution of other program threads;   (b) speculatively execute the critical section; and   (c) commit the speculative execution of the critical section if there has been no conflict and squashing the speculative execution of the critical section if there has been a conflict.   
     
     
         23 . The lock elision circuit of  claim 22  wherein the conflict is:
 (a) another thread writing data read by the given program thread in the critical section, or 
 (b) another thread reading or writing data written by the given, program thread. 
 
     
     
         24 . The lock elision circuit of  claim 23  wherein the computer architecture includes a cache and the conflict is detected by an invalidation of a cache block holding data of the critical section. 
     
     
         25 . The lock elision circuit of  claim 22  wherein the speculative execution is committed at the end of the critical section. 
     
     
         26 . The lock elision of  claim 25  wherein the end of the critical section is detected by a pattern of instructions typically associated with a lock release. 
     
     
         27 . The lock elision circuit of  claim 26  wherein the pattern of instructions is a store instruction to a deduced lock variable address. 
     
     
         28 . The lock elision circuit of  claim 22  wherein the speculative execution is committed at a resource boundary limiting further speculation. 
     
     
         29 . The lock elision circuit of  claim 28  wherein when there is no conflict from the execution of another thread acquiring a lock variable, the lock elision circuit allows the given thread to have exclusive access to the critical section and continues execution from the commitment point to the conclusion of the critical section. 
     
     
         30 . The lock elision circuit of  claim 22  wherein the lock elision circuit reads a lock variable and speculatively executes the critical section only when the lock variable is not held by another program thread. 
     
     
         31 . The lock elision circuit of  claim 22  wherein the speculative execution is committed upon the occurrence of a non cacheable operation limiting further speculation. 
     
     
         32 . The lock elision circuit of  claim 31  including the step of:
 (d) if at step (c) there was no conflict from the execution of another thread acquiring lock variable allowing the given thread to have exclusive access to the critical section and continuing execution from the commitment point to the conclusion of the critical section. 
 
     
     
         33 . The lock elision circuit of  claim 22  including a prediction table holding historical data indicating past successes in speculatively executing the critical section and wherein the lock elision circuit speculatively executes the critical section only when the prediction table indicates a likelihood of successful speculative execution of the critical section of above a predetermined threshold. 
     
     
         34 . The lock elision circuit of  claim 22  wherein the lock elision circuit determines the beginning of a critical section by detecting patterns of instructions typically associated with a lock acquisitions. 
     
     
         35 . The lock elision circuit of  claim 34  wherein the pattern includes an atomic read/modify/write sequence. 
     
     
         36 . The lock elision circuit of  claim 22  wherein the critical section is preceded by a lock acquisition section and wherein the lock elision circuit elides the lock acquisition before speculation. 
     
     
         37 . The lock elision circuit of  claim 22  wherein the critical section ends with a lock release section and wherein the lock elision circuit elides the lock release section after speculation when upon reaching the end of the critical section, no conflict from the execution of another thread occurred. 
     
     
         38 . The lock elision circuit of  claim 22  wherein after squashing the speculative execution of the critical section, if there has been a conflict, the lock elision circuit re-executes the critical section speculatively. 
     
     
         39 . The lock elision circuit of  claim 38  wherein the lock elision circuit repeats the speculative re-execution of the critical section up to a predetermined number of times until there is not a conflict. 
     
     
         40 . The lock elision circuit of  claim 39  wherein if after the predetermined number of tries there remains a conflict from the execution of another thread, the lock elision, circuit allows acquisition of a lock variable allowing the given thread to have exclusive access to the critical section and continuing execution of the critical section from its beginning. 
     
     
         41 . The lock elision circuit of  claim 22  wherein the computer architecture includes a cache memory and the lock elision circuit uses the cache memory to record the speculative execution without visibility to other processing units. 
     
     
         42 . The lock elision circuit of  claim 22  wherein the lock elision circuit elides write instructions within the critical section that do not change a value of memory location being written to.