Method and system for scalable job processing

Abstract

The present invention relates to methods for processing jobs within a cluster architecture. One method comprises the pausing of a job when waiting upon external dependencies. Another method comprises the transmission of messages relating to the ongoing processing of jobs back to a client via a persistent messaging channel. Yet another method comprises determining capacity at a node before allocating a job for processing by the node or adding the job to a cluster queue. A system for processing jobs within a cluster architecture is also disclosed.

1 . A method for processing jobs in a cluster architecture, including:
 one node of a plurality of processing nodes within the architecture receiving a job;   processing the job until the job is waiting for an external dependency to complete;   when a job is waiting for an external dependency, pausing the job;   processing another job; and   when the external dependency is completed, continuing the job.   
     
     
         2 . A method as claimed in  claim 1 , wherein each job is deserialised from a job definition. 
     
     
         3 . A method as claimed in  claim 2 , wherein the job definition includes task parameters and definition type. 
     
     
         4 - 54 . (canceled)