Accelerating software builds

Abstract

A set of source files is stored in a shared storage repository for nodes of a distributed computing environment for software compilation. An object file is created based on at least a portion of the set of source files. A directed acyclic graph (DAG) is generated corresponding to a group of software build tasks and the relationship between the software build tasks based on the set of source files. A replication factor for the object file is determined based on the number of relationships of the object file identified from the DAG. The object file is stored in a local memory cache of at least one of the number of the nodes, wherein the number of the nodes is based on the replication factor for the object file.

What is claimed is: 
     
         1 . A computer system comprising:
 one or more computer processors;   one or more computer readable storage devices; and   program instructions stored on the computer readable storage devices for execution by at least one of the one or more computer processors, the program instructions comprising:   program instructions to store a set of source files in a shared storage repository for nodes of a distributed computing environment for software compilation, wherein each of the nodes includes a program configured to perform all of software build tasks selected from the group consisting of: preprocessing by a preprocessor component of the program, compiling by a compiler component of the program, assembling by an assembler component of the program, and linking by a linker component of the program;   program instructions to create an object file based on at least a portion of the set of source files;   program instructions to generate via a scheduler component, a directed acyclic graph (DAG) corresponding to a group of software build tasks and a relationship between the software build tasks based on the set of source files;   program instructions to receive a file I/O request for the object file;   program instructions to determine a replication factor for the object file based on a number of relationships of the object file identified from the DAG, wherein the replication factor indicates a number of copies of the object file to replicate and store to a local memory cache of each node of the number of nodes;   program instructions to, responsive to determining that the object file is not stored in the local memory cache, transmit a status message to a metadata server indicating that the object file is unavailable in the local memory cache;   program instructions to determine whether the object file is stored in another local memory cache using the information stored in the metadata server;   program instructions tom responsive to determining that the object file is stored in the other local memory cache, transmit the object file from the other local memory cache to the local memory cache;   program instructions to, responsive to determining that the object file is not stored in the other local memory cache, invoke the scheduler to assign a software build task for assembling the object file and subsequently transmitting the object file to the local memory cache to complete the file I/O request; and   program instructions to store via the metadata server, the object file in the local memory cache of at least one of the number of the nodes, wherein the number of the nodes is based on the replication factor for the object file, wherein each of the nodes includes an executer component configured to manage the respective local memory cache and manage the respective program.