Network Computer with External Memory

Abstract

A computer comprising a plurality of processor devices connected in a ring, wherein each of the processor devices is connected to each of two neighbouring ones of the processor devices by a respective physical inter-processor link. Each of a set of external memory device stores a local portion of the externally stored dataset. Each processor device executes instructions to: determine that a synchronisation point has been reached by the plurality of processor devices; responsive to the determination, access from its connected external memory device its local portion of the externally stored dataset stored; record a copy of its local portion of the externally stored dataset in its local memory; transmit its local portion of the externally stored dataset to at least one of its connected neighbouring processing devices; and receive an incoming portion of the externally stored dataset from at least one of its connected neighbouring processing devices.

1 . A computer for executing an operation on locally stored data and an externally stored dataset, the computer comprising:
 a plurality of processor devices, each of the processor devices having local memory and being connected to at least one external memory device via a physical processor—memory link,   the processor devices being connected in a ring wherein each of the processor devices is connected to each of two neighbouring ones of the processor devices by a respective physical inter-processor link,   wherein the local memory of each processor device holds locally stored data,   wherein each external memory device stores a local portion of the externally stored dataset, the externally stored dataset being collectively stored in the external memory devices connected to the plurality of processor devices, and wherein   each processor device is configured to execute computer readable instructions of a computer program which operates the computer according to the following method:   determining that a synchronisation point has been reached by the plurality of processor devices of the computer,   responsive to the determination, accessing from its connected at least one external memory device by the physical processor-memory link its local portion of the externally stored dataset stored in its at least one external memory device   recording a copy of its local portion of the externally stored dataset in its local memory,   transmitting its local portion of the externally stored dataset to at least one of its connected neighbouring processing device, and   receiving an incoming portion of the externally stored dataset from at least one of its connected neighbouring processing devices, the incoming portion being a further local portion held in the external memory device connected to the at least one other connected neighbouring device whereby, for execution of the operation, each of the plurality of processor devices has recorded in its local memory a copy of all portions of the externally stored dataset for operating on their locally stored data.   
     
     
         2 . The computer of  claim 1  wherein a bandwidth of each inter-processor link is greater than a bandwidth of each physical processor-memory link. 
     
     
         3 . The computer of  claim 2  wherein the plurality of processor devices comprises four processor devices which are connected in the ring, wherein the bandwidth of each inter-processor link is three times as large as the bandwidth of each physical processor-memory link. 
     
     
         4 . The computer of  claim 1  wherein the locally stored data comprises an onboard vector. 
     
     
         5 . The computer of  claim 4  wherein the externally stored dataset comprises an external vector and wherein the operation is a vector multiplication. 
     
     
         6 . The computer of  claim 5  wherein the vector multiplication comprises a dot product. 
     
     
         7 . The computer of  claim 1  wherein the step of transmitting its local portion of the externally stored dataset to at least one of its connected neighbouring processing devices comprises transmitting its local portion of the externally stored dataset to both of its connected neighbouring processing devices. 
     
     
         8 . The computer of  claim 1  wherein the inter-processor links are bi-directional links configured to support data transmission in two directions at the same time. 
     
     
         9 . The computer of  claim 1  wherein each processor device comprises a plurality of interconnected processing tiles, each tile storing part of the locally stored data. 
     
     
         10 . The computer of  claim 1  wherein each processor device is configured to record the incoming portion of the externally stored dataset in its local memory and to transmit the incoming portion of the externally stored dataset to the at least one of its connected neighbouring processing devices. 
     
     
         11 . A processor device comprising:
 at least one execution unit configured to execute computer readable instructions;   a local memory;   a first interface configured for connection to a first physical inter-processor link for connecting the processor device to a first neighbouring processing device of a computer;   a second interface configured for connection to a second physical inter-processor link for connecting the processor device to a second neighbouring processing device of the computer;   a third interface configured to connect the processor device to a memory device via a physical processor-memory link, the processor device configured to execute computer readable instructions to implement the following method:
 determining that a synchronisation point has been reached by a plurality of processor devices in the computer in which the processor device is connected; 
 responsive to the determination, accessing via the physical processor-memory link a local portion of an externally stored dataset stored in at least one external memory device connected via the physical processor-memory link, 
 recording a copy of the local portion of the externally stored dataset in the local memory, 
 transmitting the local portion of the externally stored dataset from the first interface the local portion of the externally stored dataset and receiving an incoming portion of the externally stored dataset via the second interface from a neighbouring processing device of the computer and recording the incoming portion of the externally stored dataset in its local memory, 
 transmitting the incoming portion of the externally stored dataset via the first interface; and 
 performing an operation using all portions of the externally stored dataset and the locally stored data. 
   
     
     
         12 . A computer for executing an operation to apply an external vector to an onboard vector, the computer comprising:
 four processor devices, each processor device having local memory and being connected to two external memory devices via respective physical processor-memory links, each of the processor devices being further connected to each of two neighbouring processing devices of the computer via respective physical inter-processor links, whereby the four processor devices are connected in a ring,   wherein a bandwidth of each inter-processor link is three times a bandwidth of the physical processor memory link,   each local memory storing a portion of the onboard vector, the onboard vector being collectively stored in the local memories of the four processor devices of the computer,   each external memory device storing a local portion of the external vector, the external vector being collectively stored in the eight external memory devices, each processor device configured to execute a computer program which executes computer readable instructions of a computer program which operates the computer according to the following method:
 determining that a synchronisation point has been reached by the four processor devices, 
 responsive to the determination accessing from its two memory devices the local portion of the external vector stored in the two memory devices, 
 recording the portions of the external vector in its local memory, 
 transmitting the local portions of the external vector to each of the connected neighbouring processor devices and receiving from each of the neighbouring processor devices an incoming portion of the external vector, the incoming portion being a further local portion held in at least one of the two memory devices connected to each of the neighbouring processor devices. 
   
     
     
         13 . The computer of  claim 12  wherein each processor device is configured to record in its local memory the incoming portion of the external vector received from one of its connected neighbouring processor devices and to transmit that incoming portion to the other of its connected neighbouring processor devices. 
     
     
         14 . A method of operating a computer to execute an operation on locally stored data and an externally stored dataset, the method comprising:
 causing a first processor device to determine that a synchronisation point has been reached by a plurality of processor devices of a computer, the plurality of processor devices including the first processor device,   responsive to the determination, accessing from at least one external memory device connected to the first processor device a local portion of an externally stored dataset, the local portion of the externally stored dataset being stored in the at least on external memory device, wherein the step of accessing is carried out via a physical processor memory-link connecting the first processor device to the at least one external memory device,   recording a copy of the local portion of the externally stored dataset in a local memory of the first processor device,   transmitting the local portion of the externally stored dataset to at least a second processor device which is connected to the first processor device via a physical inter-processor link,   receiving an incoming portion of the externally stored dataset from a third processing device, the third processor device connected to the second processor device via a second physical inter-processor memory link, and   executing the operation using all received portions of the externally stored dataset and locally stored data.   
     
     
         15 . The method of  claim 14  wherein the first processor device is caused to implement the method by executing computer readable instructions of a computer program distributed over the plurality of processor devices of the computer. 
     
     
         16 . The method of  claim 14  comprising recording in the local memory of the first processor device the incoming portion of the externally stored dataset and transmitting that incoming portion of the externally stored dataset to the second processor device. 
     
     
         17 . The method of  claim 14  wherein the steps of accessing, recording, transmitting and receiving are carried out in an exchange phase of a bulk synchronisation protocol. 
     
     
         18 . The method of  claim 17  wherein the operation is carried out in a compute phase of the bulk synchronisation protocol. 
     
     
         19 . The method of  claim 14  wherein the step of accessing is carried out from each of the memory devices connected to the plurality of processor devices at the same time and responsive to the determination that the synchronisation point has been reached. 
     
     
         20 . Non-transitory computer readable media on which is stored computer readable instructions which cause a processor device to implement the following method:
 determining that a synchronisation point has been reached by a plurality of processor devices in the computer in which the processor device is connected;   responsive to the determination, accessing via a physical processor-memory link a local portion of an externally stored dataset stored in at least one external memory device connected to the processor device via the physical processor-memory link,   recording a copy of the local portion of the externally stored dataset in a local memory,   transmitting the local portion of the externally stored dataset from a first interface to a first connected processor device , receiving an incoming portion of the externally stored dataset from a second connected processor device via a second interface and recording the incoming portion of the externally stored dataset in the local memory,   transmitting the incoming portion of the externally stored dataset via the first interface; and   performing an operation using all portions of the externally stored dataset and the locally stored data.