Mesh based convolutional neural network techniques

Abstract

Convolutional operators for triangle meshes are determined to construct one or more neural networks. In at least one embodiment, convolutional operators, pooling operators, and unpooling operators are determined to construct the one or more neural networks, in which the same learned weights from the one or more neural networks can further be used for triangle meshes with different topologies.

What is claimed is: 
     
         1 . A system, comprising:
 one or more computers having one or more processors to train a neural network by:
 performing a convolution on data input, at a layer of the neural network, wherein the convolution is performed by applying convolutional operators on the data input, the convolutional operators being determined by:
 selecting a vertex from a plurality of vertices of the data input; 
 sampling the plurality of vertices, based on a length of the convolutional operators, to generate a plurality of sampling points for the selected vertex; 
 adding the plurality of vertices to a list such that each vertex from the plurality of vertices is paired with a sampled point from the plurality of sampling points; and 
 determining, based at least in part of the list, a set of vertices; and 
 
 applying the convolutional operators to the data input to generate a set of outputs of the convolution, the convolutional operators being defined at least in part based on the set of vertices. 
   
     
     
         2 . The system of  claim 1 , wherein the one or more processors are further to train the neural network by:
 performing a set of operations, using additional layers of the neural network, on the output from the layer, wherein at least one operation from the set of operations comprises a pooling operation, wherein operators for the pooling operation are determined by:
 performing a selection of a vertex from the plurality of vertices of the data input to combine with at least one other vertex from the plurality of vertices to generate one or more shared vertices, wherein the one or more shared vertices are used as the operators; and 
 applying the operators to the set of outputs of the pooling operation. 
   
     
     
         3 . The system of  claim 2 , wherein the one or more processors are further to train the neural network by using a lowest-quadratic error formula to perform the selection of the vertex from plurality of vertices. 
     
     
         4 . The system of  claim 2 , wherein the selected vertex and the at least one other vertex are independent of one another. 
     
     
         5 . The system of  claim 1 , wherein the one or more processors are further to train the neural network by:
 performing a set of operations, using additional layers of the neural network, on the output from the layer, wherein at least one operation from the set of operations comprises an unpooling operation, wherein operators for the unpooling operation are determined by:
 selecting a vertex from the plurality of vertices of the data input and copying a value of the vertex to at least one other vertex from the plurality of vertices to generate one or more shared vertices, wherein the one or more shared vertices are used as the operators; and 
 applying the operators to the set of outputs of the unpooling operation. 
   
     
     
         6 . The system of  claim 1 , wherein a sum of distances between each vertex and corresponding sampled point is minimized. 
     
     
         7 . The system of  claim 6 , wherein parameters from the trained neural network are applied to a second data input to determine convolutional operators, wherein the second data input is a triangle mesh different from the data input. 
     
     
         8 . A processor comprising:
 one or more arithmetic logic units (ALUs) to train one or more neural networks, at least in part, by:
 determining one or more convolutional operators to perform a convolution on received data input, wherein the determination of the one or more convolutional operators is performed by: 
 sampling a plurality of vertices of the data input to generate a plurality of sampling points for a vertex of the plurality of vertices; 
 generating an index to indicate that the vertex and each vertex of the plurality of vertices is paired with a sampled point from the plurality of sampling points; and 
 determining the convolutional operators using information from the index. 
   
     
     
         9 . The processor of  claim 8 , further comprising the one or more ALUs to train the one or more neural networks by:
 performing a pooling operation, wherein operators for the pooling operation are determined by:
 determining a vertex from the plurality of vertices of the data input to merge with at least one other vertex from the plurality of vertices to generate one or more shared vertices, wherein the vertex and the at least one other vertex is independent of one another; and 
 applying the one or more shared vertices as operators for the pooling operation. 
   
     
     
         10 . The processor of  claim 9 , wherein the operators for the pooling operation are determined for the one or more neural networks by using results from applying a lowest-quadratic error formula when merging the vertex and the at least one other vertex. 
     
     
         11 . The processor of  claim 8 , further comprising the one or more ALUs to train the one or more neural networks by:
 performing an unpooling operation, wherein operators for the unpooling operation are determined by:
 determining a vertex from the plurality of vertices of the data input and copying a value of the vertex to at least one other vertex from the plurality of vertices to generate one or more shared vertices, wherein the one or more shared vertices are used as the operators for the unpooling operation. 
   
     
     
         12 . The processor of  claim 8 , wherein the data input is a triangle mesh. 
     
     
         13 . The processor of  claim 8 , wherein parameters from the one or more neural networks are applied to a second data input, different from the data input, to determine convolutional operators for the second data input. 
     
     
         14 . The processor of  claim 8 , wherein sampling the plurality of vertices of the data input is based on a length of the convolutional operators. 
     
     
         15 . A method, comprising:
 training one or more neural networks by:
 determining convolutional operators to perform a convolution on data, wherein the convolutional operators are determined by:
 sampling a plurality of vertices of the data, based on a length of the convolutional operators, to generate a plurality of sampling points for a vertex of the plurality of vertices; and 
 determining a set of vertices to define the convolutional operators based at least in part on the plurality of vertices and the plurality of sampling points. 
 
   
     
     
         16 . The method of  claim 15 , further comprising:
 determining operators for a pooling operation to be performed by a layer of the one or more neural networks, wherein operators for the pooling operation are determined by:
 selecting a first vertex and a second vertex from the plurality of vertices of the data, wherein the first and second vertex are independent from one another; 
 merging the first vertex and second vertex to generate one or more shared vertices; and 
 applying the one or more shared vertices as operators for the pooling operation. 
   
     
     
         17 . The method of  claim 16 , wherein a quadratic error formula is used when merging the first vertex and second vertex to generate the one or more shared vertices. 
     
     
         18 . The method of  claim 15 , further comprising:
 determining operators for an unpooling operation, using output from the layer, to be performed by a second layer of the one or more neural networks, wherein operators for the unpooling operation are determined by:
 selecting a first vertex from the plurality of vertices of the data and copying a value from the first vertex to a second vertex to generate one or more shared vertices; and 
 applying the one or more shared vertices as operators for the unpooling operation. 
   
     
     
         19 . The method of  claim 15 , wherein the data is a manifold triangle mesh. 
     
     
         20 . The method of  claim 15 , wherein learned weights from the one or more neural networks are applied to a second data having a different topology than a topology of the data.