Discrete variational auto-encoder systems and methods for machine learning using adiabatic quantum computers

Abstract

A computational system can include digital circuitry and analog circuitry, for instance a digital processor and a quantum processor. The quantum processor can operate as a sample generator providing samples. Samples can be employed by the digital processing in implementing various machine learning techniques. For example, the computational system can perform unsupervised learning over an input space, for example via a discrete variational auto-encoder, and attempting to maximize the log-likelihood of an observed dataset. Maximizing the log-likelihood of the observed dataset can include generating a hierarchical approximating posterior.

1 - 38 . (canceled) 
     
     
         39 . A method of unsupervised learning by a computational system, the method executed by circuitry including at least one processor and comprising:
 determining by the circuitry a first approximating posterior distribution over at least one group of a set of discrete random variables;   sampling by the circuitry from at least one group of a set of supplementary continuous random variables using the first approximating posterior distribution over the at least one group of the set of discrete random variables to generate one or more samples, wherein a transforming distribution comprises a conditional distribution over the set of supplementary continuous random variables, conditioned on the at least one group of a set of discrete random variables;   determining by the circuitry a second approximating posterior distribution and a first prior distribution, the first prior distribution over at least one layer of a set of continuous variables;   sampling by the circuitry from the second approximating posterior distribution;   determining by the circuitry an auto-encoding loss on an input space comprising discrete or continuous variables, the auto-encoding loss conditioned on the one or more samples;   determining by the circuitry a first KL-divergence, or at least an approximation thereof, between the second approximating posterior distribution and the first prior distribution;   determining by the circuitry a second KL-divergence, or at least an approximation thereof, between the first approximating posterior distribution and a second prior distribution, the second prior distribution over the set of discrete random variables; and   backpropagating by the circuitry a sum of the first and the second KL-divergence and the auto-encoding loss on the input space conditioned on the one or more samples.   
     
     
         40 . The method of  claim 39  wherein the auto-encoding loss is a log-likelihood. 
     
     
         41 . A method of unsupervised learning by a computational system, the method executed by circuitry including at least one processor and comprising:
 determining by the circuitry a first approximating posterior distribution over a first group of discrete random variables conditioned on an input space comprising discrete or continuous variables;   sampling by the circuitry from a first group of supplementary continuous random variables based on the first approximating posterior distribution;   determining by the circuitry a second approximating posterior distribution over a second group of discrete random variables conditioned on the input space and samples from the first group of supplementary continuous random variables;   sampling by the circuitry from a second group of supplementary continuous random variables based on the second approximating posterior distribution;   determining by the circuitry a third approximating posterior distribution and a first prior distribution over a first layer of additional continuous random variables, the third approximating posterior distribution conditioned on the input space, samples from at least one of the first and the second group of supplementary continuous random variables, and the first prior distribution conditioned on samples from at least one of the first and the second group of supplementary continuous random variables;   sampling by the circuitry from the first layer of additional continuous random variables based on the third approximating posterior distribution;   determining by the circuitry a fourth approximating posterior distribution and a second prior distribution over a second layer of additional continuous random variables, the fourth approximating posterior distribution conditioned on the input space, samples from at least one of the first and the second group of supplementary continuous random variables, samples from the first layer of additional continuous random variables, and the second prior distribution conditioned on at least one of samples from at least one of the first and the second group of supplementary continuous random variables, and samples from the first layer of additional continuous random variables;   determining by the circuitry a first gradient of a KL-divergence, or at least a stochastic approximation thereof, between the third approximating posterior distribution and the first prior distribution with respect to the third approximating posterior distribution and the first prior distribution;   determining by the circuitry a second gradient of a KL-divergence, or at least a stochastic approximation thereof, between the fourth approximating posterior distribution and the second prior distribution with respect to the fourth approximating posterior distribution and the second prior distribution;   determining by the circuitry a third gradient of a KL-divergence, or at least a stochastic approximation thereof, between an approximating posterior distribution over a third group of discrete random variables and a third prior distribution with respect to the approximating posterior distribution over the third group of discrete random variables and the third prior distribution, wherein the approximating posterior distribution over the third group of discrete random variables is a combination of the first approximating posterior distribution over the first group of discrete random variables, and the second approximating posterior distribution over the second group of discrete random variables; and   backpropagating by the circuitry the first, the second and the third gradients of the KL-divergence to the input space.   
     
     
         42 . The method of  claim 41  wherein determining by the circuitry a third gradient of a KL-divergence, or at least a stochastic approximation thereof, between an approximating posterior distribution over the third group of discrete random variables and a third prior distribution with respect to the approximating posterior distribution over the third group of discrete random variables and the third prior distribution comprises determining by the circuitry a third gradient of a KL-divergence, or at least a stochastic approximation thereof, between an approximating posterior distribution over the third group of discrete random variables and a third prior distribution with respect to the approximating posterior distribution over the third group of discrete random variables and the third prior distribution, the third prior distribution is comprising a restricted Boltzmann machine. 
     
     
         43 . The method of  claim 39  wherein determining by the circuitry a first KL-divergence comprises computing by the circuitry a loss function analytically. 
     
     
         44 . The method of  claim 39  wherein determining by the circuitry a first KL-divergence comprises estimating by the circuitry a loss function stochastically. 
     
     
         45 . The method of  claim 39  wherein determining by the circuitry a second KL-divergence comprises computing by the circuitry a loss function analytically. 
     
     
         46 . The method of  claim 39  wherein determining by the circuitry a second KL-divergence comprises estimating by the circuitry a loss function stochastically. 
     
     
         47 . The method of  claim 39  wherein determining by the circuitry a second approximating posterior distribution and a first prior distribution, the first prior distribution over at least one layer of a set of continuous variables comprises determining by the circuitry a second approximating posterior distribution and a first prior distribution, the first prior distribution comprising a restricted Boltzmann machine. 
     
     
         48 . The method of  claim 39  wherein determining by the circuitry a second KL-divergence, or at least an approximation thereof, between the first approximating posterior distribution and a second prior distribution, the second prior distribution over the second group of discrete random variables comprises determining by the circuitry a second KL-divergence, or at least an approximation thereof, between the first approximating posterior distribution and a second prior distribution, the second prior comprising a restricted Boltzmann machine. 
     
     
         49 . The method of  claim 39  wherein sampling by the circuitry from the second approximating posterior distribution includes at least one of generating samples by the circuitry or causing samples to be generated by a digital processor. 
     
     
         50 . The method of  claim 39  wherein sampling by the circuitry from the second approximating posterior distribution includes at least one of generating samples by the circuitry or causing samples to be generated by a quantum processor. 
     
     
         51 . The method of  claim 41  wherein sampling by the circuitry from a first group of supplementary continuous variables based on the first approximating posterior distribution includes at least one of generating samples by the circuitry or causing samples to be generated by one of a digital processor and a quantum processor. 
     
     
         52 . The method of  claim 41  wherein sampling by the circuitry from a second group of supplementary continuous variables based on the second approximating posterior distribution includes at least one of generating samples by the circuitry or causing samples to be generated by one of a digital processor and a quantum processor. 
     
     
         53 . The method of  claim 41  wherein sampling by the circuitry from the first layer of additional continuous random variables based on third first approximating posterior distribution includes at least one of generating samples by the circuitry or causing samples to be generated by one of a digital processor and a quantum processor. 
     
     
         54 . The method of  claim 41  wherein determining by the circuitry a third approximating posterior distribution and a first prior distribution over a first layer of additional continuous random variables comprises determining by the circuitry a third approximating posterior distribution and a first prior distribution over a first layer of additional continuous random variables, the first prior distribution comprising a restricted Boltzmann machine. 
     
     
         55 . The method of  claim 41  wherein determining by the circuitry a fourth approximating posterior distribution and a second prior distribution over a second layer of additional continuous random variables comprises determining by the circuitry a fourth approximating posterior distribution and a second prior distribution over a second layer of additional continuous random variables, the second prior comprising a restricted Boltzmann machine.