System and method for designing of dictionaries for sparse representation

Abstract

A signal processing system adapted for sparse representation of signals is provided, comprising: (i)one or more training signals; (ii) a dictionary containing signal-atoms; (iii) a representation of each training signal using a linear combination of said dictionary's signal-atoms; (iv) means for updating the representation of the training signal; (v) means for updating the dictionary one group of atoms at a time, wherein each atom update may include all representations referring to said updated atom; and (vi) means for iterating (iv) and (v) until a stopping rule is fulfilled. The system uses the K-SVD algorithm for designing dictionaries for sparse representation of signals.

1 . A computerized signal processing system adapted for sparse representation of signals, said system comprising:
 (i) one or more training signals;   (ii) a dictionary containing signal-atoms;   (iii) a representation of each training signal using a linear combination of said dictionary's signal-atoms;   (iv) a module for updating the representation of the training signal;   (v) a module for updating the dictionary a single atom or one group of atoms at a time, wherein each atom update includes an update of all representations referring to said updated atom; and   (vi) a module for iterating (iv) and (v) until a stopping rule is fulfilled.   
     
     
         2 . The computerized signal processing system according to  claim 1 , wherein said group of atoms contains one atom only. 
     
     
         3 . The computerized signal processing system according to  claim 1 , wherein all training signals belong to the same family and have one or more common traits. 
     
     
         4 . The computerized signal processing system according to  claim 1 , wherein said dictionary is an overcomplete dictionary. 
     
     
         5 . The computerized signal processing system according to  claim 1 , wherein the representation of all training signals is a coefficient matrix. 
     
     
         6 . The computerized signal processing system according to  claim 1 , wherein the generation of each coefficients matrix is achieved by a pursuit algorithm. 
     
     
         7 . The computerized signal processing system according to  claim 6 , wherein said pursuit algorithm includes Orthogonal Matching Pursuit, Matching Pursuit, Basis Pursuit, FOCUSS or any combination or variation thereof. 
     
     
         8 . The computerized signal processing system according to  claim 1 , wherein said dictionary is updated in a predefined order of the signal-atoms. 
     
     
         9 . The computerized signal processing system according to  claim 1 , wherein only selected signal-atoms of said dictionary are updated. 
     
     
         10 . The computerized signal processing system according to  claim 1 , wherein a signal-atom is modified when the difference between said signal-atom to another signal atom is below a predefined value. 
     
     
         11 . The computerized signal processing system according to  claim 1 , wherein a signal-atom is modified when it is not used in any representation. 
     
     
         12 . The computerized signal processing system according to  claim 1 , wherein a signal-atom is modified when its usage frequency in the representation of signal-atoms is below a predefined value. 
     
     
         13 . The computerized signal processing system according to  claim 1 , wherein updating said group of atoms and their coefficients best reduces the Mean Square Error. 
     
     
         14 . The computerized signal processing system according to  claim 1 , wherein said dictionary is designed with custom properties. 
     
     
         15 . The signal processing system according to  claim 14 , wherein said custom properties include:
 (i) shift invariant;   (ii) non-negative dictionary values so that each atom contains only non-negative entries;   (iii) zeros in predetermined places; and   (iv) any structure of a matrix.   
     
     
         16 . The computerized signal processing system according to  claim 1 , comprising multiscale dictionaries. 
     
     
         17 . The computerized signal processing system according to  claim 1 , for compression, regularization in inverse problems, feature extraction, denoising, separation of texture and cartoon content in images, signal analysis, signal synthesis, inpainting and restoration.