Neural network model for generation of compressed haptic actuator signal from audio input

Abstract

A method comprises inputting an audio signal into a machine learning circuit to compress the audio signal into a sequence of actuator signals. The machine learning circuit being trained by: receiving a training set of acoustic signals and pre-processing the training set of acoustic signals into pre-processed audio data. The pre-processed audio data including at least a spectrogram. The training further includes training the machine learning circuit using the pre-processed audio data. The neural network has a cost function based on a reconstruction error and a plurality of constraints. The machine learning circuit generates a sequence of haptic cues corresponding to the audio input. The sequence of haptic cues is transmitted to a plurality of cutaneous actuators to generate a sequence of haptic outputs.

What is claimed is: 
     
         1 . A method, comprising:
 determining characteristics of actuator signals;   generating the actuator signals according to the determined characteristics by signal processing; and   sending the generated actuators signals to cutaneous actuators;   wherein the generated actuators cause vibrations to be generated by the cutaneous actuators spaced apart from each other such that the generated vibrations constructively or destructively interfere on a target location.   
     
     
         2 . The method of  claim 1 , wherein the generated vibrations comprise:
 main vibrations generated at a main cutaneous actuator of the cutaneous actuators responsive to one of the actuator signals, and   counter-vibrations generated by at least one auxiliary cutaneous actuator of the cutaneous actuators responsive to another of the actuator signals, the at least one auxiliary cutaneous actuator adjacent to the main cutaneous actuator, and a phase of the counter-vibrations opposite to a phase of the main vibrations.   
     
     
         3 . The method of  claim 2 , wherein an amplitude of combined vibrations at the main cutaneous actuator as a result of interference is above a sensory threshold whereas an amplitude of the combined vibrations at one or more auxiliary cutaneous actuators is below the sensory threshold. 
     
     
         4 . The method of  claim 1 , wherein the generated vibrations comprise:
 first vibrations generated at a first cutaneous actuator of the cutaneous actuators responsive to one of the actuator signals, and   second vibrations generated at a second cutaneous actuator of the cutaneous actuators responsive to another of the actuator signals, the first vibrations and the second vibrations of opposite phases, combined vibrations resulting from interference between the first and second vibrations having peak amplitudes at the first cutaneous actuator or the second cutaneous actuator above a sensory threshold.   
     
     
         5 . The method of  claim 1 , wherein the cutaneous actuators are arranged in a lattice pattern. 
     
     
         6 . The method of  claim 1 , wherein the vibrations generated by each of the cutaneous actuators is below a sensory threshold but combined vibrations resulting from constructive interference of the vibrations exceeds the sensory threshold. 
     
     
         7 . The method of  claim 6 , wherein the actuator signals change over time so that one or more locations at which the combined vibrations have peak amplitude move along a path. 
     
     
         8 . The method of  claim 1 , wherein the vibrations generated comprise:
 first vibrations of a first frequency generated at a first cutaneous actuator of the cutaneous actuators responsive to one of the actuator signals, and   second vibrations of a second frequency different from the first frequency generated at a second cutaneous actuator of the cutaneous actuators responsive to another of the actuator signals, combined vibrations resulting from interference between the first and second vibrations having a beat pattern embedded with haptic information.   
     
     
         9 . The method of  claim 1 , wherein the generated vibrations comprise:
 first vibrations generated at a first cutaneous actuator of the cutaneous actuators positioned at a first location responsive to one of the actuator signals, and   second vibrations generated at a second cutaneous actuator of the cutaneous actuators positioned at a second location responsive to another of the actuator signals, interference between the first and second vibrations causing a physical sensation at a location that is distinct from the first location and the second location.   
     
     
         10 . The method of  claim 1 , wherein the generated vibrations comprise:
 first vibrations generated at a first cutaneous actuator of the cutaneous actuators positioned at a first location responsive to one of the actuator signals, and   second vibrations generated at a second cutaneous actuator of the cutaneous actuators positioned at a second location responsive to another of the actuator signals, interference between the first and second vibrations increasing a physical sensation at the first location.   
     
     
         11 . A haptic system comprising:
 a signal generator operably communicating with cutaneous actuators spaced apart from each other and configured to generate vibrations responsive to actuator signals, the signal generator configured to:
 determine characteristics of the actuator signals that cause the generated vibrations to constructively or destructively interfere on a target area, 
 generate the actuator signals according to the determined characteristics by signal processing, and 
 send the generated actuators signals to the cutaneous actuators. 
   
     
     
         12 . The haptic system of  claim 11 , wherein the cutaneous actuators comprise:
 a main cutaneous actuator configured to generate main vibrations responsive to one of the actuator signals, and   at least one auxiliary cutaneous actuator configured to generate counter-vibrations responsive to another of the actuator signals, the at least one auxiliary cutaneous actuator adjacent to the main cutaneous actuator, and a phase of the counter-vibrations opposite to a phase of the main vibrations.   
     
     
         13 . The haptic system of  claim 11 , wherein an amplitude of combined vibrations at the main cutaneous actuator as a result of interference is above a sensory threshold whereas an amplitude of the combined vibrations at one or more auxiliary cutaneous actuators is below the sensory threshold. 
     
     
         14 . The haptic system of  claim 11 , wherein the cutaneous actuators comprise:
 a first cutaneous actuator configured to generate first vibrations responsive to one of the actuator signals, and   a second cutaneous actuator configured to generate second vibrations responsive to another of the actuator signals, the first vibrations and the second vibrations of opposite phases, combined vibrations resulting from interference between the first and second vibrations having peak amplitudes at the first cutaneous actuator or the second cutaneous actuator above a sensory threshold.   
     
     
         15 . The haptic system of  claim 11 , wherein the cutaneous actuators are arranged in a lattice pattern. 
     
     
         16 . The haptic system of  claim 11 , wherein the vibrations generated by each of the cutaneous actuators is below a sensory threshold but combined vibrations resulting from constructive interference of the vibrations exceeds the sensory threshold. 
     
     
         17 . The haptic system of  claim 16 , wherein the actuator signals change over time so that one or more locations at which the combined vibrations have peak amplitude, the one or more locations moving along a path. 
     
     
         18 . The haptic system of  claim 11 , wherein the cutaneous actuators comprise:
 a first cutaneous actuator configured to generate first vibrations of a first frequency at a first cutaneous actuator of the cutaneous actuators responsive to receiving one of the actuator signals, and   a second cutaneous actuator configured to generate second vibrations of a second frequency different from the first frequency at a second cutaneous actuator of the cutaneous actuators responsive to receiving another of the actuator signals, combined vibrations resulting from interference between the first and second vibrations resulting a beat pattern embedded with haptic information.   
     
     
         19 . The haptic system of  claim 11 , wherein the cutaneous actuators comprise:
 a first cutaneous actuator configured to generate first vibrations at a first cutaneous actuator of the cutaneous actuators positioned at a first location responsive to one of the actuator signals, and   a first cutaneous actuator configured to generate second vibrations at a second cutaneous actuator of the cutaneous actuators positioned at a second location responsive to another of the actuator signals, interference between the first and second vibrations causing a physical sensation at a location that is distinct from the first location and the second location.   
     
     
         20 . The haptic system of  claim 11 , wherein the cutaneous actuators comprise:
 a first cutaneous actuator configured to generate first vibrations at a first cutaneous actuator of the cutaneous actuators positioned at a first location responsive to one of the actuator signals, and   a first cutaneous actuator configured to generate second vibrations at a second cutaneous actuator of the cutaneous actuators positioned at a second location responsive to another of the actuator signals, interference between the first and second vibrations increasing a physical sensation at the first location.   
     
     
         21 . A non-transitory computer-readable storage medium storing instructions, the instructions when executed by a processor cause the processor to:
 determine characteristics of actuator signals;   generate the actuator signals according to the determined characteristics by signal processing; and   send the generated actuators signals to cutaneous actuators;   wherein the generated actuators cause vibrations to be generated by the cutaneous actuators spaced apart from each other such that the generated vibrations constructively or destructively interfere on a target area.