Handheld haptic device for realistic power tool feedback in vr-based manufacturing training

Abstract

A handheld haptic device and method for providing haptic feedback during a virtual interaction with a virtual power tool is disclosed. The handheld haptic device has actuators configured to generate haptic vibrations. A haptic vibration database is provided having a plurality of haptic vibration profiles that are selected as a function of the or more state parameters including an operating speed of the virtual power tool, an orientation of the handheld haptic device, and a material of a virtual workpiece. The system leverages the haptic vibration database to generate authentic haptic vibrations with the handheld haptic device, depending on the state parameters.

What is claimed is: 
     
         1 . A method for providing haptic feedback for a virtual interaction using a handheld haptic device, the virtual interaction including operating a virtual power tool, the method comprising:
 storing, in a memory, a plurality of haptic vibration profiles, each respective haptic vibration profile being associated with different respective values of at least one state parameter;   determining current values of the at least one state parameter as a user holds the handheld haptic device;   selecting, from the plurality of haptic vibration profiles, a selected haptic vibration profile based on the current values of the at least one state parameter; and   operating, during the virtual interaction, at least one actuator of the handheld haptic device to provide the haptic feedback according to the selected haptic vibration profile.   
     
     
         2 . The method according to  claim 1 , wherein the virtual interaction includes controlling the virtual power tool to perform an operation on a virtual workpiece, the method further comprising:
 receiving a material parameter indicating a material of the virtual workpiece, the at least one state parameter including the material parameter; and   selecting the selected haptic vibration profile based on the material parameter.   
     
     
         3 . The method according to  claim 1 , wherein the handheld haptic device includes an input device configured to provide a control input parameter depending on an actuation of the input device by a user, the method further comprising:
 receiving a current value of the control input parameter from the input device, the at least one state parameter including the control input parameter; and   selecting the selected haptic vibration profile based on the current value of the control input parameter.   
     
     
         4 . The method according to  claim 1  further comprising:
 determining a current value of an orientation parameter indicating an orientation of the handheld haptic device, the at least one state parameter including the orientation parameter; and 
 selecting the selected haptic vibration profile based on the current value of the orientation parameter. 
 
     
     
         5 . The method according to  claim 4 , the determining the current value of the orientation parameter further comprising:
 measuring the orientation of the handheld haptic device with a gyroscopic sensor that is integrated into the handheld haptic device.   
     
     
         6 . The method according to  claim 4 , the determining the current value of the orientation parameter further comprising:
 capturing images of the handheld haptic device with a camera; and   determining the orientation of the handheld haptic device based on the images using a pose-estimation algorithm.   
     
     
         7 . The method according to  claim 1 , the operating the at least one actuator of the handheld haptic device further comprising:
 mapping the selected haptic vibration profile to haptic control parameters configured to operate the at least one actuator to reproduce the selected haptic vibration profile in the handheld haptic device.   
     
     
         8 . The method according to  claim 7 , the mapping further comprising:
 mapping the selected haptic vibration profile to the haptic control parameters using a machine learning model.   
     
     
         9 . The method according to  claim 7 , wherein each haptic vibration profile of the plurality of haptic vibration profiles includes an acceleration frequency and an acceleration amplitude. 
     
     
         10 . The method according to  claim 7 , wherein the at least one actuator of the handheld haptic device includes a plurality of actuators, the plurality of actuators being arranged at different locations in the handheld haptic device and including multiple different types of actuators. 
     
     
         11 . The method according to  claim 10 , the mapping further comprising:
 determining which actuators of the plurality of actuators are to be operated to reproduce the selected haptic vibration profile in the handheld haptic device.   
     
     
         12 . The method according to  claim 7 , wherein the haptic control parameters include at least one of voltages, currents, or frequencies used to operate the at least one actuator to reproduce the selected haptic vibration profile in the handheld haptic device. 
     
     
         13 . The method according to  claim 7 , the operating the at least one actuator of the handheld haptic device further comprising:
 transmitting, with a transceiver, the haptic control parameters to the handheld haptic device; and   operating the at least one actuator based on the haptic control parameters using a controller of the handheld haptic device.   
     
     
         14 . The method according to  claim 1 , wherein the at least one actuator of the handheld haptic device includes at least one of a linear resonant actuator, an eccentric rotating mass motor, or linear magnetic ram. 
     
     
         15 . The method according to  claim 1 , wherein the handheld haptic device includes (i) a handle configured to be grasped by a user and (ii) a support structure extending from the handle configured to detachably connect with at least one detachable weight, the method further comprising:
 attaching, prior to the virtual interaction, the at least one detachable weight to the support structure at a location on the support structure such that a center of gravity of the handheld haptic device relative to the handle of the handheld haptic device matches a center of gravity of a physical power tool relative to a handle of the physical power tool, the physical power tool corresponding to the virtual power tool.   
     
     
         16 . The method according to  claim 15 , wherein (i) the at least one detachable weight includes a pair of detachable weights having a same mass and (ii) the support structure has a substantially planar first surface configured to detachably connect with a first detachable weight of the pair of detachable weights and a substantially planar second surface, opposite the substantially planar first surface, configured to detachably connect with a second detachable weight of the pair of detachable weights. 
     
     
         17 . The method according to  claim 1 , wherein the plurality of haptic vibration profiles was generated by recording vibrations of a physical power tool corresponding to the virtual power tool under operating conditions corresponding to the different respective values of the at least one state parameter. 
     
     
         18 . The method according to  claim 1  further comprising:
 tracking a pose of the handheld haptic device in an environment over time; and 
 displaying, in an augmented reality or virtual reality graphical user interface on a display screen, a graphical representation of the virtual interaction including rendering the virtual power tool having the pose of the handheld haptic device. 
 
     
     
         19 . The method according to  claim 18 , wherein the handheld haptic device includes an input device configured to output a control input parameter depending on an actuation of the input device by a user, the method further comprising:
 receiving a current value of the control input parameter from the input device; and   displaying, concurrently with the operating the at least one actuator of the handheld haptic device, in the augmented reality or virtual reality graphical user interface, an animation of the virtual power tool performing an operation based on the current value of the control input parameter.   
     
     
         20 . A system for providing haptic feedback for a virtual interaction, the virtual interaction including operating a virtual power tool, the system comprising:
 an augmented reality or virtual reality device including (i) a processor, (ii) a memory that stores a plurality of haptic vibration profiles, each respective haptic vibration profile being associated with different respective values of at least one state parameter, and (iii) a display screen configured to display an augmented reality or virtual reality graphical user interface including a graphical representation of the virtual interaction; and   a handheld haptic device having at least one actuator arranged within a housing,   wherein the processor of the augmented reality or virtual reality device is configured to (i) determine current values of the at least one state parameter as a user holds the handheld haptic device, (ii) select, from the plurality of haptic vibration profiles, a selected haptic vibration profile based on the current values of the at least one state parameter, and (iii) operate, during the virtual interaction, the at least one actuator of the handheld haptic device to provide the haptic feedback according to the selected haptic vibration profile.