System, method and medium editing moving pictures using biometric signals

Abstract

A system, method and medium editing a moving picture using biometric signals is provided. The system includes a biometric signal generation module to measure signals that reflect an emotional state of a user while capturing a moving picture, and to generate a first and a second biometric signal based on the measured signals, an event section extraction module to extract a first event section that reflects preferences of the user from a playback section of the moving picture based on the first biometric signal, extract a second event section that reflects preferences of the user from the playback section of the moving picture based on a second biometric signal, and extract a final event section based on the first and second event sections, and an indexing module to edit the moving picture by indexing the final event section in synchronization with the playback section of the moving picture.

1 . A system editing a moving picture using biometric signal, the system comprising:
 a biometric signal generation module to measure at least one signal that reflects an emotional state of a user who captures a moving picture, and to generate at least one biometric signal based on the measured signal;   an event section extraction module to extract at least one event section that reflects preferences of the user from a playback section of the moving picture based on the at least one biometric signal, and extract a final event section based on the at least one event section; and   an indexing module to index the final event section in synchronization with the playback section of the moving picture.   
     
     
         2 . The system of  claim 1 , wherein the at least one signal that reflects the emotional state of the user comprises at least one of a signal that reflect variations in the amount of blood flow resulting from variations in the heart rate of the user or a signal that reflects variations in the skin resistance and the degree of perspiration of the user. 
     
     
         3 . The system of  claim 1 , wherein the at least one biometric signal comprises a first biometric signal and a second biometric signal, and
 wherein the first biometric signal is generated by a photoplethysmograghy (PPG) sensor, and the second biometric signal is generated by a galvanic skin response (GSR) sensor, wherein the PPG sensor and the GSR sensor are attached to a portion of a digital device capturing the moving picture.   
     
     
         4 . The system of  claim 3 , wherein the biometric signal generation module comprises:
 a first biometric signal generation unit to generate the first biometric signal using the PPG sensor that is placed in contact with the skin of the user;   a second biometric signal generation unit to generate the second biometric signal using the GSR sensor that is placed in contact with the skin of the user;   a PPG noise section setting unit to set a PPG noise section using PPG data, for the first biometric signal; and   a GSR noise section setting unit to set a GSR noise section using GSR data, for the second biometric signal.   
     
     
         5 . The system of  claim 4 , wherein the PPG noise section setting unit sets an RR interval which is a time interval between peaks in the PPG data, removes noise from the RR interval, and sets a time interval between T_RR(n)+RR(n)/2 and T_RR(n+1)-RR(n+1)/2 (where RR(n) indicates n-th data of the RR interval, T_RR(n) indicates the time of detection of RR(n), RR(n+1) indicates (n+1)-th data of the RR interval, and T_RR(n+1) indicates the time of detection of RR(n+1)) as the PPG noise section if T_RR(n+1)−T_RR(n)>{RR(n)+RR(n+1)}/2° C. (where C is a constant between 1 and 3). 
     
     
         6 . The system of  claim 4 , wherein the GSR data comprises SIL data which indicates absolute skin resistance measured as a signal and SIR data which indicates the rate of change in the SIL data over time, and the GSR noise section setting module extracts an SIR count value indicating the number of peaks in the SIR data from the SIR data, sets a point where SIL(t+Δt 1 )−SIL(t)>Th 1  (where SIL(t) indicates SIL data measured at a time t, Δt 1  indicates a predetermined time interval, and Th 1  indicates a first threshold) as a beginning point of the GSR noise section, sets a point where 0<SIL(t)−SIL(t+Δt 2 )<Th 2  and SIL(t)<Th 3  (where Δt 2  indicates a predetermined time interval and Th 2  and Th 3  respectively indicate second and third thresholds) as an ending point of the GSR noise section, and then removes noise from the GSR noise section. 
     
     
         7 . The system of  claim 6 , wherein Δt 1  is within the range of 0.1-0.5, Δt 2  is within the range of 0.2-1.0, and Th 1  is within the range of 20-60 kΩ, Th 2  is within the range of 0-5 kΩ, and Th 3  is higher than 2MΩ when the SIL data ranges between 10 kΩ and 2MΩ. 
     
     
         8 . The system of  claim 3 , wherein the event section extraction module comprises:
 a first event section extraction unit to extract n first event sections respectively having first through n-th priorities from the PPG noise section;   a second event section extraction unit to extract m second event sections respectively having first through m-th priorities from the GSR noise section; and   a final event section extraction module to determine overlapping sections between the n first event sections and the m second event sections as first priority overlapping sections, and to extract a first priority overlapping section that is longer than a minimum required time set by the user as the final event section.   
     
     
         9 . The system of  claim 8 , wherein, if none of the first priority overlapping sections are longer than the minimum required time, the final event section extraction module determines overlapping sections between the n first event sections and the GSR noise section as second priority overlapping sections and extracts a second priority overlapping section that is longer than the minimum required time as the final event section. 
     
     
         10 . The system of  claim 9 , wherein, if none of the second priority overlapping sections are longer than the minimum required time, the final event section extraction module determines overlapping sections between the m second event sections and the PPG noise section as third priority overlapping sections, and extracts a third priority overlapping section that is longer than the minimum required time as the final event section. 
     
     
         11 . The system of  claim 10 , wherein, if none of the third priority overlapping sections are longer than the minimum required time, the final event section extraction module prioritizes remaining first and second event sections to be generated as final event sections, and extracts the remaining first or second event section with a highest priority as the final event section. 
     
     
         12 . The system of  claim 3 , wherein the at least one event section comprises a first event section and a second event section, and
 wherein the first event section comprises a time interval including the time of detection of a peak in the first biometric signal, the second event section comprises a time interval including the time of detection of a SIR count value higher than a SIR threshold that is determined based on the average and the standard deviation of SIR data measured during a predetermined time interval.   
     
     
         13 . A method of editing a moving picture using biometric signal, the method comprising:
 measuring at least one signal that reflects an emotional state of a user who captures a moving picture;   generating at least one biometric signal based on the measured signal;   extracting at least one event section that reflects preferences of the user from a playback section of the moving picture based on the at least one biometric signal;   extracting a final event section based on the at least one event section; and   indexing the final event section in synchronization with the playback section of the moving picture.   
     
     
         14 . The method of  claim 13 , wherein the at least one signal that reflects the emotional state of the user comprises at least one of a signal that reflect variations in the amount of blood flow resulting from variations in the heart rate of the user or a signal that reflects variations in the skin resistance and the degree of perspiration of the user. 
     
     
         15 . The method of  claim 13 , wherein the at least one biometric signal comprises a first biometric signal and a second biometric signal, and
 wherein the first biometric signal is generated by a photoplethysmograghy (PPG) sensor, and the second biometric signal is generated by a galvanic skin response (GSR) sensor, wherein the PPG sensor and the GSR sensor are attached to a predetermined portion of a digital device capturing the moving picture.   
     
     
         16 . The method of  claim 15 , wherein the measuring of the signals comprises:
 generating the first biometric signal using the PPG sensor that is placed in contact with the skin of the user, and generating the second biometric signal using the GSR sensor that is placed in contact with the skin of the user;   setting a PPG noise section using PPG data, wherein the PPG data is the first biometric signal; and   setting a GSR noise section using GSR data, wherein the GSR data is the second biometric signal.   
     
     
         17 . The method of  claim 16 , wherein the setting of the PPG noise section using PPG data comprises:
 setting an RR interval which is a time interval between peaks in the PPG data;   removing noise from the RR interval; and   setting a time interval between T_RR(n)+RR(n)/2 and T_RR(n+1)−RR(n+1)/2 (where RR(n) indicates n-th data of the RR interval, T_RR(n) indicates the time of detection of RR(n), RR(n+1) indicates (n+1)-th data of the RR interval, and T_RR(n+1) indicates the time of detection of RR(n+1)) as the PPG noise section if T_RR(n+1)−T_RR(n)>{RR(n)+RR(n+1)}/2° C. (where C is a constant between 1 and 3).   
     
     
         18 . The method of  claim 16 , wherein the GSR data comprises SIL data, which indicates absolute skin resistance measured as a signal and SIR data which indicates the rate of change in the SIL data over time, and the setting of the GSR noise section using GSR data comprises:
 extracting a SIR count value indicating the number of peaks in the SIR data from the SIR data;   setting a point where SIL(t+Δt 1 )−SIL(t)>Th 1  (where SIL(t) indicates SIL data measured at a time t, Δt 1  indicates a predetermined time interval, and Th 1  indicates a first threshold) as a beginning point of the GSR noise section, and setting a point where 0<SIL(t)−SIL(t+Δt 2 )<Th 2  and SIL(t)<Th 3  (where Δt 2  indicates a predetermined time interval and Th 2  and Th 3  respectively indicate second and third thresholds) as an ending point of the GSR noise section; and   removing noise from the GSR noise section.   
     
     
         19 . The method of  claim 18 , wherein Δt 1  is within the range of 0.1-0.5, Δt 2  is within the range of 0.2-1.0, and Th 1  is within the range of 20-60 kΩ, Th 2  is within the range of 0-5 kΩ, and Th 3  is higher than 2MΩ when the SIL data ranges between 10 kΩ and 2MΩ. 
     
     
         20 . The method of  claim 15 , wherein the extracting of the at least one event section comprises:
 extracting n first event sections respectively having first through n-th priorities from the PPG noise section, and extracting m second event sections respectively having first through m-th priorities from the GSR noise section; and   determining the overlapping sections between the n first event sections and the m second event sections as first priority overlapping sections, and extracting a first priority overlapping section that is longer than a minimum required time set by the user as the final event section.   
     
     
         21 . The method of  claim 20 , wherein the extracting of the at least one event section further comprises:
 if none of the first priority overlapping sections are longer than the minimum required time, determining overlapping sections between the n first event sections and the GSR noise section as second priority overlapping sections and extracting a second priority overlapping section that is longer than the minimum required time as the final event section;   if none of the second priority overlapping sections are longer than the minimum required time, determining overlapping sections between the m second event sections and the PPG noise section as third priority overlapping sections, and extracting a third priority overlapping section that is longer than the minimum required time as the final event section; and   if none of the third priority overlapping sections are longer than the minimum required time, prioritizing remaining first and second event sections to be generated as final event sections, and extracting the remaining first or second event section with a highest priority as the final event section.   
     
     
         22 . The method of  claim 15 , wherein the at least one event section comprises a first event section and a second event section, and
 wherein the first event section comprises a time interval including the time of detection of a peak in the first biometric signal, the second event section comprises a time interval including the time of detection of a SIR count value higher than a SIR threshold that is determined based on the average and the standard deviation of SIR data measured during a predetermined time interval.   
     
     
         23 . At least one medium comprising computer readable code to control at least one processing element to implement the method of  claim 13 .