Malicious behavior detection method for detecting malicious behavior by analyzing program code, recording medium and apparatus for performing the same

Abstract

A method of detecting malicious behavior in a malicious behavior detection apparatus that detects malicious behavior by analyzing the code of a program, comprises generating a first control flow graph (CFG) by performing dynamic analysis on the program, generating a second CFG by extracting a code block that is likely to be executed by a conditional branch instruction included in the program, and comparing the first CFG and the second CFG to detect hidden code included in the program, and detecting a malicious behavior of the program by determining whether the detected hidden code is a malicious code. According to the constitutions, the time it takes to discover and respond to new types of malicious code can be shorten by extracting hidden code related to the malicious behavior and detecting malicious behavior.

The invention claimed is: 
     
         1 . A method for detecting a malicious behavior in a malicious behavior detection apparatus that detects a malicious behavior by analyzing a code of a program comprising:
 generating a first control flow graph (CFG) by performing dynamic analysis on the program;   generating a second CFG by extracting a code block that is likely to be executed by a conditional branch instruction included in the program; and   comparing the first CFG and the second CFG to detect hidden code included in the program, and detecting a malicious behavior of the program by determining whether the detected hidden code is a malicious code,   wherein the hidden code is code that does not exist in the first CFG, which is generated through code that is executed when the program's execution results are not affected by not manipulating the program's execution flow, but exists in the second CFG,   wherein the detecting the malicious behavior comprises,   analyzing a pattern of the hidden code to find a branching node among the nodes included in the hidden code and splitting the hidden code into at least one or more based on the branching node to generate a split CFG;   generating a signature for each split CFG; and   determining that the hidden code is a malicious code in response to determining the generated signature exists in a database storing signatures of malicious code prepared in advance.   
     
     
         2 . The method of  claim 1 , wherein the generating the second CFG comprises:
 generating the second CFG while executing the program using a forced execution engine.   
     
     
         3 . The method of  claim 2 , wherein the generating the second CFG comprises:
 setting an entry point of the program as a starting point of the forced execution engine; and   detecting whether the conditional branch instruction exists in a confirmed node while executing the program using the forced execution engine.   
     
     
         4 . The method of  claim 3 , wherein the detecting whether the conditional branch instruction exists comprises:
 setting, in response to determining that the conditional branch instruction exists in the confirmed node, a target address of the conditional branch instruction to a root node, repeatedly calling the root node so as to operate recursively to extract all nodes that are likely to be executed as the code block, and generating the second CFG.   
     
     
         5 . The method of  claim 4 , wherein the detecting whether the conditional branch instruction exists comprises:
 preventing duplicate inspection by checking whether each confirmed node has a history of inspection through a preset function.   
     
     
         6 . A non-transitory computer-readable recording medium having stored thereon a computer program comprising instructions that, when executed by a processor, cause the processor to:
 generate a first control flow graph (CFG) by performing dynamic analysis on the program;   generate a second CFG by extracting a code block that is likely to be executed by a conditional branch instruction included in the program; and   compare the first CFG and the second CFG to detect hidden code included in the program, and detect a malicious behavior of the program by determining whether the detected hidden code is a malicious code,   wherein the hidden code is code that does not exist in the first CFG, which is generated through code that is executed when the program's execution results are not affected by not manipulating the program's execution flow, but exists in the second CFG,   wherein the detection of the malicious behavior comprises,   analyzing a pattern of the hidden code to find a branching node among the nodes included in the hidden code and splitting the hidden code into at least one or more based on the branching node to generate a split CFG;   generating a signature for each split CFG; and   determining that the hidden code is a malicious code if the generated signature exists in a database storing signatures of malicious code prepared in advance.   
     
     
         7 . A malicious behavior detection apparatus for detecting a malicious behavior by analyzing a code of a program comprising:
 one or more processors; and   a memory for storing one or more instructions,   wherein the one or more processors execute the stored one or more instructions to perform operations comprising,   generating a first control flow graph (CFG) by performing dynamic analysis on the program;   generating a second CFG by extracting a code block that is likely to be executed by a conditional branch instruction included in the program; and   comparing the first CFG and the second CFG to detect hidden code included in the program, and detecting a malicious behavior of the program by determining whether the detected hidden code is a malicious code,   wherein the hidden code is code that does not exist in the first CFG, which is generated through code that is executed when the program's execution results are not affected by not manipulating the program's execution flow, but exists in the second CFG,   wherein the detecting the malicious behavior comprises,   analyzing a pattern of the hidden code to find a branching node among the nodes included in the hidden code and splitting the hidden code into at least one or more based on the branching node to generate a split CFG;   generating a signature for each split CFG; and   determining that the hidden code is a malicious code in response to determining that the generated signature exists in a database storing signatures of malicious code prepared in advance.   
     
     
         8 . The apparatus of  claim 7 , wherein the generating the second CFG comprises:
 generating the second CFG while executing the program using a forced execution engine.   
     
     
         9 . The apparatus of  claim 8 , wherein the generating the second CFG comprises:
 setting an entry point of the program as a starting point of the forced execution engine and detecting whether the conditional branch instruction exists in a confirmed node while executing the program using the forced execution engine.   
     
     
         10 . The apparatus of  claim 9 , wherein the generating the second CGF comprises:
 setting, in response to determining that the conditional branch instruction exists in the confirmed node, a target address of the conditional branch instruction to a root node, repeatedly calling the root node so as to operate recursively to extract all nodes that are likely to be executed as the code block, and generating the second CFG.   
     
     
         11 . The apparatus of  claim 10 , wherein the generating the second CFG comprises:
 preventing duplicate inspection by checking whether each confirmed node has a history of inspection through a preset function.