Commodity recognition apparatus and commodity recognition method

Abstract

The commodity recognition apparatus displays a frame for surrounding a commodity in an image captured by an image capturing module. Then the commodity recognition apparatus recognizes a candidate of the commodity imaged in the frame according to the feature amount of the image in the area surrounded by the frame, and outputs information of a highest ranked candidate commodity. If a change instruction for the candidate is received, the commodity recognition apparatus outputs information of the commodity other than the highest ranked candidate selected from the candidates.

What is claimed is: 
     
         1 . A commodity recognition apparatus, comprising:
 a memory configured to store computer-executable instructions; and   a processor configured to execute the computer-executable instructions to perform operations, the operations comprising:
 displaying, on a display screen, an image in which a plurality of commodities are displayed; 
 receiving an instruction input at a position on the image; 
 displaying a first frame surrounding a recognition area in the image for which a commodity is recognized by recognition processing for the image in the first frame, which is determined to be displayed at the position where the instruction input is received, from other commodities of the plurality of commodities as a recognition subject in response to determining that the instruction input is received while the first frame is not displayed on the image; 
 displaying a second frame surrounding a recognition area in the image for which a commodity is recognized by recognition processing for the image in the second frame, which is determined to be displayed at the position where the instruction input is received, from other commodities of the plurality of commodities as the recognition subject in response to determining that the instruction input is received at an area outside the first frame while the first frame is displayed on the image; 
 recognizing candidates of a commodity determined to be displayed in the first or second frame according to a feature amount of the image in an area surrounded by the first or second frame, and displaying, within the first or second frame, information of a best candidate of the commodity among the candidates; and 
 in response to determining that the instruction input is received at an area inside the first or second frame while the first or second frame is displayed on the image, displaying, within the first or second frame, information of another commodity, other than the best candidate among the candidates, determined to be displayed in the first or second frame where the instruction input is received. 
   
     
     
         2 . The apparatus according to  claim 1 , wherein the operations further comprise:
 erasing a frame containing a deleting mark in response to determining that the instruction input is received at a location corresponding to the deleting mark.   
     
     
         3 . The apparatus according to  claim 1 , wherein the operations further comprise:
 calculating a similarity degree between a parameter indicating an appearance feature amount of each of the plurality of commodities and a feature amount of an image in an area surrounded by a border of the first or second frame for each of the plurality of commodities and recognizing candidates in descending order of the similarity degree; and   outputting information of a commodity, of the plurality of commodities, having a highest similarity degree as the information of the best candidate.   
     
     
         4 . The apparatus according to  claim 3 , wherein the operations further comprise:
 in response to sequential inputs of the input instruction received at the area inside the first or second frame while the first or second frame is displayed on the image:   sequentially selecting, for each of the sequential inputs, a next candidate commodity for which the similarity degree is ranked just below that of a current candidate, wherein the selecting of the next candidate commodity causes the next candidate commodity to become a selected candidate commodity, and outputting, for each of the sequential inputs, information of the selected candidate commodity.   
     
     
         5 . The apparatus according to  claim 1 , wherein the operations further comprise:
 displaying the first or second frame such that a center of the first or second frame corresponds to a position at which the instruction input is received.   
     
     
         6 . The apparatus according to  claim 1 , wherein the operations further comprise:
 displaying information of a commodity, of the plurality of commodities, in the first or second frame to display; and   making a display form of the first or second frame different from a frame in which information of the commodity is displayed.   
     
     
         7 . The apparatus according to  claim 1 , wherein the first frame and the second frame has a predetermined size. 
     
     
         8 . A commodity recognition method, comprising:
 displaying, on a display screen, an image in which a plurality of commodities are rendered;   receiving an instruction input at a position on the image;   displaying a first frame surrounding a recognition area in the image for which a commodity is recognized by recognition processing for the image in the first frame, which is determined to be displayed at the position where the instruction input is received, from other commodities of the plurality of commodities as a recognition subject in response to determining that the instruction input is received while the first frame is not displayed on the image;   displaying a second frame surrounding a recognition area in the image for which a commodity is recognized by recognition processing for the image in the second frame, which is determined to be displayed at the position where the instruction input is received, from other commodities of the plurality of commodities as the recognition subject in response to determining that the instruction input is received at an area outside the first frame while the first the frame is displayed on the image;   recognizing candidates of a commodity determined to be displayed in the first or second frame according to a feature amount of the image in an area surrounded by the first or second frame, and displaying information of a best candidate of the commodity among the candidates; and   in response to determining that the instruction input is received at an area inside the first or second frame while the first or second frame is displayed on the image, displaying, within the first or second frame, information of another commodity, other than the best candidate among the candidates, determined to be displayed in the first or second frame where the instruction input is received.   
     
     
         9 . The method according to  claim 8 , further comprising:
 erasing a frame containing a deleting mark in response to determining that the position on which the instruction input is received corresponds to a position of the deleting mark.   
     
     
         10 . The method according to  claim 8 , further comprising:
 calculating a similarity degree between a parameter indicating an appearance feature amount of each of the plurality of commodities and a feature amount of an image in an area surrounded by a border of the frame for each of the plurality of commodities and recognizing candidates in a descending order of the similarity degree; and   outputting information of a commodity, of the plurality of commodities, having a highest similarity degree as the information of the best candidate.   
     
     
         11 . The method according to  claim 10 , further comprising:
 for consecutive inputs of the input instruction received at the area inside the first or second frame while the first or second frame is displayed on the image:   
       sequentially selecting, for each of the consecutive inputs, a next candidate commodity for which the similarity degree is ranked just below that of a current candidate, wherein the selecting of the next candidate commodity causes the next candidate commodity to become a selected candidate commodity, and outputting, for each of the consecutive inputs, information of the selected candidate commodity. 
     
     
         12 . The method according to  claim 8 , further comprising:
 displaying the first or second frame such that a center of the first or second frame corresponds to a position at which the instruction input is received.   
     
     
         13 . The method according to  claim 8 , further comprising:
 displaying information of a commodity, of the plurality of commodities, in the first or second frame; and   
       making a display form of the first or second frame different from a frame in which information of the commodity is displayed. 
     
     
         14 . The method according to  claim 8 , wherein the first frame and the second frame has a predetermined size.