System and method for detecting the duplicate product on the e-commerce platforms

Abstract

Disclosed is a duplicate product entry detection system and method for detecting duplicate products in product catalogs of e-commerce platforms running on mobile devices or IT devices. The duplicate product entries can be searched by a duplicate product detection engine running on a deduplication engine server, the similarity calculations can be carried out, and the duplicate product detection can be performed for the finally detected product pairs.

The invention claimed is: 
     
         1 . A duplicate product entry detection method for detecting duplicate products in a product catalog of an e-commerce platform running on a mobile or IT device having a product catalog database server having data on the product in the e-commerce platform, a deduplication engine server that detects the duplicate products, a product similarity metrics database server having similarity data calculated for duplicate product pairs by the deduplication engine server, an FTP server in which the similarity data on the duplicate product pairs detected by the deduplication engine server is stored in a CSV filed, an application server that labels duplicate products, a similar product database server having data on the duplicate product pairs, and a service server that searches, logs, updates and enters operation results for the labeled duplicate product entries, the duplicate product entry detection method comprising:
 sending the product information from the product catalog database server to the deduplication engine server so as to detect similar product pairs by a duplicate product detection engine running on the deduplication engine server by using a test similarity and accepting the product pairs above a threshold value designated as candidate duplicate product pairs;   sending the data on the candidate duplicate product pairs to the FTP server as the CSV file so as to perform the product labeling operations by a product specialist in a duplicate or not duplicate form or a variation of each other on a deduplication screen of the application server;   sending the data on the candidate duplicate product pairs to a similarity metric database server so as to form a classification model or similarity metrics;   sending the similarity metric to the deduplication engine server so as to develop the classification model;   sending data for a duplicate product file in the FTP server and detected as duplicate from the FTP server to the application server;   sending the data on the duplicate product file from the application server to the similar product database server;   listing potential duplicate product entries on the deduplication screen on the application server;   searching the listed potential duplicate product entries;   labeling searched listed potential duplicate product entries as to whether the labeled searched listed potential duplicate product entries are a duplicate product or not a duplicate product via a duplicate product platform screen;   sending the labeled searched listed potential duplicate product entries to the service server;   entering results of the step of labeling to the similar product database server by the service server;   retrieving the entered results of the step of labeling on the deduplication screen from the similar product database server by the deduplication engine server so as to be used by the duplicate product detection model;   training binary classification models with the label product pairs data;   using text similarity scores for the binary classification models;   enriching a text-based model with additional variables that express similarities of product images and price information and trademark characteristics; and   updating the product catalog database server by the service server to close the detected products as duplicates.