Object identification and device communication through image and audio signals

Abstract

Deterministic identifiers fuel reliable efficient capture of product discovery, purchase and consumption events, which in turn enable more reliable product recommendation, more accurate shopping list generation and in-store navigation. A mobile device, equipped with image and audio detectors, extracts product identifiers from objects, display screens and ambient audio. In conjunction with a cloud-based service, a mobile device application obtains product information and logs product events for extracted identifiers. The cloud service generates recommendations, and mapping for in-store navigation. The detectors also provide reliable and efficient product identification for purchase events, and post shopping product consumption events.

The invention claimed is: 
     
       1. A system comprising:
 a first computer configured with instructions to encode a first digital identifier in a first audio watermark and embed the first audio watermark in first audio; 
 a cloud computer, configured with instructions to receive the first digital identifier extracted from the first audio watermark at a user computer, wherein the user computer comprises a smart speaker configured with instructions to extract the first digital identifier from the first audio watermark embedded in the first audio, look up first product information associated with the first digital identifier, and submit the first product information to a machine learning service, the machine learning service being trained to provide a product recommendation in response to the first product information. 
 
     
     
       2. The system of  claim 1  wherein the smart speaker comprises a microphone, and wherein the smart speaker is configured with instructions to extract the first audio watermark from the first audio captured through the microphone. 
     
     
       3. The system of  claim 1  wherein the first computer is configured with instructions to embed the first audio watermark in frequencies of the first audio above 18 kHz, and the user computer is configured to extract the first digital identifier from audio frequencies above 18 kHz. 
     
     
       4. The system of  claim 1  wherein the first computer is configured with instructions to embed the first audio watermark in frequencies of the first audio below 16 kHz, and the user computer is configured to extract the first digital identifier from audio frequencies below 16 kHz. 
     
     
       5. The system of  claim 1  wherein the cloud computer is configured to receive plural digital identifiers extracted from audio watermarks by the user computer, look up product information corresponding to the plural digital identifiers, and submit the product information corresponding to the plural digital identifiers to a machine learning service, the machine learning service being trained to provide a product recommendation in response to the product information corresponding to the plural digital identifiers. 
     
     
       6. The system of  claim 1  wherein the cloud computer is configured to receive plural digital identifiers extracted from image watermarks by the user computer, look up product information corresponding to the plural digital identifiers, and submit the product information corresponding to the plural digital identifiers to a machine learning service, the machine learning service being trained to provide a product recommendation in response to the product information corresponding to the plural digital identifiers. 
     
     
       7. A method of communicating data through an audio signal, the method comprising:
 at a first computer, encoding a first digital identifier in a first audio watermark; 
 embedding the first audio watermark in first audio; 
 at a second computer, receiving the first digital identifier extracted from the first audio watermark at a smart speaker; 
 looking up first product information associated with the first digital identifier; and 
 submitting the first product information to a machine learning service, the machine learning service being trained to provide a product recommendation in response to the first product information; and 
 providing the product recommendation to a memory corresponding to a user of the smart speaker. 
 
     
     
       8. The method of  claim 7  wherein the smart speaker comprises a microphone, and wherein the smart speaker is configured to extract the first audio watermark from the first audio captured through the microphone. 
     
     
       9. The method of  claim 8  further comprising:
 embedding the first audio watermark in frequencies of the first audio above 18 kHz, and the smart speaker is configured to extract the first digital identifier from audio frequencies above 18 kHz. 
 
     
     
       10. The method of  claim 8  further comprising:
 embedding the first audio watermark in frequencies of the first audio below 16 kHz, and the smart speaker is configured to extract the first digital identifier from audio frequencies below 16 kHz. 
 
     
     
       11. The method of  claim 7  further comprising:
 receiving plural digital identifiers extracted from audio watermarks by the smart speaker; 
 looking up product information corresponding to the plural digital identifiers, and 
 submitting the product information corresponding to the plural digital identifiers to a machine learning service, the machine learning service being trained to provide a product recommendation in response to the product information corresponding to the plural digital identifiers. 
 
     
     
       12. The method of  claim 7  further comprising:
 receiving plural digital identifiers extracted from image watermarks by a mobile device, 
 looking up product information corresponding to the plural digital identifiers, and 
 submitting the product information corresponding to the plural digital identifiers to a machine learning service, the machine learning service being trained to provide a product recommendation in response to the product information corresponding to the plural digital identifiers.