Server-assisted object recognition and tracking for mobile devices

Abstract

Exemplary embodiments for performing server-assisted object recognition and tracking are disclosed herein. For example, in certain embodiments of the disclosed technology, one or more objects are efficiently recognized and tracked on a mobile device by using a remote server that can provide high capacity computing and storage resources. With the benefit of high-speed image processing on a remote server and high-bandwidth communication networks connecting the mobile device and the remote server, it is possible to identify an object and to track changes in the object's characteristics or location, so that a user experiences seamless, real-time tracking.

1 . A method, comprising:
 continuously capturing images by a camera of a mobile device;   periodically transmitting selected ones of the images to a remote server for object recognition;   receiving from the remote server object data for a recognized object in the transmitted, images, the recognized object being a landmark in one or more of the selected ones of the images, the object data including one or more of data indicating the landmark's size, data indicating the landmark's history, data indicating the landmark's hours of operation, data indicating the landmark's telephone contact information; and   displaying the object data on the mobile device.   
     
     
         2 . The method of  claim 1 , further comprising performing preliminary object detection on the mobile device prior to the transmitting. 
     
     
         3 . The method of  claim 2 , wherein the preliminary object detection comprises a speculative detection of objects of interest in the images. 
     
     
         4 . The method of  claim 3 , further comprising refining the preliminary object detection based at least in part on the received object data. 
     
     
         5 . The method of  claim 4 , wherein the refining the preliminary object detection involves eliminating one or more objects recognized through the preliminary object detection. 
     
     
         6 . The method of  claim 1 , further comprising:
 capturing one or more additional images using the camera; and   tracking changes in a location of the recognized object in the one or more additional instances of digital image data, the tracking being performed based at least in part on the object data received from the remote server.   
     
     
         7 . The method of  claim 1 , wherein the object data received from the server comprises a model of the recognized object, the model include edge and corner data for the recognized object. 
     
     
         8 . The method of  claim 1 , wherein the transmitting and the receiving reduces a computational burden on the mobile device relative to performing object recognition only on the mobile device itself. 
     
     
         9 . The method of  claim 1 , further comprising transmitting sensor data to the remote server in addition to the images. 
     
     
         10 . The method of  claim 9 , wherein the sensor data further comprises one or more of data from an accelerometer associated with the mobile device, data from a compass associated with the mobile device, data from a gyroscope associated with the mobile device, or data associated with global positioning system coordinates for the mobile device. 
     
     
         11 . The method of  claim 9 , wherein the object data comprises data predicting a location of the recognized object based at least in part on the transmitted sensor data. 
     
     
         12 . One or more computer-readable storage media storing computer-executable instructions which when executed by a computing device cause the computing device to perform the method of  claim 1 . 
     
     
         13 . A method, comprising:
 by a mobile device:
 capturing an image using a camera on the mobile device, 
 performing a coarse-grained detection technique to detect one or more text objects in the image, and 
 making a service call to a remote server to perform remote translation services for the one or more text objects in the image. 
   
     
     
         14 . The method of  claim 13 , wherein the method further comprises:
 by the remote server,
 receiving at least a portion of the image from the mobile device; 
 translating text from the one or more text objects into one or more different languages; and 
 transmitting data to the mobile device indicative of the translated text. 
   
     
     
         15 . The method of  claim 13 , wherein the method further comprises:
 by the remote server:
 receiving sensor data from the mobile device; and 
 transmitting data indicating a predicated location of the translated text within an image, the predicted location being based at least in part on the sensor data. 
   
     
     
         16 . The method of  claim 14 , wherein the translating comprises recognizing text in the one or more text objects using optical character recognition. 
     
     
         17 . The method of  claim 16 , wherein the optical character recognition is adapted to recognize multiple languages. 
     
     
         18 . A system, comprising:
 a memory or storage device storing a program;   a processing unit operable to execute the program, wherein said execution of the program causes the processing unit to:
 receive image data from a mobile device; 
 perform object recognition for the image data; 
   generate data for an object recognized in the image data, the generated data including feature data modeling a shape of the recognized object within the image data; and   transmit the feature data to the mobile device.   
     
     
         19 . The system of  claim 18 , wherein said execution of the program causes the processing unit to:
 receive sensor data from the mobile device; and   generate prediction information indicative of an updated location of the recognized data, the prediction information being based at least in part on the sensor data.   
     
     
         20 . The system of  claim 18 , wherein said execution of the program causes the processing unit to generate data for multiple objects recognized in the image data as a result of the object recognition.