Multi-dimensional surround view based search

Abstract

According to various examples, a surround view can be used as a visual search query for an object to be searched and this surround view can be compared to a database of three dimensional models. A determination can then be made about whether any of the three dimensional models match the visual search query. Based on how closely the three dimensional models match the visual search query, matching objects can be provided in various formats such as ranked lists.

What is claimed is: 
     
         1 . A method comprising;
 receiving a visual search query, wherein the visual search query includes a first surround view of an object to be searched;   comparing the first surround view to one or more stored surround views;   determining whether any of the stored surround views correspond to the first surround view;   generating a ranked list of matching results, wherein the ranked list is based on how closely the one or more stored surround views correspond to the first surround view included in the visual search query.   
     
     
         2 . The method of  claim 1 , wherein determining whether any of the stored surround views correspond to the first surround view includes determining whether subject matter in any of the stored surround views are similar in shape to the object in the first surround view. 
     
     
         3 . The method of  claim 1 , wherein determining whether any of the stored surround views correspond to the first surround view includes determining whether any of the subject matter in the stored surround views are similar in appearance to the object in the first surround view. 
     
     
         4 . The method of  claim 1 , wherein determining whether any of the stored surround views correspond to the first surround view includes determining whether any subject matter in the stored surround views include similar textures included in the first surround view. 
     
     
         5 . The method of  claim 1 , wherein the first surround view includes a first context, wherein the first context includes scenery surrounding the object, and wherein determining whether any of the stored surround views correspond to the first surround view includes determining whether any of the surround views includes a second context that matches the first context. 
     
     
         6 . The method of  claim 1 , wherein comparing the first surround view to one or more stored surround views includes extracting first measurement information for the object in the first surround view, extracting second measurement information for the one or more stored surround views, and comparing the first measurement information to the second measurement information. 
     
     
         7 . The method of  claim 6 , wherein determining whether any of the stored surround views correspond to the first surround view includes determining whether the second measurement information dimensionally fits the object associated with the first measurement information. 
     
     
         8 . A method comprising;
 receiving a visual search query, wherein the visual search query includes a surround view of an object to be searched;   selecting one or more object views from the surround view, wherein the object views correspond to two-dimensional representations of the object from different viewing angles in the surround view;   comparing the one or more object views to a plurality of stored images;   determining whether any of the stored images correspond to the one or more object views;   generating a ranked list of matching results, wherein the ranked list is based on how closely the stored images correspond to the object views of the surround view included in the visual search query.   
     
     
         9 . The method of  claim 8 , wherein one or more of the plurality of stored images are included in one or more stored surround views. 
     
     
         10 . The method of  claim 8 , wherein the stored images are extracted from stored surround views. 
     
     
         11 . The method of  claim 10 , wherein the stored surround views are stored in a database. 
     
     
         12 . The method of  claim 8 , wherein comparing the one or more object views to a plurality of stored images includes comparing the shape of the object in the surround view to the plurality of stored images. 
     
     
         13 . The method of  claim 8 , wherein comparing the one or more object views to a plurality of stored images includes comparing the appearance of the object in the surround view to the plurality of stored images. 
     
     
         14 . The method of  claim 8 , wherein comparing the one or more object views to a plurality of stored images includes comparing the texture of the object in the surround view to the plurality of stored images. 
     
     
         15 . The method of  claim 8 , wherein comparing the one or more object views to a plurality of stored images includes comparing the context of the object in the surround view to the plurality of stored images. 
     
     
         16 . A method comprising;
 receiving a visual search query, wherein the visual search query includes a two-dimensional view of an object to be searched;   comparing the two-dimensional view to one or more surround views, wherein the surround views include multi-view interactive digital media representations of content within the surround views;   determining whether any of the surround views correspond to the two-dimensional view;   generating a ranked list of matching results, wherein the ranked list is based on how closely the surround views correspond to the two-dimensional view of the object included in the visual search query.   
     
     
         17 . The method of  claim 16 , wherein comparing the two-dimensional view to one or more surround views includes comparing the two-dimensional view to one or more content views in the surround views. 
     
     
         18 . The method of  claim 16 , wherein comparing the two-dimensional view to one or more surround views includes comparing the two-dimensional view to one or more two-dimensional images extracted from the surround views from different viewing angles. 
     
     
         19 . The method of  claim 18 , wherein the two-dimensional images extracted from the surround views correspond to viewing angles that provide recognition of distinctive characteristics of the content. 
     
     
         20 . The method of  claim 16 , wherein the two-dimensional view of an object to be searched is obtained from an object surround view, wherein the object surround view includes a three-dimensional model of the object.