Map Search Recommendation System Based on Image Content Analysis Driven Geo-Semantic Index

Abstract

The present disclosure provides systems and methods that enable map search recommendations based on a geo-semantic index developed using image content analysis. In one example, a computer-implemented method can include obtaining, by one or more computing devices, a vocabulary of image feature types associated with user activities. The method can include obtaining a collection of imagery. The method can include performing image content analysis on the collection of imagery based on the vocabulary of image feature types. The method can include generating at least one activity score for each of a plurality of location cells in a geo-semantic index based at least in part on the vocabulary of image feature types. The method can include populating the geo-semantic index of location cells, the geo-semantic index of location cells including data indicative of the at least one activity score for each location cell. The method can include providing the geo-semantic index of location cells for use in generating location recommendations in response to a query.

1 .- 20 . (canceled) 
     
     
         21 . A computer-implemented method, the method comprising:
 obtaining, by one or more computing devices, a vocabulary of image feature types associated with user activities;   obtaining, by the one or more computing devices, a collection of imagery;   performing, by the one or more computing devices, image content analysis on the collection of imagery based on the vocabulary of image feature types;   generating, by the one or more computing devices, at least one activity score for each of a plurality of locations based at least in part on the vocabulary of image feature types; and   providing, by the one or more computing devices, a location recommendation based on the activity scores for the plurality of locations in response to a query associated with a particular activity.   
     
     
         22 . The computer-implemented method of  claim 21 , wherein the plurality of locations are represented by a plurality of location cells in a geo-semantic index of location cells and the method further comprises:
 populating, by the one or more computing devices, the geo-semantic index of location cells, the geo-semantic index of location cells including data indicative of the at least one activity score for each location cell; and   providing, by the one or more computing devices, the geo-semantic index of location cells for use in generating location recommendations in response to a query.   
     
     
         23 . The computer-implemented method of  claim 22 , wherein generating, by the one or more computing devices, the at least one activity score for each of the locations cells includes:
 generating the at least one activity score for each location cell based at least in part on corresponding image feature types identified by the image content analysis.   
     
     
         24 . The computer-implemented method of  claim 22 , wherein the method further comprises:
 associating, by the one or more computing devices, each activity type of a plurality of activity types with a respective subset of image feature types from the vocabulary of image feature types;   wherein performing, by the one or more computing devices, the image content analysis on the collection of imagery comprises determining whether corresponding imagery for each location cell includes feature types from the vocabulary of image feature types; and   wherein generating, by the one or more computing devices, the at least one activity score for each of the location cells comprises determining an activity score for each activity type based at least in part on whether such location cell includes the respective subset of image feature types for such activity type.   
     
     
         25 . The computer-implemented method of  claim 22 , wherein a first location cell includes a first activity score associated with a first activity type and a second activity score for a second activity type, and wherein the first activity score is based at least in part on identifying a first subset of feature types associated with the first location cell and the second activity score is based at least in part on identifying a second subset of feature types associated with the first location cell. 
     
     
         26 . The computer-implemented method of  claim 22 , wherein the method further comprises:
 obtaining, by the one or more computing devices, a query for location recommendations for a desired activity;   obtaining, by the one or more computing devices, the geo-semantic index of location cells;   determining, by the one or more computing devices, one or more location cells within a geographic area associated with the query based on an activity score associated with the desired activity from the geo-semantic index of location cells; and   providing, by the one or more computing devices, a recommendation including one or more selected locations based on the determined location cells.   
     
     
         27 . The computer-implemented method of  claim 22 , wherein the method further comprises:
 obtaining, by the one or more computing devices, a query for activity recommendations within a defined geographic area;   obtaining, by the one or more computing devices, the geo-semantic index of location cells;   determining, by the one or more computing devices, one or more user activities associated with location cells within the defined geographic area based in part on activity scores associated with the one or more user activities in the geo-semantic index of location cells; and   providing, by the one or more computing devices, a recommendation including the one or more user activities determined based in part on the activity scores and a location within the defined geographic area for each of the one or more user activities.   
     
     
         28 . The computer-implemented method of  claim 22 , wherein the method further comprises:
 generating, by the one or more computing devices, a description for a location cell based in part on feature types and activities associated with the location cell as identified by the content image analysis.   
     
     
         29 . The computer-implemented method of  claim 22 , wherein the method further comprises:
 filtering out, by the one or more computing devices, feature types identified by the image content analysis that are identified in restricted geographic locations such that they are not populated in the geo-semantic index.   
     
     
         30 . The computer-implemented method of  claim 21 , wherein generating, by the one or more computing devices, an activity score for each of the locations based on supported user activities comprises generating a weighted activity score for each location based at least in part on an amount, diversity, and recency of identified image feature types corresponding to activities that are associated with the location. 
     
     
         31 . The computer-implemented method of  claim 21 , wherein obtaining, by the one or more computing devices, a vocabulary of image feature types associated with user activities comprises cataloguing a vocabulary of known feature types from an image content analysis lexicon along with user activities that the feature types can accommodate. 
     
     
         32 . The computer-implemented method of  claim 21 , wherein the collection of imagery comprises a periodic collection of street-level imagery. 
     
     
         33 . A computing system, comprising:
 one or more processors; and   one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
 obtaining a vocabulary of image feature types associated with user activities; 
 obtaining a collection of imagery; 
 performing image content analysis on the collection of imagery based on the vocabulary of image feature types; 
 generating at least one activity score for each of a plurality of locations based at least in part on the vocabulary of image feature types; and 
 providing a location recommendation based on the activity scores for the plurality of locations in response to a query associated with a particular activity. 
   
     
     
         34 . The computing system of  claim 33 , wherein the plurality of locations are represented by a plurality of location cells in a geo-semantic index of location cells and the operations further comprise:
 populating the geo-semantic index of location cells, the geo-semantic index of location cells including data indicative of the at least one activity score for each location cell; and   providing the geo-semantic index of location cells for use in generating location recommendations in response to a query   
     
     
         35 . The computing system of  claim 34 , wherein generating the at least one activity score for each of the location cells includes generating the at least one activity score for each location cell based at least in part on corresponding image feature types identified by the image content analysis. 
     
     
         36 . The computing system of  claim 34 , the operations further comprising:
 obtaining a query for location recommendations for a desired activity;   obtaining the geo-semantic index of location cells;   determining one or more location cells within a geographic area associated with the query based on an activity score associated with the desired activity in the geo-semantic index of location cells; and   providing a recommendation including one or more selected locations based on the determined location cells.   
     
     
         37 . The computing system of  claim 34 , the operations further comprising:
 generating a description for a location cell based in part on feature types and activities associated with the location cell as identified by the content image analysis.   
     
     
         38 . The computing system of  claim 34 , wherein generating an activity score for each of the location cells in the geo-semantic index based on supported user activities comprises generating a weighted activity score for each location cell based at least in part on an amount, diversity, and recency of semantic terms corresponding to activities that are associated with the location cell. 
     
     
         39 . One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to:
 obtain a vocabulary of image feature types associated with user activities;   obtain a collection of imagery;   perform image content analysis on the collection of imagery based on the vocabulary of image feature types;   generate at least one activity score for each of a plurality of locations based at least in part on the vocabulary of image feature types; and   provide a location recommendation based on the activity scores for the plurality of locations in response to a query associated with a particular activity.   
     
     
         40 . The one or more non-transitory computer-readable media of  claim 39 , wherein the collection of imagery comprises a periodic collection of street-level imagery.