Interfacing between digital assistant applications and navigation applications

Abstract

The present disclosure is generally related to systems and methods of interfacing among multiple applications in a networked computer environment. A data processing system can access a navigation application to retrieve point locations within a reference frame corresponding to a geographic region displayed in a viewport of the navigation application. Each point location can have an identifier. The data processing system can parse an input audio signal to identify a request and a referential word. The data processing system can identify a point location within the reference frame based on the referential word parsed from the input audio signal and the identifier for the point location. The data processing system can generate an action data structure including the point location identified. The data processing system can transmit the action data structure to the navigation application to initiate a navigation guidance process using the point location.

1 .- 20 . (canceled) 
     
     
         21 . A system to interface among multiple applications in a networked computer environment, comprising:
 a navigation interface component executed on a data processing system having one or more processors to identify a plurality of point locations within a reference frame corresponding to a geographic region visible in a viewport of a navigation application executing on a first client device, each of the plurality of point locations having an identifier;   a natural language processor component executed on the data processing system to:
 receive an input audio signal detected by a sensor of at least one of the first client device or a second client device; 
 parse the input audio signal to identify a request and a referential word; and 
 identify, responsive to the identification of the request, a subset of point locations from the plurality of point locations within the reference frame based on the referential word parsed from the input audio signal and the identifier for each point location of the subset; and 
   a direct action handler component executed on the data processing system to:
 generate an action data structure using the subset of point locations identified responsive to the detection of the input audio signal; and 
 provide, to at least one of the first client device or the second client device, the action data structure to present information corresponding to at least one of the subset of point locations. 
   
     
     
         22 . The system of  claim 21 , comprising the direct action handler component to provide the action data structure to a digital assistant application executing at least one of the first client device or the second client device, receipt of the action data structure to cause the digital assistant application to present a digital component including the information to indicate the subset of point locations identified from the plurality of point locations. 
     
     
         23 . The system of  claim 21 , comprising the direct action handler component to:
 provide the action data structure to the navigation application executing on the first client device, receipt of the action data structure to cause the navigation application to use the action data structure to generate a response including the information; and   provide, to a digital assistant application executing on at least one of the first client device or the second client device, the response generated by the navigation application to present a digital component including the information.   
     
     
         24 . The system of  claim 21 , comprising the navigation interface component to identify the first client device as having the navigation application and identify the second client device as lacking the navigation application; and
 the direct action handler component to:
 provide, responsive to the identification of the first client device as having the navigation application, the action data structure to the first client device, receipt of the action data structure to cause the navigation application to use the action data structure to initiate a navigation guidance process; and 
 provide, responsive to the identification of the second client device as lacking the navigation application, a second action data structure to present a digital component including the information. 
   
     
     
         25 . The system of  claim 21 , comprising the direct action handler component to provide the action data structure to the navigation application executing on the first client device, receipt of the action data structure to cause the navigation application to initiate a navigation guidance process using the subset of point locations and to present the information based on the navigation guidance process. 
     
     
         26 . The system of  claim 21 , comprising the natural language processor component to:
 parse the input audio signal to identify an auxiliary word different from the referential word;   determine a subset area of the viewport of the navigation application based on the auxiliary word; and   select the subset of point locations from the plurality of point locations corresponding to the subset area of the viewport.   
     
     
         27 . The system of  claim 21 , comprising the natural language processor component to:
 receive, subsequent to the receipt of the input audio signal, a second input audio signal detected by the sensor of at least one of the first client device or the second client device;   parse the second input audio signal to identify a second referential word; and   select the subset of point locations from the plurality of point locations based on the second referential word.   
     
     
         28 . The system of  claim 21 , comprising:
 the navigation interface component to:
 determine a first portion of the reference frame corresponding to the geographic region displayed concurrently to the receipt of the input audio signal; and 
 determine a second portion of the reference frame corresponding to the geographic region previously displayed in the viewport based on a measurement of the first client device acquired from an inertial motion unit; and 
   the natural language processor component to identify the subset of point locations from the plurality of point locations within the reference frame based on the measurement.   
     
     
         29 . The system of  claim 21 , comprising:
 the navigation interface component to identify a plurality of search terms received by the navigation application within a time window prior to the receipt of the input audio signal; and   the natural language processor component to:
 determine, for each point location of the plurality of point locations and each search term of the plurality of search terms, a semantic distance between the identifier of the point location and the search term using a semantic knowledge graph; and 
 select the subset of point locations from the plurality of point locations based on a plurality of semantic distances. 
   
     
     
         30 . The system of  claim 21 , comprising the
 the natural language processor component to determine a request type corresponding to an operation of a plurality of operations to be performed by the navigation application based on the request; and   the direct action handler component to generate the action data structure including the request type and to transmit the action data structure to the first client device to cause the navigation application to initiate the operation of a navigation guidance process corresponding to the request type to present the information.   
     
     
         31 . A method of interfacing among multiple applications in a networked computer environment, comprising:
 identifying, by a data processing system, a plurality of point locations within a reference frame corresponding to a geographic region visible in a viewport of a navigation application executing on a first client device, each of the plurality of point locations having an identifier;   receiving, by the data processing system, an input audio signal detected by a sensor of at least one of the first client device or a second client device;   parsing, by the data processing system, the input audio signal to identify a request and a referential word;   identifying, by the data processing system, responsive to identifying the request, a subset of point locations from the plurality of point locations within the reference frame based on the referential word parsed from the input audio signal and the identifier for each point location of the subset;   generating, by the data processing system, an action data structure using the subset of point locations identified responsive to detecting of the input audio signal; and   providing, by the data processing system, to at least one of the first client device or the second client device, the action data structure to present information corresponding to at least one of the subset of point locations.   
     
     
         32 . The method of  claim 31 , comprising providing, by the data processing system, the action data structure to a digital assistant application executing at least one of the first client device or the second client device, receipt of the action data structure to cause the digital assistant application to present a digital component including the information to indicate the subset of point locations identified from the plurality of point locations. 
     
     
         33 . The method of  claim 31 , comprising
 providing, by the data processing system, the action data structure to the navigation application executing on the first client device, receipt of the action data structure to cause the navigation application to use the action data structure to generate a response including the information; and   providing, by the data processing system, to a digital assistant application executing on at least one of the first client device or the second client device, the response generated by the navigation application to present a digital component including the information.   
     
     
         34 . The method of  claim 31 , comprising
 identifying, by the data processing system, the first client device as having the navigation application and identify the second client device as lacking the navigation application;   providing, by the data processing system, responsive to identifying the first client device as having the navigation application, the action data structure to the first client device, receipt of the action data structure to cause the navigation application to use the action data structure to initiate a navigation guidance process; and   providing, by the data processing system, responsive identifying the second client device as lacking the navigation application, a second action data structure to present a digital component including the information.   
     
     
         35 . The method of  claim 31 , comprising providing, by the data processing system, the action data structure to the navigation application executing on the first client device, receipt of the action data structure to cause the navigation application to initiate a navigation guidance process using the subset of point locations and to present the information based on the navigation guidance process. 
     
     
         36 . The method of  claim 31 , comprising:
 parsing, by the data processing system, the input audio signal to identify an auxiliary word different from the referential word;   determining, by the data processing system, a subset area of the viewport of the navigation application based on the auxiliary word; and   selecting, by the data processing system, the subset of point locations from the plurality of point locations corresponding to the subset area of the viewport.   
     
     
         37 . The method of  claim 31 , comprising:
 receiving, by the data processing system, subsequent to receipt of the input audio signal, a second input audio signal detected by the sensor of at least one of the first client device or the second client device;   parsing, by the data processing system, the second input audio signal to identify a second referential word; and   selecting, by the data processing system, the subset of point locations from the plurality of point locations based on the second referential word.   
     
     
         38 . The method of  claim 31 , comprising:
 determining, by the data processing system, a first portion of the reference frame corresponding to the geographic region displayed concurrently to the receipt of the input audio signal;   determining, by the data processing system, a second portion of the reference frame corresponding to the geographic region previously displayed in the viewport based on a measurement of the first client device acquired from an inertial motion unit; and   identifying, by the data processing system, the subset of point locations from the plurality of point locations within the reference frame based on the measurement.   
     
     
         39 . The method of  claim 31 , comprising:
 identifying, by the data processing system, a plurality of search terms received by the navigation application within a time window prior to the receipt of the input audio signal;   determining, by the data processing system, for each point location of the plurality of point locations and each search term of the plurality of search terms, a semantic distance between the identifier of the point location and the search term using a semantic knowledge graph; and   selecting, by the data processing system, the subset of point locations from the plurality of point locations based on a plurality of semantic distances.   
     
     
         40 . The method of  claim 31 , comprising:
 determining, by the data processing system, a request type corresponding to an operation of a plurality of operations to be performed by the navigation application based on the request; and   generating, by the data processing system, the action data structure including the request type and to transmit the action data structure to the first client device to cause the navigation application to initiate the operation of a navigation guidance process corresponding to the request type to present the information.