Synchronized display of street view map and video stream

Abstract

Approaches presented herein enable displaying of a street view or map in the context of a video stream object. Specifically, captured video is analyzed for a plurality of geo-coordinate and direction metadata associated with frames of the video. The video may also be analyzed for visual indicia of location or direction. A user watching the video selects an object therein, which may be then identified by its coordinates and labeled for the user. A map or street view corresponding to the selected object is shown synchronized to the video (e.g., in a side-by-side view), based on the geo-coordinate and direction metadata associated with the video frames. The synchronized video and map speed may be manipulated. The video may also be broken up by frames, and the user permitted to navigate beyond the bounds of the video, where frames are played when the user navigates to areas within the video.

What is claimed is: 
     
       1. A computer-implemented method for displaying location imagery in context with a video stream, the method comprising:
 extracting locational metadata from a set of video data, the video data comprising a plurality of frames; 
 identifying a set of geographic locations associated with the plurality of frames based on the locational metadata; 
 generating a first set of location imagery associated with the set of geographic locations; 
 displaying the first set of location imagery with the plurality of frames; 
 synchronizing the first set of location imagery to the plurality of frames based on the set of geographic locations; 
 obtaining a selection of an object in at least one frame of the set of frames; 
 identifying the selected object based on geo-coordinates and a direction in the at least one frame of the selected object in the at least one frame, conducting an Internet search based on the geo-coordinates for a textual description of the selected object, and displaying the textual description of the selected object, wherein a location of the selected object is tracked relative to a set of geo-coordinates and a frame direction of a current displayed location in the video stream as the video stream plays; 
 connecting a second set of location imagery, the second set of location imagery being associated with a geographic location adjacent to the first set of geographic locations associated with the plurality of frames, to the overlay of the first set of location imagery annotating the geographic location of at least one of the played set of frames, wherein the second set of location imagery comprises a set navigable of map features directly outside the set of geographic locations associated with the plurality of frames; and 
 permitting, in response to a user interaction, navigation of location imagery beyond the first set of location imagery by permitting navigation of the second set of location imagery. 
 
     
     
       2. The method of  claim 1 , the method further comprising:
 receiving a selection by a user of an object within the video; 
 identifying the object based on at least one of: video frame, geographic location, and visual indicia; 
 searching for descriptive information associated with the object; and 
 providing a description of the object in the first set of location imagery based on the descriptive information. 
 
     
     
       3. The method of  claim 1 , the method comprising:
 receiving, from a user, a navigation instruction in the first set of location imagery; and 
 synchronizing a set of frames of the plurality of frames to the navigation of the first set of location imagery. 
 
     
     
       4. The method of  claim 3 , wherein the plurality of frames are displayed in an order based on the navigation received from the user, and wherein the synchronizing pauses when a navigation of the first set of location imagery is received from the user to a portion of location imagery that does not have an associated frame. 
     
     
       5. The method of  claim 1 , wherein the geographic locations are identified by geographic coordinate metadata and direction metadata of the video. 
     
     
       6. The method of  claim 1 , wherein at least one of the first set of location imagery and the second set of location imagery is at least one of: a map view, a street view, an aerial view, a helicopter view, and a satellite view. 
     
     
       7. The method of  claim 1 , wherein the first set of location imagery is displayed with the plurality of frames as at least one of: a side-by-side partition, an overlay, and a pop-out window. 
     
     
       8. A system for displaying location imagery in context with a video stream, the computer system comprising:
 a memory medium comprising program instructions; 
 a bus coupled to the memory medium; and 
 a processor for executing the program instructions, the instructions causing the system to:
 extract locational metadata from a set of video data, the video data comprising a plurality of frames; 
 identify a set of geographic locations associated with the plurality of frames based on the locational metadata; 
 generate a first set of location imagery associated with the set of geographic locations; 
 display the first set of location imagery with the plurality of frames; 
 synchronize the first set of location imagery to the plurality of frames based on the set of geographic locations; 
 obtain a selection of an object in at least one frame of the set of frames; 
 identify the selected object based on geo-coordinates and a direction in the at least one frame of the selected object in the at least one frame, conducting an Internet search based on the geo-coordinates for a textual description of the selected object, and displaying the textual description of the selected object, wherein a location of the selected object is tracked relative to a set of geo-coordinates and a frame direction of a current displayed location in the video stream as the video stream plays; 
 
 connect a second set of location imagery, the second set of location imagery being associated with a geographic location adjacent to the first set of geographic locations associated with the plurality of frames, to the overlay of the first set of location imagery annotating the geographic location of at least one of the played set of frames, wherein the second set of location imagery comprises a set navigable of map features directly outside the set of geographic locations associated with the plurality of frames; and 
 permit, in response to a user interaction, navigation of location imagery beyond the first set of location imagery by permitting navigation of the second set of location imagery. 
 
     
     
       9. The system of  claim 8 , the instructions further causing the system to:
 receive a selection by a user of an object within the video; 
 identify the object based on at least one of: video frame, geographic location, and visual indicia; 
 search for descriptive information associated with the object; and 
 provide a description of the object in the first set of location imagery based on the descriptive information. 
 
     
     
       10. The system of  claim 8 , the instructions further causing the system to:
 receive, from a user, a navigation instruction in the first set of location imagery; and 
 synchronize a set of frames of the plurality of frames to the navigation of the first set of location imagery. 
 
     
     
       11. The system of  claim 10 , wherein the plurality of frames are displayed in an order based on the navigation received from the user, and wherein the synchronization pauses when a navigation of the first set of location imagery is received from the user to a portion of location imagery that does not have an associated frame. 
     
     
       12. The system of  claim 8 , wherein the geographic locations are identified by geographic coordinate metadata and direction metadata of the video. 
     
     
       13. The system of  claim 8 , wherein at least one of the first set of location imagery and the second set of location imagery is at least one of: a map view, a street view, an aerial view, a helicopter view, and a satellite view. 
     
     
       14. The system of  claim 8 , wherein the first set of location imagery is displayed with the plurality of frames as at least one of: a side-by-side partition, an overlay, and a pop-out window. 
     
     
       15. A computer program product for displaying location imagery in context with a video stream, the computer program product comprising a computer readable storage device, and program instructions stored on the computer readable storage device, to:
 extract locational metadata from a set of video data, the video data comprising a plurality of frames; 
 identify a set of geographic locations associated with the plurality of frames based on the locational metadata; 
 generate a first set of location imagery associated with the set of geographic locations; 
 display the first set of location imagery with the plurality of frames; 
 synchronize the first set of location imagery to the plurality of frames based on the set of geographic locations; 
 obtain a selection of an object in at least one frame of the set of frames; 
 identify the selected object based on geo-coordinates and a direction in the at least one frame of the selected object in the at least one frame, conducting an Internet search based on the geo-coordinates for a textual description of the selected object, and displaying the textual description of the selected object, wherein a location of the selected object is tracked relative to a set of geo-coordinates and a frame direction of a current displayed location in the video stream as the video stream plays; 
 connect a second set of location imagery, the second set of location imagery being associated with a geographic location adjacent to the first set of geographic locations associated with the plurality of frames, to the overlay of the first set of location imagery annotating the geographic location of at least one of the played set of frames, wherein the second set of location imagery comprises a set navigable of map features directly outside the set of geographic locations associated with the plurality of frames; and 
 permit, in response to a user interaction, navigation of location imagery beyond the first set of location imagery by permitting navigation of the second set of location imagery. 
 
     
     
       16. The computer program product of  claim 15 , the computer readable storage device further comprising instructions to:
 receive a selection by a user of an object within the video; 
 identify the object based on at least one of: video frame, geographic location, and visual indicia; 
 search for descriptive information associated with the object; and 
 provide a description of the object in the first set of location imagery based on the descriptive information. 
 
     
     
       17. The computer program product of  claim 15 , the computer readable storage device further comprising instructions to:
 receive, from a user, a navigation instruction in the first set of location imagery; and 
 synchronize a set of frames of the plurality of frames to the navigation of the first set of location imagery. 
 
     
     
       18. The computer program product of  claim 17 , wherein the plurality of frames are displayed in an order based on the navigation received from the user, and wherein the synchronization pauses when a navigation of the first set of location imagery is received from the user to a portion of location imagery that does not have an associated frame. 
     
     
       19. The computer program product of  claim 15 , wherein the geographic locations are identified by geographic coordinate metadata and direction metadata of the video. 
     
     
       20. The computer program product of  claim 15 , wherein at least one of the first set of location imagery and the second set of location imagery is at least one of: a map view, a street view, an aerial view, a helicopter view, and a satellite view.