Method for Representing Points of Interest in a View of a Real Environment on a Mobile Device and Mobile Device Therefor

Abstract

There is disclosed a method and mobile device for displaying points of interest in a view of a real environment displayed on a screen of the mobile device with a functionality for interaction with a user, which comprises the steps of: capturing an image of the real environment or a part of the real environment using a camera, determining at least one point of interest related to the real environment, determining an image position of the at least one point of interest in the image, displaying at least part of the image on at least part of the screen, overlaying a computer-generated indicator with the at least part of the image on the screen at a screen position according to the image position of the at least one point of interest, displaying a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator and which is adjacent to a bottom edge of the screen, displaying a visually perceivable relation indication indicative of a relation between the computer-generated virtual object and the computer-generated indicator. The mobile device may perform an action related to the at least one point of interest if at least part of the computer-generated virtual object displayed on the screen is touched.

1 . A non-transitory computer readable medium comprising computer readable code which, when executed by one or more processors, causes the one or more processors to:
 determine a position of at least one point of interest relative to a view of a real environment;   display a computer-generated indicator on a screen at a screen position according to the position of the at least one point of interest; and   display a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator, wherein the computer-generated virtual object includes information about the at least one point of interest,   wherein a change in the position of the at least one point of interest relative to the view of the real environment triggers a change in the screen position of the corresponding computer-generated indicator and the computer-generated virtual object.   
     
     
         2 . The computer readable medium of  claim 1 , further comprising computer code which, when executed, causes one or more processors to:
 detect input at a computer-generated virtual object; and   in response to detecting input at the computer-generated virtual object, perform an action related to the at least one point of interest associated with the computer-generated virtual object.   
     
     
         3 . The computer readable medium of  claim 1 , further comprising computer code which, when executed, causes one or more processors to:
 display a visually perceivable relation indication indicative of a relation between the computer-generated virtual object and the computer-generated indicator.   
     
     
         4 . The computer readable medium of  claim 3 , wherein the visually perceivable relation indication is a computer-generated line segment connecting the computer-generated virtual object and the computer-generated indicator and wherein the line segment runs along a direction which is defined by projecting a gravity direction to a plane of the screen. 
     
     
         5 . The computer readable medium of  claim 1 , wherein the computer-generated virtual object comprises at least one of an icon, a graphical object, text, a figure, a video, an annotation, a name, and a description of the at least one point of interest or a combination thereof. 
     
     
         6 . The computer readable medium of  claim 1 , wherein the position of the at least one point of interest is determined according to a location of the at least one point of interest in the real environment. 
     
     
         7 . The computer readable medium of  claim 1 , wherein the position of the at least one point of interest relative to the view of the real environment is determined based on an orientation of a camera relative to the at least one point of interest when capturing an image corresponding to the view of the real environment. 
     
     
         8 . The computer readable medium of  claim 7 , wherein the position of the at least one point of interest is determined from analyzing at least part of the image corresponding to the view of the real environment. 
     
     
         9 . The computer readable medium of  claim 7 , wherein the image corresponding to the view of the real environment comprises depth data. 
     
     
         10 . The computer readable medium of  claim 7 , wherein the camera comprises a time of flight camera. 
     
     
         11 . The computer readable medium of  claim 1 , further comprising computer code which, when executed, causes one or more processors to:
 determine a number of computer-generated virtual objects displayed on the screen exceeds a predetermined threshold; and   in response to the determination, hide at least one computer-generated virtual object from the number of computer-generated virtual objects displayed on the screen.   
     
     
         12 . A mobile device comprising:
 a display screen;   one or more processors; and   a memory comprising computer code executable by the one or more processors to:
 determine a position of at least one point of interest relative to a view of a real environment, 
 display a computer-generated indicator on the screen at a screen position according to the position of the at least one point of interest, and 
 display a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator, wherein the computer-generated virtual object includes information about the at least one point of interest, 
 wherein a change in the position of the at least one point of interest relative to the view of the real environment triggers a change in the screen position of the corresponding computer-generated indicator and the computer-generated virtual object. 
   
     
     
         13 . The mobile device of  claim 12 , wherein the memory further comprises computer code which, when executed causes one or more processors to:
 detect input at a computer-generated virtual object; and   in response to the detected input, perform an action related to the at least one point of interest associated with the computer-generated virtual object.   
     
     
         14 . The mobile device of  claim 13 , wherein the computer code to perform an action related to the at least one point of interest further comprises computer code which, when executed causes one or more processors to:
 determine a route to the at least one point of interest; and   display a path indicator on the screen at a screen position corresponding to the route.   
     
     
         15 . The mobile device of  claim 12 , wherein the display screen comprises a semi-transparent screen and wherein the computer code to display the computer-generated indicator and the computer-generated virtual object further comprise computer code which, when executed causes one or more processors to blend the computer-generated indicator and the computer-generated virtual object. 
     
     
         16 . The mobile device of  claim 12 , wherein the display screen comprises a head-mounted display. 
     
     
         17 . A method for displaying a virtual object in a real environment, comprising:
 determining a position of at least one point of interest relative to a view of a real environment;   displaying a computer-generated indicator on a screen at a screen position according to the position of the at least one point of interest; and   displaying a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator,   wherein a change in the position of the at least one point of interest relative to the view of the real environment triggers a change in the screen position of the corresponding computer-generated indicator and the computer-generated virtual object.   
     
     
         18 . The method of  claim 17 , further comprising:
 detecting input at a computer-generated virtual object; and   in response to detecting input at the computer-generated virtual object, performing an action related to the at least one point of interest associated with the computer-generated virtual object.   
     
     
         19 . The method of  claim 17 , further comprising:
 displaying a visually perceivable relation indication indicative of a relation between the computer-generated virtual object and the computer-generated indicator.   
     
     
         20 . The method of  claim 17 , further comprising:
 displaying a radar view with at least one point of interest according to a position of the at least one point of interest relative to the view of the real environment.