Head-mounted display device, control method of head-mounted display device, and display system

Abstract

A head-mounted display device that enables a user to visually recognize a virtual image and an external scene includes an image display unit that causes the user to visually recognize the virtual image and an augmented reality processing unit that forms on the image display unit the virtual image indicating a virtual object which is an object for providing augmented reality to the user and of which the user's visual discomfort is reduced.

What is claimed is: 
     
         1 . A head-mounted display device that enables a user to visually recognize a virtual image and an external scene, comprising:
 an image display unit that causes the user to visually recognize the virtual image; and   an augmented reality processing unit that forms on the image display unit the virtual image indicating a virtual object which is an object for providing augmented reality to the user and of which the user's visual discomfort is reduced.   
     
     
         2 . The head-mounted display device according to  claim 1 , wherein the augmented reality processing unit reduces the discomfort of the virtual object by fitting the virtual object with an ambient environment of the user using three-dimensional information in which the ambient environment is expressed as a virtual three-dimensional object in a three-dimensional space. 
     
     
         3 . The head-mounted display device according to  claim 2 , wherein the augmented reality processing unit arranges the virtual object in the three-dimensional information and reduces the discomfort of the virtual object by adding a visual effect based on the environment to at least one of the virtual object and the three-dimensional information and then converting the virtual object into a two-dimensional image. 
     
     
         4 . The head-mounted display device according to  claim 3 , wherein the visual effect based on the environment includes any one of:
 trimming a portion of the virtual object which is a shadow of the three-dimensional object in the three-dimensional information;   lighting the virtual object based on the environment; and   adjusting behavior of the virtual object based on at least one of a restitution coefficient and a friction coefficient of the three-dimensional object in the three-dimensional information set on the basis of the environment.   
     
     
         5 . The head-mounted display device according to  claim 3 , further comprising an image acquiring unit that acquires an image in the user's viewing field in a state where the user wears the head-mounted display device,
 wherein the augmented reality processing unit estimates the environment by recognizing the image in the viewing field acquired by the image acquiring unit.   
     
     
         6 . The head-mounted display device according to  claim 3 , wherein the augmented reality processing unit considers a distance from the user's eyes to the user's point of regard to convert the virtual object into a two-dimensional image. 
     
     
         7 . The head-mounted display device according to  claim 1 , wherein the augmented reality processing unit adds a visual effect of enabling a stereoscopic view of the virtual object to the virtual object to reduce the discomfort of the virtual object by generating right-eye image data for a right eye and left-eye image data for a left eye which indicate the virtual object,
 wherein the image display unit causes the user's right and left eyes to visually recognize the different virtual images using the right-eye image data and the left-eye image data,   wherein the head-mounted display device further comprises a pixel parallactic angle storage unit that stores a difference between a first convergence angle of the virtual image displayed on the basis of the right-eye image data and the left-eye image data which are identical to each other and a second convergence angle of the virtual image displayed on the basis of the right-eye image data and the left-eye image data which are laterally deviated from each other, and wherein the augmented reality processing unit generates the right-eye image data and the left-eye image data for fitting the virtual object to the external scene using the difference stored in the pixel parallactic angle storage unit.   
     
     
         8 . The head-mounted display device according to  claim 7 , wherein the augmented reality processing unit
 determines a target distance at which the user is caused to visually recognize the virtual object,   calculates a target convergence angle, which is a convergence angle at the target distance, from the determined target distance,   calculates a target parallactic angle which is a parallactic angle at the target distance using the calculated target convergence angle and the first convergence angle, and   generates the right-eye image data and the left-eye image data from single image data indicating the virtual object using the target parallactic angle and the difference stored in the pixel parallactic angle storage unit.   
     
     
         9 . The head-mounted display device according to claim  7 , wherein the augmented reality processing unit
 determines a target distance at which the user is caused to visually recognize the virtual object,   sets two virtual cameras for acquiring a 2D projection image in a 3D model space on the basis of the determined target distance,   calculates a target convergence angle, which is a convergence angle at the target distance, from the determined target distance,   calculates a target parallactic angle which is a parallactic angle at the target distance using the calculated target convergence angle and the first convergence angle, and   generates the right-eye image data by scaling the 2D projection image captured by one virtual camera and generates the left-eye image data by scaling the 2D projection image captured by the other virtual camera using the target parallactic angle and the difference stored in the pixel parallactic angle storage unit.   
     
     
         10 . The head-mounted display device according to  claim 9 , further comprising an interocular distance storage unit that stores the user's interocular distance,
 wherein the augmented reality processing unit arranges a virtual viewpoint which is a virtual point of view in the 3D model space on the basis of the determined target distance at the time of setting the two virtual cameras, arranges one virtual camera at a position separated by the interocular distance/2 from the arranged virtual viewpoint, and arranges the other virtual camera at a position separated by the interocular distance/2 from the arranged virtual viewpoint.   
     
     
         11 . The head-mounted display device according to  claim 10 , further comprising an interpupillary distance measuring unit that measures the user's interpupillary distance, wherein the measurement result by the interpupillary distance measuring unit is stored as the interocular distance in the interocular distance storage unit. 
     
     
         12 . The head-mounted display device according to  claim 1 , further comprising an object detecting unit that detects a coordinate position and a shape of a real object in a real space,
 wherein the augmented reality processing unit reduces the discomfort of the virtual object by controlling the virtual object, of which a coordinate position in the real space is set, on the basis of the coordinate position and the shape of the real object.   
     
     
         13 . The head-mounted display device according to  claim 12 , wherein the augmented reality processing unit controls behavior of the virtual object on the basis of the coordinate position and the shape of the real object. 
     
     
         14 . The head-mounted display device according to  claim 13 , wherein the augmented reality processing unit changes the virtual object on the basis of the detected coordinate position of the real object and the set coordinate position of the virtual object so as to make an overlapping portion of the virtual object with the real object disappear when the virtual object is located in the back of the real object from the user's viewpoint. 
     
     
         15 . The head-mounted display device according to  claim 13 , wherein the augmented reality processing unit causes the virtual object to behave so as to avoid the real object in a coordinate system of a real space. 
     
     
         16 . The head-mounted display device according to  claim 13 , wherein the augmented reality processing unit causes the virtual object to behave as if the virtual object collided with the real object when the virtual object moves and reaches the real object in the coordinate system of the real space. 
     
     
         17 . The head-mounted display device according to  claim 12 , further comprising an eyeline detecting unit that detects the user's eyeline,
 wherein the augmented reality processing unit displays the virtual object on the basis of the detected eyeline.   
     
     
         18 . The head-mounted display device according to  claim 12 , further comprising a distance detecting unit that detects a distance from the user to the real object,
 wherein the augmented reality processing unit displays the virtual object on the basis of the detected distance.   
     
     
         19 . The head-mounted display device according to  claim 12 , wherein the image display unit includes:
 an image light output unit that outputs image light of the image; and   a light guide section that guides the output image light to the user's eyes,   wherein the light guide section transmits external light and causes the external light along with the image light to be incident on the user's eyes.   
     
     
         20 . A transmissive display system that causes a user to visually recognize a display image as a virtual image, comprising:
 an object detecting unit that detects a coordinate position and a shape of a real object in a real space;   an augmented reality processing unit that displays a virtual object, of which a coordinate position in the real space is set, so as to visually recognize the virtual object on the basis of a coordinate system of the real space; and   an image display unit that displays the display image, wherein the augmented reality processing unit controls the virtual object on the basis of the coordinate position and the shape of the real object.   
     
     
         21 . A control method of a head-mounted display device that enables a user to visually recognize a virtual image and an external scene, comprising:
 causing the user to visually recognize the virtual image; and   forming the virtual image indicating a virtual object which is an object for providing augmented reality to the user and of which the user's visual discomfort is reduced in the causing of the user to visually recognize the virtual image.   
     
     
         22 . The control method of a head-mounted display device according to  claim 21 , wherein the forming of the virtual image on the image display unit includes reducing the discomfort of the virtual object by fitting the virtual object with an ambient environment of the user using three-dimensional information in which the ambient environment is expressed as a virtual three-dimensional object in a three-dimensional space. 
     
     
         23 . The control method of a head-mounted display device according to  claim 21 ,
 wherein the forming of the virtual image on the image display unit includes adding a visual effect of enabling a stereoscopic view of the virtual object to the virtual object to reduce the discomfort of the virtual object by generating right-eye image data for a right eye and left-eye image data for a left eye which indicate the virtual object,   wherein the causing of the user to visually recognize the virtual image includes causing the user's right and left eyes to visually recognize the different virtual images using the right-eye image data and the left-eye image data, and   wherein the forming of the virtual image on the image display unit includes generating the right-eye image data and the left-eye image data for fitting the virtual object to the external scene using a difference between a first convergence angle of the virtual image displayed on the basis of the right-eye image data and the left-eye image data which are identical to each other and a second convergence angle of the virtual image displayed on the basis of the right-eye image data and the left-eye image data which are laterally deviated from each other.   
     
     
         24 . The control method of a head-mounted display device according to  claim 21 , further comprising detecting a coordinate position and a shape of a real object in the real space,
 wherein the forming of the virtual image on the image display unit includes reducing the discomfort of the virtual object by controlling the virtual object, of which a coordinate position in the real space is set, on the basis of the coordinate position and the shape of the real object.