Image pick-up apparatus for stereoscope

Abstract

An image pick-up apparatus for capturing parallax images for the left eye and right eye which meet a fusion condition. A stereo camera image-captures a subject and displays captured parallax images on a stereoscopic display. An observer looks at the screen of the display. The lines of sight of both eyes of the observer are then detected, and a distance from the view point of the observer to the subject is measured based on the line-of-sight data. Instead of the observer, the apparatus determines whether the subject distance satisfies the fusion condition. The apparatus thus determines an image-fusible range, based on a (known) convergence angle, a (known) base line distance, a (known) size of the display, and the distance of distinct vision of the observer.

What is claimed is:  
     
         1 . An image pick-up apparatus for capturing images for stereoscopic viewing, comprising: 
 image pick-up means for picking up left and right parallax images of a main subject respectively for the left and right eyes;    display means for displaying the left and right parallax images, picked up by said image pick-up means;    line-of-sight detection means for detecting the lines of sight of the left and right eyes looking toward the respective images displayed by said display means; and    determining means for determining, based on the output of said line-of-sight detection means, whether said main subject falls within an image-fusible range.    
     
     
         2 . An image pick-up apparatus according to  claim 1 , further comprising informing means for informing a user of a determination result provided by said determining means.  
     
     
         3 . An image pick-up apparatus according to  claim 2 , wherein said informing means displays said determination result.  
     
     
         4 . An image pick-up apparatus according to  claim 1 , wherein said determining means defines said image-fusible range according to the inter-pupillary distance and the distance of distinct vision of a user.  
     
     
         5 . An image pick-up apparatus according to  claim 1 , wherein the output of said image pick-up means is stored in a memory in response to the output of said determining means.  
     
     
         6 . An image pick-up apparatus comprising: 
 an image pick-up optical system for picking up a plurality of parallax images;    a plurality of image display means, each having line-of-sight detection means;    calculator means for performing a predetermined calculation to an output value provided by said line-of-sight detection means; and    control means for controlling said image display means in response to the result provided by said calculator means.    
     
     
         7 . An image pick-up apparatus according to  claim 6 , wherein said image display means comprises a pair of display unit and line-of-sight detector unit for the left eye and a pair of display unit and line-of-sight detector unit for the right eye.  
     
     
         8 . An image pick-up apparatus according to  claim 1 , wherein said line-of-sight detection means further comprises: 
 converting means for converting the left and right lines of sight of a user into left and right direction vectors that are respectively expressed in left and right coordinate systems of said image pick-up means; and    coordinates calculator means for calculating the coordinate values of the crossing point of the left and right direction vectors in the world coordinate system.    
     
     
         9 . An image pick-up apparatus according to  claim 8 , wherein said determining means determines whether the calculated coordinate values of the crossing point in the world coordinate system fall within said image-fusible range.  
     
     
         10 . An image pick-up apparatus according to  claim 1 , wherein said determining means expresses said image-fusible range according to a farthest position and a nearest position from the view point of a user in the direction of depth.  
     
     
         11 . An image pick-up apparatus according to  claim 10 , wherein said determining means sets the farthest position of said image-fusible range to be a point so that the horizontal distance between two second points on the left and right image planes of said image pick-up means corresponding to a first point (A) of said farthest position is substantially equal to the inter-pupillary distance of the user.  
     
     
         12 . An image pick-up apparatus according to  claim 10 , wherein said determining means sets the nearest position of said image-fusible range to be a point so that a position (C′) where two points on the left and right image planes of said image pick-up means corresponding to a first point (C) of the nearest position, through perspective transformation based on the left and right view points of the user, look standing out to the user, is approximately equal to the point at the distance of distinct vision of the user.  
     
     
         13 . An image pick-up apparatus according to  claim 1 , wherein said image display means and said line-of-sight detection means are built into a head-mounted display.  
     
     
         14 . An image pick-up apparatus according to claim  13 , wherein said head-mounted display comprises a light-emitting diode.  
     
     
         15 . An image pick-up apparatus comprising: 
 a first optical system and a second optical system with a predetermined convergence angle made therebetween;    a first electronic shutter and a second electronic shutter for electronically blocking the respective optical paths of said first and second optical systems;    control means for driving said first and second electronic shutters in a time-division manner;    optical path integrator means for integrating the optical paths of said first and second optical systems;    a third optical system having the integrated optical path;    a charge-coupled device for photoelectrically converting an optical image transmitted through said third optical system;    reading means for reading the output of said charge-coupled device in a time-division manner in synchronization with time-division driving of said first and second electronic shutter by said control means;    distance measuring means for measuring a distance to a subject; and    adjusting means for adjusting the convergence angle between said first and second optical systems in accordance with the measured distance.    
     
     
         16 . An image pick-up apparatus according to  claim 15 , wherein said optical path integrator means comprises: 
 a prism arranged at the entrance of the optical path of said third optical system;    a first mirror for deflecting the optical path of said first optical system toward said prism;    a second mirror for deflecting the optical path of said second optical system toward said prism; and    said adjusting means comprises angle adjusting means for controlling the angles of pivot of said first and second mirrors.    
     
     
         17 . An image pick-up apparatus according to  claim 16 , wherein said angle adjusting means pivots said first and second mirrors in the same angles but in opposite directions.  
     
     
         18 . An image pick-up apparatus according to  claim 15 , wherein said distance measuring means uses a triangulation method.  
     
     
         19 . An image pick-up apparatus according to  claim 15 , wherein each of said first and second optical systems comprises a plurality of lenses, and wherein said distance measuring means measures distance based on position information about said plurality of lenses.  
     
     
         20 . An image pick-up apparatus according to  claim 15 , wherein each of said electronic shutters comprises a liquid-crystal panel interposed between polarizers.  
     
     
         21 . An image pick-up apparatus according to  claim 15 , further comprising: 
 detector means for detecting a change, in the distance to the subject, in excess of a predetermined value; and    activating means for activating said adjusting means in response to the output of said detector means.    
     
     
         22 . An image pick-up apparatus according to  claim 15 , further comprising: 
 detector means for detecting a predetermined number or larger number of changes in the distance to the subject, each change in excess of a predetermined value; and    activating means for activating said adjusting means in response to the output of said detector means.    
     
     
         23 . An image pick-up apparatus according to  claim 15 , wherein the crossing point of the optical paths of said first and second optical systems includes infinity.  
     
     
         24 . An image pick-up apparatus according to  claim 15 , further comprising: 
 a camera main unit for processing an output signal from said charge-coupled device;    a lens unit for driving said optical systems; and    an interconnection unit for electrically connecting said camera main unit to said lens unit.    
     
     
         25 . An image pick-up apparatus according to  claim 24 , further comprising a camera mount for said camera main unit and a lens mount for said lens unit, wherein said lens unit is detachably mounted onto said camera main unit.  
     
     
         26 . An image pick-up apparatus according to  claim 24 , wherein said camera main unit and said lens unit are controlled by respective dedicated microcomputers.  
     
     
         27 . An image pick-up apparatus according to  claim 15 , wherein said charge-coupled device is arranged on one-for-each-color basis.  
     
     
         28 . An image pick-up apparatus according to  claim 15 , further comprising display means for displaying the output signal of said charge-coupled device read by said reading means.  
     
     
         29 . An image pick-up apparatus according to  claim 28 , wherein said display means is a view finder.  
     
     
         30 . An image pick-up method for capturing images for stereoscopic viewing, comprising: 
 the step of picking up left and right parallax images of a main subject respectively for the left and right eyes;    the step of displaying the picked left and right parallax images;    the step of detecting the lines of sight of the left and right eyes looking toward the respective displayed images; and    the step of determining, based on the detected lines of sight of the eyes, whether said main subject falls within an image-fusible range.    
     
     
         31 . An image pick-up method according to  claim 30 , further comprising the step of informing a user of the determination result of said determining step.  
     
     
         32 . An image pick-up method according to  claim 31 , wherein said informing step displays said determination result.  
     
     
         33 . An image pick-up method according to  claim 30 , wherein said determining step defines said image-fusible range according to the inter-pupillary distance and the distance of distinct vision of a user.  
     
     
         34 . An image pick-up method according to  claim 30 , further comprising the step of storing the output of said image pick-up step in a memory in response to the output of said determining step.  
     
     
         35 . An image pick-up method according to  claim 30 , wherein said step of detecting the lines of sight comprises the step of converting the left and right lines of sight of a user into left and right direction vectors that are respectively expressed in left and right coordinate systems in said image pick-up step; and 
 the step of calculating the coordinate values of a crossing point of the left and right direction vectors in the world coordinate system.    
     
     
         36 . An image pick-up method according to  claim 35 , wherein said determining step determines whether the calculated coordinate values of the crossing point in the world coordinate system fall within said image-fusible range.  
     
     
         37 . An image pick-up method according to  claim 30 , wherein said determining step expresses said image-fusible range according to a farthest position and a nearest position from the view point of a user in the direction of depth.  
     
     
         38 . An image pick-up method according to  claim 37 , wherein said determining step sets the farthest position of said image-fusible range to be a point so that the horizontal distance between two second points in left and right image planes of said image pick-up step corresponding to a first point (A) of said farthest position is substantially equal to the inter-pupillary distance of the user.  
     
     
         39 . An image pick-up method according to  claim 37 , wherein said determining step sets the nearest position of said image-fusible range to be a point so that a position (C′) where two points on the left and right image planes of said image pick-up step corresponding to a first point (C) of the nearest position, through perspective transformation based on the left and right view points of the user, look standing out to the user, is approximately equal to the point at the distance of distinct vision of the user.  
     
     
         40 . An image pick-up method for using an image pick-up apparatus comprising a first optical system and a second optical system with a predetermined convergence angle made therebetween, a first electronic shutter and a second electronic shutter for electronically blocking the respective optical paths of said first and second optical systems, optical path integrator means for integrating the optical paths of said first and second optical systems, a third optical system having the integrated optical path, and a charge-coupled device for photoelectrically converting an optical image transmitted through said third optical system, said method comprising: 
 the shutter driving step of driving said first electronic shutter and said second electronic shutter in a time-division manner so that one of the optical paths of said first optical system and the optical path of said second optical path is selected in a time-division manner to lead to the optical path of said third optical system;    the reading step of reading the output of said charge-coupled device in a time-division manner in synchronization with time-division driving of said first and second electronic shutters;    the distance measuring step of measuring a distance to a subject; and    the adjusting step of adjusting the convergence angle between said first and second optical systems in accordance with the measured distance to the subject.    
     
     
         41 . An image pick-up method according to  claim 40 , wherein said image pick-up apparatus comprises: 
 a prism arranged at the entrance of the optical path of said third optical system;    a first mirror for deflecting the optical path of said first optical system toward said prism;    a second mirror for deflecting the optical path of said second optical system toward said prism; and    wherein said adjusting step comprises the angle adjusting step of controlling the angles of pivot of said first and second mirrors.    
     
     
         42 . An image pick-up method according to  claim 41 , wherein said angle adjusting step pivots said first and second mirrors in the same angles but in opposite directions.  
     
     
         43 . An image pick-up method according to  claim 40 , wherein said distance measuring step uses a triangulation method.  
     
     
         44 . An image pick-up method according to  claim 40 , wherein each of said first and second optical systems comprises a plurality of lenses, and wherein said distance measuring step measures distance based on position information about said plurality of lenses.  
     
     
         45 . An image pick-up method according to  claim 40 , wherein each of said electronic shutters comprises a liquid-crystal panel interposed between polarizers, and wherein said shutter driving step controls a voltage applied to said liquid-crystal panel.  
     
     
         46 . An image pick-up method according to  claim 40 , further comprising: 
 the detecting step of detecting a change, in the distance to the subject, in excess of a predetermined value; and    the activating step of activating said adjusting step in response to the output of said detecting step.    
     
     
         47 . An image pick-up method according to  claim 40 , further comprising: 
 the detecting step of detecting a predetermined number or larger number of changes in the distance to the subject, each change in excess of a predetermined value; and    the activating step of activating said adjusting step in response to the output of said detecting step.    
     
     
         48 . An image pick-up method according to  claim 40 , wherein the crossing point of the optical paths of said first and second optical systems includes infinity.  
     
     
         49 . An image pick-up method according to  claim 40 , further comprising the display step of displaying the output signal of said charge-coupled device read in said reading step.  
     
     
         50 . An image pick-up method according to  claim 49 , wherein said display step presents an image on a view finder.  
     
     
         51 . An image pick-up apparatus for capturing images for stereoscopic viewing, comprising: 
 image pick-up means for picking up left and right parallax images of a main subject respectively for the left and right eyes;    line-of-sight detection means for detecting the lines of sight of the left and right eyes looking toward said main subject;    determining means for determining, based on the output of said line-of-sight detection means, whether said main subject falls within an image-fusible range; and    recording means for recording, in accordance with the output of said determining means, the left parallax image signal and the right parallax image signal from said image pick-up means.    
     
     
         52 . An image pick-up method for capturing images for stereoscopic viewing, comprising: 
 the image pick-up step of picking up left and right parallax images of a main subject respectively for the left and right eyes;    the line-of-sight detecting step of detecting the lines of sight of the left and right eyes, looking toward said main subject;    the determining step of determining, based on the output of said line-of-sight detecting step, whether said main subject falls within an image-fusible range; and    the recording step of recording, in accordance with the output of said determining step, the left parallax image signal and the right parallax image signal obtained through said image pick-up step.    
     
     
         53 . An image pick-up apparatus according to  claim 1 , wherein said image pick-up means comprises adjusting means for adjusting either the base line distance between said first and second optical systems or the convergence angle between said first and second optical systems, or for adjusting both the base line distance and the convergence angle.  
     
     
         54 . A computer readable recording medium for storing a program that is executed by a computer, wherein said medium stores a program that executes the method according to  claim 30 .  
     
     
         55 . A computer readable recording medium for storing a program that is executed by a computer, wherein said medium stores a program that executes the method according to  claim 40.