Perspective-correct communication window with motion parallax

Abstract

A perspective-correct communication window system and method for communicating between participants in an online meeting, where the participants are not in the same physical locations. Embodiments of the system and method provide an in-person communications experience by changing virtual viewpoint for the participants when they are viewing the online meeting. The participant sees a different perspective displayed on a monitor based on the location of the participant's eyes. Embodiments of the system and method include a capture and creation component that is used to capture visual data about each participant and create a realistic geometric proxy from the data. A scene geometry component is used to create a virtual scene geometry that mimics the arrangement of an in-person meeting. A virtual viewpoint component displays the changing virtual viewpoint to the viewer and can add perceived depth using motion parallax.

What is claimed is: 
     
         1 . A method for communicating with a remote participant in a meeting, comprising:
 creating a geometric proxy for each participant in the meeting including a local participant and the remote participant, where the local participant and the remote participant are in different physical locations;   rendering the geometric proxy for each participant in a scene geometry that is consistent with an in-person conversation;   transmitting each rendered geometric proxy and the scene geometry to each of participant;   displaying the geometric proxy for the remote participant and an associated background to the local participant; and   changing a virtual viewpoint of the geometric proxy for the remote participant and the associated background based on an orientation of the local participant's face.   
     
     
         2 . The method of  claim 1 , further comprising using a face tracking technique to track the orientation of the local participant's face. 
     
     
         3 . The method of  claim 2 , further comprising keeping the virtual viewpoint level with an eye gaze of the local participant using the face tracking technique. 
     
     
         4 . The method of  claim 1 , further comprising capturing images of each of the participants in the meeting using a plurality of camera pods. 
     
     
         5 . The method of  claim 4 , further comprising creating the virtual viewpoint using a virtual camera that is a composition of images from at least two of the plurality of camera pods. 
     
     
         6 . The method of  claim 4 , further comprising capturing RGB data and depth information using the plurality of camera pods. 
     
     
         7 . The method of  claim 6 , further comprising creating the geometric proxy for each participant by adding the RGB data to the depth information. 
     
     
         8 . The method of  claim 1 , further comprising:
 determining a number of participants in the meeting; and   generating the scene geometry based on the number of participants to simulate an in-person conversation between the participants.   
     
     
         9 . The method of  claim 8 , further comprising:
 using virtual boxes to ensure that an eye gaze and conversational geometry between the participants is correct and that to other participants the conversational geometry looks correct so the local participant can see correctly the other participants; and   determining a number of virtual boxes to use based on the number of participants.   
     
     
         10 . The method of  claim 9 , further comprising:
 determining that there are two participants in the meeting; and   creating a first virtual box and a second virtual box that are facing each other.   
     
     
         11 . The method of  claim 9 , further comprising:
 determining that there are three participants in the meeting; and   creating a first virtual box, a second virtual box, and a third virtual box that are place around a virtual round table in an equidistant manner.   
     
     
         12 . The method of  claim 1 , further comprising adding an illusion of depth to the virtual viewpoint using motion parallax. 
     
     
         13 . The method of  claim 12 , further comprising using a face tracking technique to shift the virtual viewpoint as the local participant's head moves. 
     
     
         14 . A method for changing a virtual viewpoint of a local participant in an online conference, comprising:
 capturing images of the local participant to obtain captured information;   creating a geometric proxy of the local participant using the captured information;   determining a number of participants in the online conference;   generating scene geometry based on the number of participants to simulate being in an in-person conversation with other participants in the online conference;   rendering the geometric proxy to geometric proxies of the other participants in the scene geometry;   transmitting the rendered geometric proxies and scene geometry to the local participant;   displaying on a monitor to the local participant a rendered geometric proxy of a remote participant that is in a different physical location than the local participant; and   changing the virtual viewpoint of the local participant based on a position of the local participant's face.   
     
     
         15 . The method of  claim 14 , further comprising adding depth to the virtual viewpoint using motion parallax. 
     
     
         16 . The method of  claim 14 , further comprising generating the virtual viewpoint using a virtual camera that is the composition of images captured by a plurality of camera pods each having a different view of a scene. 
     
     
         17 . The method of  claim 16 , further comprising:
 tracking the position of the local participant's face using a face tracking technique; and   keeping the local participant facing at least some of the other participants using the position of the local participant's face to ensure that the local participant and at least one of the other participants are always looking straight at each other.   
     
     
         18 . A method for creating an in-person communication experience between participants in an online meeting, comprising:
 arranging a plurality of camera pods around a monitor that a local participant is viewing;   capturing RGB data and depth information of the local participant from the plurality of camera pods;   creating a geometric proxy for the local participant by adding the RGB and the depth information;   generating scene geometry based on a number of participants in the online meeting, including the local participant;   rendering the geometric proxy for the local participant and geometric proxies for each of the other participants in the online meeting to each other in the scene geometry so that the scene geometry is consistent with an in-person conversation;   transmitting the rendered geometric proxies and scene geometry to each of the participants;   displaying a virtual viewpoint to the local participant that includes a rendered geometric proxy for each of the participants along with a background that is part of the scene geometry;   changing the virtual viewpoint based on a position of the local participant's face such that the local participant's view of other participants and the background is dependent on an orientation of the local participant's face; and   adding an illusion of depth to the virtual viewpoint using motion parallax to create the in-person communication experience between the participants in the online meeting.   
     
     
         19 . The method of  claim 18 , further comprising:
 having the local participant and another participant at the same physical location and viewing the monitor, where the monitor has a lenticular display that allows multiple viewing angles; and   having the local participant view the monitor from the right side to obtain a right field-of-view and having the other participant view the monitor from the left side to obtain a left field-of-view, the right field-of-view and the left field-of-view being different.   
     
     
         20 . The method of  claim 18 , further comprising:
 having the local participant and another participant at the same physical location and viewing the monitor;   using a face tracking technique to track the local participant and the other participant; and   providing different views on the monitor to the local participant and the other participant based on an orientation of their faces.