Background replacement from video images captured by a plenoptic camera

Abstract

Embodiments disclosed herein provide systems, methods, and computer readable media for replacing a video background in real-time. The video comprises a plurality of image frames. In a particular embodiment, a method provides generating a range image from a subject image frame of the plurality of image frames. The range image indicates pixel distances from a plenoptic camera that captured the plurality of image frames. The method further provides identifying background pixels that represent a background portion of the subject image frame based on the range image and replacing the background pixels with replacement background pixels in the subject image frame.

What is claimed is: 
     
         1 . A method of replacing a video background in real-time, wherein the video comprises a plurality of image frames, the method comprising:
 receiving the video captured by a plenoptic camera of a first user system;   generating a range image from a subject image frame of the plurality of image frames, wherein the range image indicates pixel distances from the plenoptic camera;   identifying background pixels that represent a background portion of the subject image frame based on the range image;   replacing the background pixels with replacement background pixels in the subject image frame; and   after replacing the background pixels, transmitting the subject image frame to a second user system over a video communication session.   
     
     
         2 . The method of  claim 1 , wherein receiving the video comprises:
 receiving the video from the first user system.   
     
     
         3 . The method of  claim 1 , wherein receiving the video comprises:
 receiving the video from the plenoptic camera.   
     
     
         4 . The method of  claim 3 , wherein transmitting the subject image frame comprises:
 transmitting the video from the first user system to the second user system.   
     
     
         5 . The method of  claim 1 , wherein the first user system is associated with a contact center and the replacement background pixels represent an image associated with the contact center. 
     
     
         6 . The method of  claim 1 , wherein the subject image frame includes a user of the first user system. 
     
     
         7 . The method of  claim 1 , comprising:
 generating a second range image from a second subject image frame of the plurality of image frames, wherein the second range image indicates second pixel distances from the plenoptic camera;   identifying second background pixels that represent a second background portion of the second subject image frame based on the second range image;   replacing the second background pixels with the replacement background pixels in the second subject image frame; and   after replacing the second background pixels, transmitting the second subject image frame to the second user system over the video communication session.   
     
     
         8 . A method of replacing a video background, wherein the video comprises a plurality of image frames, the method comprising:
 receiving the video captured by a plenoptic camera;   generating a range image from a subject image frame of the plurality of image frames, wherein the range image indicates pixel distances from the plenoptic camera;   identifying background pixels that represent a background portion of the subject image frame based on the range image;   replacing the background pixels with replacement background pixels in the subject image frame; and   after replacing the background pixels, storing the video in a storage system.   
     
     
         9 . The method of  claim 8 , comprising:
 playing the video back to a user from the storage system.   
     
     
         10 . The method of  claim 8 , comprising:
 transmitting the video to a user system.   
     
     
         11 . The method of  claim 8 , wherein receiving the video comprises:
 receiving the video from a first user system.   
     
     
         12 . The method of  claim 8 , wherein receiving the video comprises:
 receiving the video from the plenoptic camera.   
     
     
         13 . The method of  claim 8 , wherein an object is represented by foreground pixels of the subject image frame. 
     
     
         14 . A method of replacing a video background, wherein the video comprises a plurality of image frames, the method comprising:
 generating a range image from a subject image frame of the plurality of image frames, wherein the range image indicates pixel distances from a plenoptic camera that captured the plurality of image frames; and   replacing background pixels in the subject image frame with replacement background pixels based on the range image.   
     
     
         15 . The method of  claim 14 , comprising:
 after replacing the background pixels, storing the video in a storage system.   
     
     
         16 . The method of  claim 15 , comprising:
 playing the video back to a user from the storage system.   
     
     
         17 . The method of  claim 14 , comprising:
 after replacing the background pixels, transmitting the video to a second user system over a video communication session.   
     
     
         18 . The method of  claim 14 , comprising:
 receiving the video from a first user system.   
     
     
         19 . The method of  claim 14 , comprising:
 receiving the video from the plenoptic camera.   
     
     
         20 . The method of  claim 14 , comprising:
 generating a second range image from a second subject image frame of the plurality of image frames, wherein the second range image indicates second pixel distances from the plenoptic camera; and   replacing the second background pixels with the replacement background pixels in the second subject image frame based on the second range image.