Segmentation and prediction of low-level temporal plume patterns

Abstract

Computer vision based systems, and methods are provided for generating plume analysis data from a plurality of input image frames as captured by a camera. Image data associated with a plume of gas is received and provided as inputs to a predictive model used to determine plume analysis data including a plume prediction segmentation mask associated with the plume of gas. Other attributes of the plume are also estimated from the prediction segmentation mask such as cross-sectional area, cross-sectional velocity, leak source pixel and volumetric leak rates. The plume analysis data can be provided as an overlay atop the image data. The plume analysis data overlaid atop the image data can be transmitted by and/or to one or more computing devices.

What is claimed is: 
     
         1 . A method comprising:
 receiving image data associated with a plume of gas, the image data including a plurality of image frames;   determining plume analysis data using the received image data and a first predictive model trained to receive image data and, in response to the receiving, generate plume analysis data associated with the plume of gas, the plume analysis data including a plume prediction segmentation mask associated with the plume of gas and predicted in relation to a sequence of image frames included in the image data;   determining, for the plume of gas, at least one of a leak size estimation, a leak source estimation, and a volumetric leak rate estimation;   providing the leak size estimation, the leak source estimation, the volumetric leak rate estimation, and the plume analysis data, wherein the plume analysis data is provided as an overlay atop the image data; and   transmitting the plume analysis data overlaid atop the image data, wherein at least one of the receiving, the determining, the providing, and the transmitting is performed by at least one data processor forming part of at least one computing system.   
     
     
         2 . The method of  claim 1 , wherein the plume of gas includes a plume of smoke, a plume of steam, a plume of gas, or plume of a gas mixture. 
     
     
         3 . The method of  claim 1 , wherein the image data is acquired by a camera configured to capture each image frame using infrared light or visible light. 
     
     
         4 . The method of  claim 3 , wherein the plume prediction segmentation mask includes a pixel-wise segmentation delineating a boundary of the plume of gas as directly observed by the camera in the sequence of image frames. 
     
     
         5 . The method of  claim 3 , wherein the leak size estimation includes a pixel-wise classification of pre-determined leak sizes delineating a boundary of the plume of gas as predicted based on the sequence of image frames. 
     
     
         6 . The method of  claim 3 , wherein the camera is configured to acquire each image frame from a fixed location or from a mobile platform. 
     
     
         7 . The method of  claim 6 , wherein the mobile platform includes a manned ground vehicle, an unmanned ground vehicle, a manned aerial vehicle, an unmanned aerial vehicle, a manned surface vehicle, an unmanned surface vehicle, a manned underwater vehicle, an unmanned underwater vehicle, a robot, or a mobile platform attached to a human in motion. 
     
     
         8 . The method of  claim 4 , wherein determining the leak source estimation further comprises determining one or more pixels of the plume in one or more frames of the sequence of image frames based on a velocity estimation of a flow of particles within a region of the plume. 
     
     
         9 . The method of  claim 8 , wherein determining the volumetric leak rate further comprises
 determining a cross-sectional area of the plume in one or more frames of the sequence of image frames; and   determining the velocity estimation of the flow of particles as they cross the cross-sectional area.   
     
     
         10 . The method of  claim 1 , further comprising,
 determining plume analysis data using the received image data and a second predictive model trained to receive image data; and   generate, in response to the receiving, the plume analysis data associated with the plume of gas using the second predictive model, wherein the plume analysis data includes a plume prediction segmentation mask associated with the plume of gas and predicted in relation to each image frame in the sequence of image frames included in the image data.   
     
     
         11 . A system comprising:
 a first computing device, including a data processor and a memory storing computer-readable instructions and a plurality of prediction models, the processor configured to execute the computer-readable instructions, which when executed, cause the processor to perform operations including
 receiving image data associated with a plume of gas, the image data including a plurality of image frames, 
 determining plume analysis data using the received image data and a first predictive model trained to receive image data and, in response to the receiving, generate plume analysis data associated with the plume of gas, the plume analysis data including a plume prediction segmentation mask associated with the plume of gas and predicted in relation to a sequence of image frames included in the image data; 
 determining, for the plume of gas, at least one of a leak size estimation, a leak source estimation, and a volumetric leak rate estimation; 
   providing the leak size estimation, the leak source estimation, the volumetric leak rate estimation as calculated attributes, and the plume analysis data, wherein the plume analysis data is provided as an overlay atop the image data, and   transmitting the plume analysis data overlaid atop the image data; and   a second computing device coupled to the first computing device via a network, the second computing device including a display configured to present the transmitted plume analysis data via the display.   
     
     
         12 . The system of  claim 11 , wherein the plume of gas includes a plume of smoke, a plume of steam, a plume of gas, or a plume of a gas mixture. 
     
     
         13 . The system of  claim 11 , further comprising a camera configured to capture each image frame using infrared light or visible light. 
     
     
         14 . The system of  claim 13 , wherein the plume prediction segmentation mask includes a pixel-wise segmentation delineating a boundary of the plume of gas as directly observed by the camera in the sequence of image frames. 
     
     
         15 . The system of  claim 13 , wherein a leak size estimation includes a pixel-wise classification of pre-determined leak sizes delineating a boundary of the plume of gas as predicted based on the sequence of image frames. 
     
     
         16 . The system of  claim 13 , wherein the camera is configured to acquire each image frame from a fixed location or from a mobile platform. 
     
     
         17 . The system of  claim 16 , wherein the mobile platform includes a manned ground vehicle, an unmanned ground vehicle, a manned aerial vehicle, an unmanned aerial vehicle, a manned surface vehicle, an unmanned surface vehicle, a manned underwater vehicle, an unmanned underwater vehicle, a robot, or a mobile platform attached to a human in motion. 
     
     
         18 . The system of  claim 14 , wherein the computer-readable instructions further cause the processor to determine the leak source estimation by determining one or more pixels of the plume in one or more frames of the sequence of image frames based on a velocity estimation of a flow of particles within a region of the plume. 
     
     
         19 . The system of  claim 18 , wherein the computer-readable instructions further cause the processor to determine the volumetric leak rate by
 determining a cross-sectional area of the plume in one or more frames of the sequence of image frames; and   determining the velocity estimation of the flow of particles as they cross the cross-sectional area.   
     
     
         20 . The system of  claim 11 , wherein the computer-readable instructions further cause the processor to
 determine plume analysis data using the received image data and a second predictive model trained to receive image data; and   generate, in response to the receiving, the plume analysis data associated with the plume of gas using the second predictive model, wherein the plume analysis data includes a plume prediction segmentation mask associated with the plume of gas and predicted in relation to each image frame in the sequence of image frames included in the image data.