Building information model (bim) element extraction from floor plan drawings using machine learning

Abstract

A method and system provide a workflow for extracting building information model (BIM) elements for a floor plan drawing. A design drawing area of an image of the floor plan drawing is determined. Elements are extracted from the design drawing area. A synthetic floor plan design drawing dataset is obtained with known synthetic symbol labels and known synthetic symbol locations. Based on the extracted elements and the synthetic floor plan design drawing dataset, a symbol represented by the extracted elements is detected. Based on the symbol, a building information model (BIM) element is fetched and placed in the floor plan drawing.

What is claimed is: 
     
         1 . A computer-implemented method for extracting building information model (BIM) elements for a floor plan drawing, comprising:
 obtaining a design drawing area of an image of the floor plan drawing;   extracting elements from the design drawing area;   obtaining a synthetic floor plan design drawing dataset, wherein synthetic symbol labels and synthetic symbol locations of synthetic data in the synthetic floor plan design drawing dataset are known, and wherein the synthetic floor plan design drawing dataset is synthetic;   based on the extracted elements and the synthetic floor plan design drawing dataset, detecting a symbol represented by the extracted elements;   based on the symbol, fetching a building information model (BIM) element; and   placing the BIM element in the floor plan drawing.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the obtaining the image comprises:
 obtaining a portable document format (PDF) file of the floor plan drawing;   removing background gridlines with a rule-based method;   removing background elements using color information; and   rastering the PDF file into images.   
     
     
         3 . The computer-implemented method of  claim 1 , wherein the determining the design drawing area comprises:
 using a machine learning model to segment the image into multiple sections, wherein the machine learning model identifies fixed patterns of a layout of the floor plan drawing; and   selecting one or more of the multiple sections that comprise the design drawing area.   
     
     
         4 . The computer-implemented method of  claim 1 , wherein the extracting elements comprises:
 grouping the elements to form a candidate area of candidate elements; and   filtering out candidate elements based on a size of candidate element bounding boxes.   
     
     
         5 . The computer-implemented method of  claim 1 , wherein the obtaining a synthetic floor plan design drawing dataset comprises:
 programmatically generating the synthetic floor plan design drawing dataset based on a symbol library and synthetic floor plan dataset.   
     
     
         6 . The computer-implemented method of  claim 5 , further comprising:
 tiling the synthetic floor plan design drawing dataset into multiple tiles; and   processing each of the multiple tiles independently.   
     
     
         7 . The computer-implemented method of  claim 1 , further comprising determining an orientation of the symbol by:
 training a symbol orientation classification model;   predicting the orientation based on the symbol orientation classification model.   
     
     
         8 . The computer-implemented method of  claim 7 , wherein the training comprises:
 utilizing a known symbol orientation in the synthetic floor plan design drawing dataset for orientation information learning in the symbol orientation classification model.   
     
     
         9 . The computer-implemented method of  claim 1 , further comprising:
 presenting the floor plan drawing with the placed BIM element;   receiving user feedback; and   retraining an object detection model and a symbol orientation classification model based on the user feedback.   
     
     
         10 . The computer-implemented method of  claim 1 , further comprising:
 training a machine learning (ML) model using the synthetic floor plan design drawing dataset as a base;   detecting the symbol further based on the ML model; and   updating the ML model based on the BIM element in the floor plan drawing.   
     
     
         11 . A computer-implemented system for extracting building information model (BIM) elements for a floor plan drawing, comprising:
 (a) a computer having a memory;   (b) a processor executing on the computer;   (c) the memory storing a set of instructions, wherein the set of instructions, when executed by the processor cause the processor to perform operations comprising:
 (i) obtaining a design drawing area of an image of the floor plan drawing; 
 (ii) extracting elements from the design drawing area; 
 (iii) obtaining a synthetic floor plan design drawing dataset, wherein synthetic symbol labels and synthetic symbol locations of synthetic data in the synthetic floor plan design drawing dataset are known, and wherein the synthetic floor plan design drawing dataset is synthetic; 
 (iv) based on the extracted elements and the synthetic floor plan design drawing dataset, detecting a symbol represented by the extracted elements; 
 (v) based on the symbol, fetching a building information model (BIM) element; and 
 (vi) placing the BIM element in the floor plan drawing. 
   
     
     
         12 . The computer-implemented system of  claim 11 , wherein the obtaining the image comprises:
 obtaining a portable document format (PDF) file of the floor plan drawing;   removing background gridlines with a rule-based method;   removing background elements using color information; and   rastering the PDF file into images.   
     
     
         13 . The computer-implemented system of  claim 11 , wherein the determining the design drawing area comprises:
 using a machine learning model to segment the image into multiple sections, wherein the machine learning model identifies fixed patterns of a layout of the floor plan drawing; and   selecting one or more of the multiple sections that comprise the design drawing area.   
     
     
         14 . The computer-implemented system of  claim 11 , wherein the extracting elements comprises:
 grouping the elements to form a candidate area of candidate elements; and   filtering out candidate elements based on a size of candidate element bounding boxes.   
     
     
         15 . The computer-implemented system of  claim 11 , wherein the obtaining a synthetic floor plan design drawing dataset comprises:
 programmatically generating the synthetic floor plan design drawing dataset based on a symbol library and synthetic floor plan dataset.   
     
     
         16 . The computer-implemented system of  claim 15 , further comprising:
 tiling the synthetic floor plan design drawing dataset into multiple tiles; and   processing each of the multiple tiles independently.   
     
     
         17 . The computer-implemented system of  claim 11 , further comprising determining an orientation of the symbol by:
 training a symbol orientation classification model;   predicting the orientation based on the symbol orientation classification model.   
     
     
         18 . The computer-implemented system of  claim 17 , wherein the training comprises:
 utilizing a known symbol orientation in the synthetic floor plan design drawing dataset for orientation information learning in the symbol orientation classification model.   
     
     
         19 . The computer-implemented system of  claim 11 , further comprising:
 presenting the floor plan drawing with the placed BIM element;   receiving user feedback; and   retraining an object detection model and a symbol orientation classification model based on the user feedback.   
     
     
         20 . The computer-implemented system of  claim 11 , wherein the operations further comprise:
 training a machine learning (ML) model using the synthetic floor plan design drawing dataset as a base;   detecting the symbol further based on the ML model; and   updating the ML model based on the BIM element in the floor plan drawing.