ENQUIRE PROJECT DETAILS BY GENERAL PUBLIC

Project Details
Funding Scheme : General Research Fund
Project Number : 17201717
Project Title(English) : Generation of semantically rich as-built Building Information Models (BIMs): A derivative-free optimization approach 
Project Title(Chinese) : 語義豐富的建成建築信息模型（BIM）生成：一類無導數優化方法 
Principal Investigator(English) : Prof Xue, Fan 
Principal Investigator(Chinese) :  
Department : Department of Real Estate and Construction
Institution : The University of Hong Kong
E-mail Address : xuef@hku.hk 
Tel : 3917 4174 
Co - Investigator(s) :
Prof Lu, Weisheng
Panel : Engineering
Subject Area : Civil Engineering, Surveying, Building & Construction
Exercise Year : 2017 / 18
Fund Approved : 454,157
Project Status : Completed
Completion Date : 30-6-2021
Project Objectives :
To develop a general transformation to model as-built BIM generation as a nonlinear optimization problem of fitting BIM components;
To establish a semantic BIM component dataset to integrate open access BIM resources in various formats from various sources; and
To adopt and fine-tune state-of-the-art DFO algorithms for solving the nonlinear optimization problem to generate semantically rich as-built BIMs.
Abstract as per original application
(English/Chinese):
建成建築信息模型（BIM）表徵著建築物的實際狀況，在建築的全生命週期中有多種應用：如施工品質檢測、設施及應急管理、修繕建議、能耗分析及拆卸等。生成建成BIM首先需要測量建築物的實際情況，一些新型非接觸式測繪技術——如激光掃描及攝影測量學等——已可以自動收集大量數據。但是，從測量數據生成建成BIM ——尤其在考慮到諸如機械特性、熱力學或材質等非幾何語義信息時——依然十分具有挑戰性。 本研究旨在開發一類基於無導數優化（DFO）的、系統化及自動化的方法，用於生成語義豐富的建成BIM。該方法採用先進的DFO算法，選擇並擬合參數化BIM組件，從而生成與測量數據最為匹配的建成BIM。例如，我們獲獎的CMA-VNS（可變鄰域搜索的協方差矩陣適配）算法將作為研發基準。本研究的可行性已在一項先導研究中獲得證實。 本研究的成果將增強運用現代數學方法開發建成BIM的認知；亦可通過分析和理解相關最優化問題的適應性景觀，從理論上闡釋建成BIM生成過程的難點。在實用性方面，本研究將為建築工程施工/設施管理（AEC/FM）行業提供一種低成本、自動化及高復用性的建成BIM的生成方法，亦可能改變整體商業技術競爭格局。
Realisation of objectives: 5.1 Scope of investigation -- BIM is the information hub of buildings and city, but it is very challenging to reconstruct from 2D images and 3D point clouds. It is more challenging to have rich semantics at the same time. -- This project presents a brand-new "Semantic Registration" method for semantically rich BIM reconstruction. -- We apply the modern DFO (derivative-free optimization) algorithms in Applied Mathematics to realize a fully automatic semantically-rich as-built BIM creation. -- Furthermore, we looked into the integration of semantics and BIM for facilities and blockchains. 5.2 Objectives achieved -- Objective 1 achieved. The formulation was first presented in a journal paper [2] (as listed in Sect. 8, Section C) and furthered in [4] and [7]. [4] named it as a new method "Semantic Registration" to the research field. A pilot study (Conference [5]) extracted from the proposal won the Best Paper Award of CRIOCM2016. -- Objective 2 achieved. In [2] and [4], we have developed a systematic way of annotating, data converting, down-sampling and organizing BIM components for Semantic Registration. A pilot study of Objective 2 (Conference [4]) won the Special Award from LC3 Innovation Competition 2017, Heraklion, Greece. -- Objective 3 achieved. The most popular DFO algorithm, CMA-ES, was benchmarked and its parameters' sensitivities were analyzed in [2] and [4]. We further compared more DFO methods, of which the results were reported in journal papers [5] and [7] formally and invited lectures (Sect. 11 [2] and [3]) verbally. 5.3 Deviations from the original plan -- N.A. 5.4 Extensions and success beyond the original plan Apart from the successful execution of the project, we also looked into a few topics related to the project. -- Journal paper [3] extends the BIM semantics with real-time updates from the Internet of things (IoT). -- [5] extends the 3D point cloud data sources, and later triggered another GRF project (17200218, HK$ 0.5M) on architectural semantics (symmetry and parallelity) to point clouds without explicit BIM components. -- [9] extends the BIM semantics with blockchain, and later led to another GRF project (17200221, HK$ 0.8M) and an ITF project (ITP/029/20LP, HK$ 10M). -- The various optimization study for the expensive processes later triggered a Theme-based Research Scheme (T22-504/21-R, HK$34M) as Co-PI.
Summary of objectives addressed:
Objectives Addressed Percentage achieved
1.To develop a general transformation to model as-built BIM generation as a nonlinear optimization problem of fitting BIM componentsYes100%
2.To establish a semantic BIM component dataset to integrate open access BIM resources in various formats from various sourcesYes100%
3.To adopt and fine-tune state-of-the-art DFO algorithms for solving the nonlinear optimization problem to generate semantically rich as-built BIMsYes100%
Research Outcome
Major findings and research outcome: 6.1.1 Contributions to the body of knowledge -- A new BIM reconstruction methodology named “Semantic Registration,” which attracted >100 citations -- A typology and enrichment for BIM library studies -- Algorithmic experiments and analysis for expensive (time and HR cost) BIM reconstruction problem -- Extended BIM semantics studies with IoT and blockchain also attracted more attention and citations from academia and the industry. 6.1.2 Research outcomes: With citations, impact factors, and honors -- 8 journal papers and 1 book chapter [1] Citation = 55 (as of Jun 2022) Journal impact factor = 7.700 (JCR 2020) [2] Citation = 47 Journal impact factor = 11.775 [3] Citation = 30 Journal impact factor = 7.700 [4] Citation = 28 Journal impact factor = 4.64 [5] Citation = 19 Journal impact factor = 8.979 [6] Citation = 6 (Book publisher Springer) [7] Citation = 31 Journal impact factor = 5.603 [8] Citation = 8 Journal impact factor = 1.97 [9] Citation = 78 Journal impact factor = 7.700 -- 3 International research awards received: Best Paper Award [5], Sect. 11, Part C, awarded in 2016 Merit Paper Award [6], Sect. 11, Part C, awarded in 2020 Special Award, [4], Sect. 11, Part C, awarded in 2017 -- 5 International conference papers -- 3 invited lectures for SZU, HUST, and CAUC between 2018 and 2019.
Potential for further development of the research
and the proposed course of action:		 The findings of the project have successfully enabled 4 more projects funded by RGC or ITC: -- A GRF project (17200218) on architectural semantics (symmetry and parallelity) -- A GRF project (17200221) on theoretical integration blockchain with BIM semantics -- An ITF project (ITP/029/20LP, HK$ 10M) on applied blockchain BIM in construction. -- A TRS project (T22-504/21-R, HK$34M) on generative design facilitated by optimization. In addition, the concept, methodology, cases, and findings have been taught in a set of BSc and MSc courses (RECO2041, RECO3032, RECO7097, RECO7613) at the University of Hong Kong. BIM semantics hosts and offers vital decision-support information for the industry, and expensive optimization is critical for many frontier sciences and engineering problems. We are positive to see more further impacts from the findings of this project.
Layman's Summary of
Completion Report:		 BIM is the information hub of buildings and city, but it is very challenging to reconstruct from 2D images and 3D point clouds. It is more challenging to have rich semantics at the same time. This project presents a brand-new "Semantic Registration" methodology for semantically-rich BIM reconstruction. The challenging BIM reconstruction is defined as a nonlinear optimization problem first; then, semantically-rich BIM components (e.g., Revit families and SketchUp Warehouse 3D parts) can be collected freely online. We apply the modern DFO (derivative-free optimization) algorithms in Applied Mathematics to realize a fully automatic solving of the nonlinear optimization, which is visualized as a "trial-and-error" style automated creation of semantically-rich as-built BIM. Besides, we found that semantically-rich BIMs can be well integrated into smart facilities and blockchains; while DFO is also applicable to other expensive sciences and engineering problems, like generative designs of outdoor thermal comfort zones.
Research Output
Peer-reviewed journal publication(s)
arising directly from this research project :
(* denotes the corresponding author)
Year of
Publication		 Author(s) Title and Journal/Book Accessible from Institution Repository
2018 Chen, K., Lu, W.*, Xue, F., Tang, P., & Li, L. H.  Automatic building information model reconstruction in high-density urban areas: Augmenting multi-source data with architectural knowledge. Automation in Construction, 93, 22-34. Doi: 10.1016/j.autcon.2018.05.009  No 
2018 Xue, F., Lu, W.*, & Chen, K.  Automatic generation of semantically rich as-built building information models using 2D images: A derivative-free optimization approach. Computer‐Aided Civil and Infrastructure Engineering. 33(11), 926-942. Doi: 10.1111/mice.12378  No 
2018 Xue, F., Chen, K.*, Lu, W., Niu, Y., & Huang, G. Q.  Linking radio-frequency identification to Building Information Modeling: Status quo, development trajectory and guidelines for practitioners  No 
2019 Xue, F., Lu, W.*, Chen, K., & Zetkulic, A.  From semantic segmentation to semantic registration: Derivative-free optimization-based approach for automatic generation of semantically rich as-built building information models from 3D point clouds. Journal of Computing in Civil Engineering, 33(4): 04019024. Doi: 10.1061/(ASCE)CP.1943-5487.0000839  No 
2019 Xue, F., Lu, W.*, Webster, C. J., & Chen, K.  A derivative-free optimization-based approach for detecting architectural symmetries from 3D point clouds  No 
2019 Xue, F.*, Chen, K., & Lu, W.  Architectural Symmetry Detection from 3D Urban Point Clouds: A Derivative-Free Optimization (DFO) Approach  No 
2019 Fan Xue, Weisheng Lu*, Ke Chen, Chris Webster  BIM reconstruction from 3D point clouds: A semantic registration approach based on multimodal optimization and architectural design knowledge. Advanced engineering informatics, 42, 100965.  No 
2020 Jinying Xu*, Ke Chen, Anna Elizabeth Zetkulic, Fan Xue, Weisheng Lu, and Yuhan Niu  Pervasive sensing technologies for facility management: A critical review  No 
2020 F Xue, W Lu*  A semantic differential transaction approach to minimizing information redundancy for BIM and blockchain integration  No 
Recognized international conference(s)
in which paper(s) related to this research
project was/were delivered :
Month/Year/City Title Conference Name
Chongqing, China Personalized Walkability Assessment for Pedestrian Paths: An As-built BIM Approach Using Ubiquitous Augmented Reality (AR) Smartphone and Deep Transfer Learning  The 23rd International Symposium on the Advancement of Construction Management and Real Estate 
Hong Kong SAR Understanding unstructured 3D point clouds for creating digital twin city: An unsupervised hierarchical clustering approach  CIB World Building Congress 2019 
Wuhan, China Construction inspection information management with consortium blockchain  25th International Symposium on Advancement of Construction Management and Real Estate (CRIOCM2020) 
Heraklion, Crete, Greece Developing an Open Access BIM Objects Library: A Hong Kong Study  Lean and Computing in Construction Congress 
Hong Kong, China An optimization-based semantic building model generation method with a pilot case of a demolished construction  the 21st International Symposium on Advancement of Construction Management and Real Estate, Hong Kong, China 
Other impact
(e.g. award of patents or prizes,
collaboration with other research institutions,
technology transfer, etc.):
  SCREEN ID: SCRRM00542