ENQUIRE PROJECT DETAILS BY GENERAL PUBLIC

Project Details
Funding Scheme : General Research Fund
Project Number : 17200218
Project Title(English) : A derivative-free optimization (DFO) approach to architectural symmetry detection from 3D point clouds  
Project Title(Chinese) : 三維點雲中建築對稱性識別的一類無導數優化(DFO)方法 
Principal Investigator(English) : Prof Xue, Fan 
Principal Investigator(Chinese) :  
Department : Department of Real Estate and Construction
Institution : The University of Hong Kong
E-mail Address : xuef@hku.hk 
Tel : 3917 4174 
Co - Investigator(s) :
Mr Chiaradia, Alain Joseph Franck
Prof Lu, Weisheng
Panel : Engineering
Subject Area : Civil Engineering, Surveying, Building & Construction
Exercise Year : 2018 / 19
Fund Approved : 522,846
Project Status : Completed
Completion Date : 18-7-2021
Project Objectives :
To define a general formulation to model architectural symmetry detection from 3D point clouds as a nonlinear optimization problem;
To apply and revise state-of-the-art DFO algorithms for solving the formulated nonlinear optimization problem; and
To integrate and demonstrate the proposed approach to form a generalizable methodology.
Abstract as per original application
(English/Chinese):		 Symmetry is fundamental in nature, science, engineering, and art. Perception and recognition of symmetries influence not only our understanding of the world but also our design processes. Symmetry is also ubiquitous and continuous in architecture, appearing in all places over the world and at all times throughout history, for example, the Great Wall of China, the Parthenon of Greece, the Taj Mahal of India, ‘The Gherkin’ in London, and the Sydney Opera House in Australia. The ubiquity of symmetry in architecture is far from accidental, rather it is the result of considerations relating to function, economics, mechanics, manufacturing, and aesthetics. Recent advances in sensing technology, such as laser scanning and photogrammetry, provide increasingly available 3D point clouds of architectures and cities. Detection of architectural symmetries from 3D point clouds is not only an intriguing inquiry in its own right, but also an effective and essential step in creating accurate and informative digital building/city models for various applications such as architectural design, computer vision, construction management, heritage conservation, and smart and resilient city development. However, as it is too time-consuming, tedious, and costly to manually recognize and segment architectural symmetries from 3D point clouds, the challenge is to devise an automated detection of architectural symmetries. To meet this challenge, the proposed research project aims to apply modern derivative-free optimization (DFO) algorithms, which have been successfully used to solve many science and engineering problems, to automated architectural symmetry detection from 3D point clouds. Extending the mathematical definition of symmetry, this research proposes a general formulation of detecting symmetry in architecture. The major innovation here is an elegant means of detecting all types of architectural symmetries, including reflection, translation, rotation, uniform scaling, or combinations thereof, as a unified nonlinear optimization problem which is solvable by modern DFO methods. Our recent pilot study, involving a dense cloud of over one million points from a neoclassical building, has confirmed the technological feasibility of this proposed DFO approach. The project offers both academic insights and practical benefits. The findings will extend knowledge of how modern mathematical methods can be used to discover symmetry in architecture or related areas. The outcomes may lead to a breakthrough in theoretical explanation of the challenges in architectural symmetry detection. Practically, the research will offer a useful methodological tool for the creation of accurate and informative building/city models from inexpensive point clouds for industries related to architecture, construction, engineering, automobiles, and robotics.
對稱性普遍存在於世界各地各時期的建築中，例如長城、帕特農神廟、泰姬陵、倫敦“小黃瓜”及悉尼歌劇院。無處不在的對稱性并非意外使然，而是綜合功能性、經濟性、結構、可生產性及美觀性的全面考量的結果。目前，測量建築物及城市的三維點雲已變得日益準確、全面及廉價。三維點雲中的對稱性識別不僅是一種知識獲取，更是創建智慧城市所必須的建築和城市信息模型的必要步驟。然而，人工識別費時費力且成本高昂；因此，需要研究自動化對稱性識別技術。 本項目將運用最新的無導數優化（DFO）算法，自動化三維點雲中的建築對稱性識別過程。在本項目中，不論是鏡像、平移、旋轉、等比縮放或是它們的各種組合，都會被統一地形式化為一類非綫性優化問題。進而采用無導數優化算法來求解此類問題。本項目兼具學術及實踐意義。預期結果料將拓展運用當代數學方法發現建築及相關領域的對稱性的研究，亦可識別建築對稱性識別的理論難點。在實踐影響力方面，本研究將為建築和城市信息模型提供一套有效的方法論工具。
Realisation of objectives: 5.1 Scope of investigation -- The 3D point cloud is a popular means of measuring the 3D geometry of a city. Yet, it is challenging to detect the architectural symmetries within millions or billions of points for understanding the data and creating meaningful city models. -- This project presents a novel optimization-based method to automate architectural symmetry detection in urban point clouds. -- We apply the modern DFO (derivative-free optimization) algorithms in Applied Mathematics to realize a fully automatic point cloud processing. -- Furthermore, we looked into the Gestalt design laws for systematic symmetries and integration of semantics and BIM for urban facilities. 5.2 Objectives achieved -- Objective 1 achieved. The formulation of automatic symmetry detection in point clouds was first presented as ODAS (Optimization-based Detection of Architectural Symmetries) in journal paper [1] (as listed in Sect. 8, Section C) and furthered in [2] and [3]. The paper [1] won HKU Research Output Prize 2018/19. -- Objective 2 achieved. In journal paper [1], the experimental results showed that our ODAS method was 1,000 times faster – while with less error – than conventional voting methods. The ODAS was also 10 times faster and more accurate than the preliminary work in our 2019 conference paper [1]. -- Objective 3 achieved. The ODAS is open-sourced for worldwide researchers; freely available at: https://github.com/ffxue/odas. We further investigated the Gestalt law-based systematic symmetries, presented in a journal paper [3]. The paper [3] won the 2020 Featured Article of ISPRS Journal of Photogrammetry and Remote Sensing (JIF=11.774, ranked 1/48 in Physical Geography). 5.3 Deviations from the original plan -- N.A. 5.4 Extensions and success beyond the original plan Apart from the successful execution of the project, we also looked into a few topics related to the project. -- Conference paper [3] extends the symmetry detection to many buildings up to an urban scale, for semantically rich city information models. The paper [3] won the Best Paper Award from ICSBS 2019. -- The research findings led to a Key R&D Program of the Guangdong Province (2019B010151001) (Co-PI, 5% of CNY 10 million), and an HKU Small Equipment Grant (PI, HK$760,000).
Summary of objectives addressed:
Objectives Addressed Percentage achieved
1.To define a general formulation to model architectural symmetry detection from 3D point clouds as a nonlinear optimization problem;Yes100%
2.To apply and revise state-of-the-art DFO algorithms for solving the formulated nonlinear optimization problem; andYes100%
3.To integrate and demonstrate the proposed approach to form a generalizable methodology.Yes100%
N.A.
Research Outcome
Major findings and research outcome: 6.1.1 Contributions to the body of knowledge -- A new architectural symmetry detection methodology named ODAS (Optimization--based Detection of Architectural Symmetries), which attracted >70 citations -- Algorithmic experiments and sensitivity analysis for various (heritage, modern, and infrastructure in western countries and Hong Kong) real--world architectures -- Extended BIM semantics studies with city information models and digital twin [4] also attracted more attention and citations from academia and industry. 6.1.2 Research outcomes: With citations, impact factors, and honors - 4 journal papers and 1 book chapter -- [1] Citation = 20 (as of Jul 2022) Journal impact factor = 11.774 (JCR 2021) -- [2] Citation = 34 Journal impact factor = 7.862 -- [3] Citation = 41 Journal impact factor = 11.774 -- [4] Citation = 12 Journal impact factor = 5.349 -- [5] Citation = 1 (Book publisher Springer) - 5 International research awards received: -- Best Paper Award, Conference paper [3], awarded in 2019 -- Outstanding Paper Award, Conference paper [6], awarded in 2020 -- HKU Research Output Prize, Journal paper [1], awarded in 2019 -- 2020 Featured Article, Journal paper [3], awarded in 2020 -- Merit Award of Research, Journal paper [4], awarded in 2022 - 3 International conference papers - 5 invited lectures for SZU, HUST, etc. between 2018 and 2020.
Potential for further development of the research
and the proposed course of action:		 The findings of the project have successfully enabled one more ECS project, a Guangdong R&D project, an equipment grant, and a TRS project: -- ECS project (27200520) on architectonic grammar optimization for construction point clouds -- Key R&D Program of the Guangdong Province (2019B010151001) (Co-PI, 5% of CNY 10 million) -- HKU Small Equipment Grant (PI, HK$760,000). -- TRS (T22-504/21-R, HK$34M) on generative design facilitated by optimization. In addition, the concept, methodology, cases, and findings have been taught in a set of BSc and MSc courses (RECO2041, RECO3032, RECO7097, RECO7613) at the University of Hong Kong. Architectural symmetries are vital for the construction and urban models. We are positive to see further impacts from the findings of this project.
Layman's Summary of
Completion Report:		 The 3D point cloud is a popular means of measuring the 3D geometry of a city. Yet, it is challenging to detect the architectural symmetries within millions or billions of points for understanding the data and creating meaningful city models. This project presents a novel ODAS (Optimization-based Detection of Architectural Symmetries) method to automate architectural symmetry detection in urban point clouds. Our experiments showed that the modern algorithms in Applied Mathematics can realize a fully automatic point cloud processing, while the speed is way much faster and more accurate than conventional point-wise voting. Furthermore, we presented that Gestalt design laws can be detected for systematic symmetries and integration of semantics and BIM for symmetric urban objects and facilities. The novel methods were recognized by five awards and honors from top journals, international conferences, and competitions.
Research Output
Peer-reviewed journal publication(s)
arising directly from this research project :
(* denotes the corresponding author)
Year of
Publication		 Author(s) Title and Journal/Book Accessible from Institution Repository
2019 Xue, F., Lu, W.*, Webster, C. J., Chen, K.  A derivative-free optimization-based approach for detecting architectural symmetries from 3D point clouds  No 
2019 Xue, F., Lu, W.*, Chen, K., Webster, C. J.  BIM reconstruction from 3D point clouds: A semantic registration approach based on multimodal optimization and architectural design knowledge  No 
2020 Fan Xue*, Weisheng Lu, Zhe Chen, and Christopher J. Webster  From LiDAR point cloud towards digital twin city: Clustering city objects based on Gestalt principles [Journal's 2020 Featured Article]  No 
2021 Yijie Wu, Jianga Shang, Fan Xue*  RegARD: Symmetry-Based Coarse Registration of Smartphone’s Colorful Point Clouds with CAD Drawings for Low-Cost Digital Twin Buildings  No 
2021 Maosu Li*, Fan Xue, Anthony G. O. Yeh, Weisheng Lu  Classification of photo-realistic 3D window views in a high-density city: The case of Hong Kong  No 
Recognized international conference(s)
in which paper(s) related to this research
project was/were delivered :
Month/Year/City Title Conference Name
Hong Kong Understanding unstructured 3D point clouds for creating digital twin city: An unsupervised hierarchical clustering approach  CIB World Building Congress 2019 
Dalian, China Semantic Enrichment for Rooftop Modeling using Aerial LiDAR Reflectance  2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC 2019) 
Suzhou, China Semantic enrichment of city information models with LiDAR-based rooftop albedo (Best Paper Award)  The 2nd International Conference on Sustainable Buildings and Structures (ICSBS 2019) 
Other impact
(e.g. award of patents or prizes,
collaboration with other research institutions,
technology transfer, etc.):
  SCREEN ID: SCRRM00542