2D/3D data understanding

smart & connected construction
automation & robotics
2D/3D data understanding
others


Construction progress estimation is an essential component of the daily construction cycle to ensure high productivity and quality. However, using laser-scanned point clouds for the purpose of measuring the deviation between the as-built structures and the as-planned Building Information Model (BIM) remains cumbersome due to the difficulty in data registration, segmentation, annotation and modeling in large-scale point clouds. This research proposes the use of a data-driven deep learning framework to automatically detect and classify building elements from a laser-scanned point cloud scene. The point cloud is first converted into a graph representation, where vertices represent points and edges represent connections between points within a fixed distance. An edge-based classifier is used to discard edges connecting points from different objects and form connected components from points in the same object. Next, a point-based object classifier is used to determine the type of building component based on the segmented points augmented with context from surrounding points. Finally, each detected object is matched with a corresponding BIM entity based on the nearest neighbor in feature space.


Point cloud instance segmentation and semantic segmentation results
segmentation1
3D scene reconstruction with CAD models
segmentation2




Journals:

  • Chen, J., Kira, Z., and Cho, Y. (2019). "Deep Learning Approach to Point Cloud Scene Understanding for Automated Scan to 3D Reconstruction." ASCE Journal of Computing in Civil Engineering, 33(4) DOI:10.1061/(ASCE)CP.1943-5487.0000842, [Full Text]

  • Chen, J., Cho, Y., and Kira, Z. (2019). "Multi-view Incremental Segmentation of 3D Point Clouds for Mobile Robots." IEEE Robotics and Automation Letters, 4(2), pp. 1240-1246,10.1109/LRA.2019.2894915, [Full Text]

  • Zeng, S., Chen, J., and Cho, Y.(2020). "User Exemplar-based Building Element Retrieval from Raw Point Clouds using Deep Point-level Features." Automation in Construction (in press),

Proceedings:

  • Chen, J., and Cho, Y. (2019). "Exemplar-based Building Element Retrieval from Point Clouds." International Conference on Smart Infrastructure and Construction (ICSIC), Churchill College, Cambridge, UK, July 8-9; DOI: 10.1680/icsic.64669.225 [Full text]

Recognizing construction assets (e.g., materials, equipment, labors) from point cloud data of construction environments provides essential information for engineering and management applications including progress monitoring, safety management, supply-chain management, and quality control. This study introduces a novel Principal Axes Descriptor (PAD) for construction equipment classification from point cloud data. Scattered as-is point clouds are first processed with down-sampling, segmentation and clustering steps to obtain individual instances of construction equipment. A geometric descriptor consisting of dimensional variation, occupancy distribution, shape profile, and plane counting features is then calculated to encode 3D characteristics of each equipment category. Using the derived features, machine learning methods such as K-Nearest Neighbors and Support Vector Machine are employed to determine class membership among major construction equipment categories such as backhoe loader, bulldozer, dump truck, excavator and front loader. Construction equipment classification with the proposed PAD was validated using CAD-generated point clouds as training data and laser scanned point clouds from an equipment yard as testing data. The recognition performance was further evaluated using point clouds from a construction site as well as a pose variation dataset. PAD was shown to achieve a higher recall rate and lower computation time compared to competing 3D descriptors. The results indicate that the proposed descriptor is a viable solution for construction equipment classification from point cloud data.


Segmentation and clustering for an excavator in a construction site point cloud
descriptor3
Equipment yard test site recognition results
descriptor2



Journals:

  • Chen, J., Fang, Y., and Cho, Y. (2018). "Performance Evaluation of 3D Descriptors for Object Recognition in Construction Applications. Automation in Construction, Volume 86,February 2018, Pages 44-52, DOI: 10.1016/j.autcon.2017.10.033

  • Chen, J., Fang, Y., Cho, Y., Kim, C. (2016). "Principal Axes Descriptor (PAD) for Automated Construction Equipment Classification from Point Clouds." ASCE's Journal of Computing in Civil Engineering, Volume 31, Issue 2, March 2017, doi.org/10.1061/(ASCE)CP.1943-5487.0000628 [Full text]

  • Cho, Y. and Gai. M. (2014). “Projection-Recognition-Projection Method for Automatic Object Recognition and Registration for Dynamic Heavy Equipment Operations.” a special issue of ASCE Journal of Computing in Civil Engineering, Volume 28, Issue 5, September 2014, (invited from 2012 ASCE International Workshop on Computing in Civil Engineering in Clearwater Beach, FL) DOI: 10.1061/(ASCE) CP.1943-5487.0000332 [Full text]
Proceedings:

  • Chen, J., Fang, Y., and Cho, Y. (2016). “Automated Equipment Recognition and Classification from Scattered Point Clouds for Construction Management. ” International Symposium on Automation and Robotics in Construction (ISARC), Auburn, AL, July 18-21, 2016,  DOI: 10.22260/ISARC2016/0027 [Full text]

Hybrid 3D Unstructured Workspace Modeling for Automated Construction Sites

Heavy construction equipment operation requires processing of thousands of range data in real-time or near real-time about the surrounding work environment, which is difficult to achieve with the current sensor technologies. Material handling such as steel beam erection and bolting requires not only rapid visualization of workspace but highly accurate position data for safe and secure physical contact between a target object and an end-effector. To meet these challenges, a target-focused range data collection and modeling method has been developed through a hybrid approach which integrates a 3D laser scanner with other optical sensors. The hybrid system will significantly reduce data collection time while enhancing data accuracy as well as promoting a target feature extraction process from the background range data. Automatic target detection and fitting and matching algorithms have been developed for rapid model-based graphical registration for various shapes of construction equipment and materials. Properly engineered, this new hybrid method for rapid 3D graphical recognition and registration of objects will become an important component for developing more intelligent and automated construction job sites by providing a fully sensed, monitored environment for materials, equipment, and operator.

Automated Target Recognition & Registration

Safe construction and operation of heavy construction equipment such as cranes, excavators, concrete pump trucks has been considered a very important subject in construction field. It would be helpful for the operators if the accurate 3D position of the target objects and surroundings are readily available. It has been a challenging subject to recognize target objects from a scattered work environment because large and complex 3D site data obtained by a laser scanner makes it difficult to process itself in real or near real time. In this study, A target is automatically recognized and tracked through a video camera. To reduce data size and scanning time, only recognized target in a box (i.e., kernel) will be scanned. Our hybrid system consistes of a CCD camera and laser scanner were used to rapidly recognize and register dynamic target objects in a 3D space by separating target object’s point cloud data from other background point cloud data for quick process.
Journals:

  • Wang, C and Cho, Y.( 2014). “Smart Scanning and Near Real-Time 3D Surface Modeling of Dynamic Construction Equipment from A Point Cloud.” Automation in Construction, Volume 49, Part B, January 2015, Pages 239-249, DOI: 10.1016/j.autcon.2014.06.003 (selected as best paper and invited from the 2013 30th International Symposium on Automation and Robotics in Construction (ISARC) in Montreal, Canada) [Fll text]

  • Cho, Y. and Gai. M. (2014). “Projection-Recognition-Projection Method for Automatic Object Recognition and Registration for Dynamic Heavy Equipment Operations.” a special issue of ASCE Journal of Computing in Civil Engineering, Volume 28, Issue 5, September 2014, (invited from 2012 ASCE International Workshop on Computing in Civil Engineering in Clearwater Beach, FL) DOI: 10.1061/(ASCE) CP.1943-5487.0000332 [Full text]

  • Cho, Y., Wang, C., Tang, P., and Haas, C.(2012). "Target-focused Local Workspace Modeling for Construction Automation Applications." ASCE Journal of Computing in Civil Engineering, Volume 26, Issue 5, September 2012, Pages 661-670, DOI: 10.1061/(ASCE)CP.1943-5487.0000166 [Fulltext]

Proceedings:

  • Wang, C., Cho, Y., and Park, J. (2014). “Performance Tests for Automatic Geometric Data Registration Technique for Construction Progress Monitoring.” International Conference on Computing in Civil and Building Engineering (ICCCBE), ASCE, Jun. 23-25, Orlando, FL.  pp. 1053-106, DOI: 10.1061/9780784413616.131 [Full text]

  • Cho, Y., Wang, C., Gai, M., and Park, J. (2014). “Rapid Dynamic Target Surface Modeling for Crane Operation Using Hybrid LADAR System” Construction Research Congress (CRC), ASCE, May 19-21, Atlanta, GA, DOI: 10.1061/9780784413517.108 [Full text]

  • Gai, M. and Cho, Y. (2013). “Automatic object recognition of dyanmic construction equipment from a 3D point cloud.” the 30th International Symposium on Automation and Robotics in Construction (ISARC), August 11-15, Montréal, Canada, DOI: 10.22260/ISARC2013/0061 [Full text]

  • Gai, M., Cho, Y., and Qinghua, X. (2013). “Real-time 3D visualization of multiple heavy construction equipment operations using LADAR.” Proceedings of the 2013 ASCE International Workshop on Computing in Civil Engineering, June 23-25, Univ. of Southern California, LA, California. pp.889-896, DOI: 10.1061/9780784413029.111 [Full text]

  • Gai, M., Cho, Y., and Qinghua, X. (2013). “Target-free automatic point clouds registration using 2D images.” Proceedings of the 2013 ASCE International Workshop on Computing in Civil Engineering, June 23-25, Univ. of Southern California, LA, California. pp.865-872, DOI: 10.1061/9780784413029.108 [Full text]

  • Gai, M., Cho, Y., and Wang, C. (2012). “Projection-Recognition-Projection (PRP) Method for Rapid Object Recognition from a 3D point cloud." Proceedings of the 2012 ASCE International Workshop on Computing in Civil Engineering, June 17-20, Clearwater, FL, pp. 325-332, DOI: 10.1061/9780784412343.0041 [Full text]

Existing buildings represent the greatest opportunity to improve building energy efficiency and reduce environmental impacts. However, millions of decision makers of the buildings usually lack sufficient information or tools for measuring their building's energy performance. The goal of Energy BIM project is to develop and evaluate rapid low-cost measurement and modeling approaches that will allow "virtual" representations for the energy and environmental performance of existing houses to be created for retrofit decision-support tools for the decision makers. This project developed a hybrid 3D light detection and ranging (LIDAR) system that can rapidly collect and fuse 3D point cloud and temperature data from existing buildings, from which we can automatically create an as-is conceptual building information model (BIM) from the 3D thermal point cloud and make it ready for energy simulation tools. The objective is to validate and evaluate building performance by comparing actual performance with the analysis results obtained from the detailed BIM. The created model consists of point clouds, in which each point contains coordinates, temperature, and other information.

Journals:

  • Zheng, K., Cho, Y., Wang, C., and Li, H. (2015)."A Non-invasive Residential Building Envelope R-value Measurement Method Based on Interfacial Thermal Resistance." ASCE Journal of Architectural Engineering, Volume 22, Issue 4, December 2016, DOI: 10.1061/(ASCE)AE.1943-5568.0000182, A4015002 [Full text]

  • Wang, C., Cho, Y., and Kim, C. (2015). "Automatic BIM Component Extraction from Point Clouds of Existing Buildings for Sustainability Applications." Automation in Construction, Volume 56, August 2015, Pages 1-13, DOI: 10.1016/j.autcon.2015.04.001 [Full text]

  • Cho, Y., Ham, Y., and Golpavar-Fard, M. (2015). “3D As-is Building Energy Modeling and Diagnostics: A Review of the State-of-the-Art.” Journal of Advanced Engineering Informatics, Volume 29, Issue 2, April 2015, Pages 184-195, DOI: 10.1016/j.aei.2015.03.004 [Full text]

  • Sharp, S., Cho, Y., and Li, Haorong (2014). " Residential Energy Recovery Radiant Heated Slab System (ERRHSS) for Melting Snow." ASCE Journal of Architectural Engineering, Volume 21, Issue 1, March 2015, DOI: 10.1061/(ASCE)AE.1943-5568.0000165, 04014003 [Full text]

  • Wang, C., Cho, Y., and Gai, M. (2013). “As-is 3D Thermal Modeling for Existing Buildings using a Hybrid LIDAR System.” a special issue of ASCE Journal of Computing in Civil Engineering, Volume 27, Issue 6, November 2013, Pages 646-656 (invited from 2011 ASCE International Workshop on Computing in Civil Engineering in Miami, FL) DOI: 10.1061/(ASCE)CP.1943-5487.0000273 [Full text]

Proceedings:

  • Cho, Y., Park, J., and Li, H. (2015). “A Framework for Cloud-based Energy Evaluation and Management for Sustainable Decision Support in the Built Environments”, 2015 International Conference on Sustainable Design, Engineering and Construction (ICSDEC). Chicago, May 10-13, 2015, DOI: 10.1016/j.proeng.2015.08.445 [Full text]

  • Zheng, K., Watt, J., and Cho, Y. (2015). “Building Central Plant System Performance Optimization through A Virtual Ambient Wet-bulb Temperature Sensor.” Proceedia Engineering, Vol 118. pp. 284-390, DOI: 10.1016/j.proeng.2015.08.438 [Full text]

  • Zheng, K., and Cho, Y. (2015). “A Convenient Thermal Performance Evaluating Method for Unitary AC (DX) Units.” Proceedia Engineering, Vol 118. pp. 391-403, DOI: 10.1016/j.proeng.2015.08.438 [Full text]

  • Wang, C. and Cho, Y. (2013). “Automated As-is BIM Extraction for Sustainable Simulation of Built Environments.” The Fifth International Conference on Construction Engineering and Project Management (ICCEPM-2013), Anaheim, CA, USA Jan. 9-11, 2013 [Link] [Full text]

  • Wang, C., and Cho, Y. (2012). “Automated 3D Building Envelope Recognition in Point Clouds.” ASCE, Proceedings of Construction Research Congress 2012, West Lafayette, IN, pp. 1155-1164, DOI: 10.1061/9780784412329.116 [Full text]

  • Im, H., Gai, M., Wang, C., and Cho, Y. (2012). “Hybrid Approach to Visualize Building Energy Information Model in Geospatial Application Programs.” ASCE, Proceedings of Construction Research Congress 2012, West Lafayette, IN, pp.1262-1270, DOI: 10.1061/9780784412329.127 [Full text]

  • Wang, C., Peng, Y., Cho, Y. and Li, H. (2011). “As-Built Residential Building Information Collection and Modeling Methods for Energy Analysis.” Proceedings of the 28th International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, June 29-July 2, pp.226-231. [Link] [Full text]

  • Cho, Y. and Wang, C. (2011). “3D Thermal Modeling for Existing Buildings Using Hybrid LIDAR System.” Proceedings of the 2011 ASCE International Workshop on Computing in Civil Engineering, Miami, DOI: 10.1061/41182(416)68 [Full text]

  • Wang, C., Cho, Y. (2011). “Non-invasive 3D Thermal Modeling for Buildings.” ASCE, International Conference on Sustainable Design and Construction in Kansas City, MO, DOI: 10.1061/41204(426)59 [Full text]

1. Automated Planning of Scaffolding in Building Information Modeling (BIM)
Considering their significant impact on construction projects, scaffolding as part of the temporary facilities category in construction must be thoroughly designed, planned, procured, and managed. The current practices in planning and managing scaffolding though is often manual and reactive, especially when a construction project is already underway. Widespread results are code compliance problems, inefficiency, and waste of procuring and managing material for scaffolding systems. We developed a rule-based system that automatically plans scaffolding systems for pro-active management in Building Information Modeling (BIM). The scope of the presented work is limited to traditional pipe and board scaffolding systems. A rule was prepared based on the current practice of planning and installing scaffolding systems. Our computational algorithms automatically recognize geometric and non-geometric conditions in building models and produce a scaffolding system design which a practitioner can use in the field. We implemented our automated scaffolding system for a commercially-available BIM software and tested it in a case study project. The system thoroughly identified the locations in need of scaffolding and generated the corresponding scaffolding design in BIM. Further results show, the proposed approach successfully generated a scaffolding system-loaded BIM model that can be utilized in communication, billing of materials, scheduling simulation, and as a benchmark for accurate field installation and performance measurement.

2. Automated Scaffolding-Related Safety Hazard Identification and Prevention in BIM
Construction remains as a hazardous industry that can expose construction workers to fatal accidents and illnesses. With recent advances in BIM technology, project information in BIM can be analyzed in the early design and planning stages to address potential safety issues. However, despite the impact on safety and productivity of the entire construction project, temporary structures, such as formwork and scaffolds, are often omitted from drawings or BIM. In practice, it is challenging to consider temporary structures in current manual jobsite safety analysis which is time-consuming and error-prone. As a result, in construction plans, potential safety hazards related to temporary structures are unknowingly created and they need to be identified and prevented during the construction phases. Focusing on scaffolds, this research integrates temporary structures into automated safety checking approach using BIM. A safety planning platform was created to simulate and visualize spatial movements of work crews using scaffolds. Computational algorithms in the platform identify safety hazards related to activities working on scaffolds and suggest preventive measures automatically. Then, the algorithms were implemented in a commercially available BIM software and validated with a real-world construction project. The results show that the algorithms could identify safety hazards that were not noticed by project managers participating in the case study project. The simulated results are visualized in the developed safety planning platform to facilitate early safety communications.


Journals:
  • Kim, K., and Cho, Y. (2015)."Construction-Specific Spatial Information Reasoning in Building Information Models " Journal of Advanced Engineering Informatics, Volume 29, Issue 4, October 2015, Pages 1013-1027, DOI: 10.1016/j.aei.2015.08.004 [Full text]
  • Kim, K., Walewski, J., and Cho, Y. (2015). “Multi-objective` Construction Schedule Optimization using Modified Niched Pareto Genetic Algorithm.” ASCE Journal of Management in Engineering, Volume 32, Issue 2, March 2016, DOI: 10.1061/(ASCE)ME.1943-5479.0000374 [Full tect]

Proceedings:
  • Kim, K., Cho, Y., and Kwak, Y. (2016). "BIM-Based Optimization of Scaffolding Plans for Safety." Construction Research Congress 2016: pp. 2709-2718, DOI: 10.1061/9780784479827.270 [Full text]
  • Kim, K., and Cho Y. (2015). “Automated Safety Planning of Scaffolding-Related Hazards in Building Information Modeling (BIM).” The 6th International Conference on Construction Engineering and Project Management (ICCEPM). Busan, Korea, Oct. 11-14, 2015, [Full text]
  • Kim, K. and Cho, Y., (2015). "BIM-based Planning Of Temporary Structures For Construction Safety." 2015 ASCE International Workshop on Computing in Civil Engineering, June 21-23, Austin, TX, DOI: 10.1061/9780784479247.054 [Full text]