Automation & Robotics

smart & connected construction
automation & robotics
2D/3D data understanding
others




The main objective of this research is to define and examine an advanced human-multi-robot cooperative disaster relief framework which can rapidly assist human crisis intervention and search and rescue (SAR) teams, with an emphasis on nuclear power plants. The teams at Georgia Tech and Hanyang University in South Korea are formed to collaboratively resolve this global challenge. The research will provide a unique but generally acceptable and scalable framework which can rapidly analyze the risk levels and priority to efficiently manage and retrofit local heavy equipment to unmanned robots to operate collaborative semi-autonomous disaster relief field robots at a damaged nuclear plant site. The research will identify and develop semantically enriched as-built damaged site visualization algorithms derived from scattered point clouds fused with heterogeneous data such as thermal, radiation, RGB, and geo-tagged data. This research is carried out in collaboration with the CIM and Robotics Laboratory and Sustainable Building and Construction Management Lab from Hanyang University ERICA.


Feedback loop for teleoperation with 3D workspace modelling
loop
Visualization and pose estimation of excavator workspace
pose_estimation




Journals:

  • Chen, J., Kira, Z., and Cho, Y. (2019). "Deep Learning Approach to Point Cloud Scene Understanding for Automated Scan to 3D Reconstruction." ASCE Journal of Computing in Civil Engineering, 33(4) DOI:10.1061/(ASCE)CP.1943-5487.0000842, [Full Text]

  • Chen, J., Cho, Y., and Kira, Z. (2019). "Multi-view Incremental Segmentation of 3D Point Clouds for Mobile Robots." IEEE Robotics and Automation Letters, 4(2), pp. 1240-1246,10.1109/LRA.2019.2894915, [Full Text]

Proceedings:

  • LiBretto, M., Cho, Y., Ahn, Y., Han, C., and Ueda, J. (2019). "Configuration Optimization For End-Point Stabilization Of Redundant Manipulators With Base Flexibility." the 2019 ASME Dynamic Systems and Control Conference October 8-11, 2019, Park City, Utah

  • Chen, J. and Cho, Y. (2019). “Detection of Damaged Infrastructure on Disaster Sites using Mobile Robots.” IEEE 2019 16th International Conference on Ubiquitous Robots (UR), Jeju, Korea, June 24-27, [Full Text]

  • Chen, J., Kim, K.N., Cho,Y., Lee, J., Kim, B., Ahn, Y., and Kang, J. (2019). “Nuclear Power Plant Disaster Site Simulation using Rigid Body Physics.” Proceedings of the ASCE 2019 International Conference on Computing in Civil Engineering (i3CE), Atlanta, GA, USA, June 17-19, DOI:10.1061/9780784482421.069 [Full Text]

  • Chen, J., Cho, Y., and Ueda, J. (2018). "Sampled-Point Network for Classification for Deformed Building Element Point Clouds." Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), BrisBane, Australia, May 21-25, 10.1109/ICRA.2018.8461095 [Full text]



With the rapid advancement of laser scanning and photogrammetry technologies, frequent geometric data collection at construction sites by contractors has been increased for the purpose of improving constructability, productivity, and onsite safety. However, the conventional static laser scanning method suffers from operational limitations due to the presence of many occlusions commonly found in a typical construction site. Obtaining a complete scan of a construction site without information loss requires that laser scans are obtained from multiple scanning locations around the site, which also necessitates extra work for registering each scanned point cloud. As an alternate solution to this problem, this paper introduces an autonomous mobile robot which navigates a scan site based on a continuously updated point cloud map. This mobile robot system utilizes the 2D Hector Simultaneous Localization and Mapping (SLAM) technique to estimate real-time positions and orientations of the robot in the x-y plane. Then, the 2D localization information is used to create 3D point clouds of unknown environments in real time to determine its navigation paths as a pre-scanning process. The advantage of this framework is the ability to determine the optimal scan position and scan angle to reduce the scanning time and effort for gathering high resolution point cloud data in real-time. The mobile robot system is able to capture survey-quality RGB-mapped point cloud data, and automatically register the scans for geometric reconstruction of the site. The performance of the overall system was tested in an indoor environment and validated with promising results.


Framework for automatic registration of UAV and UGV point clouds
eg8
Point cloud registration using visual and planar features
bh_scans




Journals: Proceedings:

  • Kim, P., Price, L., and Cho, Y. (2019). “UAV-UGV Cooperative 3D Environmental Mapping.” Proceedings of the ASCE 2019 International Conference on Computing in Civil Engineering (i3CE), Atlanta, GA, USA, June 17-19, DOI: doi/pdf/10.1061/9780784482438.049 [Full Text].

  • Kim, P., Park, J., and Cho., Y. (2019). “As-is Geometric Data Collection and 3D Visualization through the Collaboration between UAV and UGV.” Proceedings of the 36th International Symposium on Automation and Robotics in Construction (ISARC), Banff, AB, Canada, May 21-24 DOI: 10.22260/ISARC2019/0073 [Full Text].

  • Kim, P. and Cho, Y. (2017). An Automatic Robust Point Cloud Registration on Construction Sites. Proceedings of the 2017 International Workshop on Computing for Civil Engineering (IWCCE), Seattle, WA, USA, June 25-27, DOI: 10.1061/9780784480823.049 [Full text]

  • Kim, P., Cho, Y., and Chen, J. (2016). Automatic Registration of Laser Scanned Color Point Clouds Based on Common Feature Extraction. 16th International Conference on Construction Applications of Virtual Reality (CONVR), Hong Kong, Dec. 11-13 [Full text]

  • Kim, P., Cho, Y. Chen, J. (2016). Target-Free Automatic Registration of Point Clouds. International Symposium on Automation and Robotics in Construction (ISARC), Auburn, AL, July 18-21, 2016, DOI: 10.22260/ISARC2016/0083 [Full text]



Journals:
Proceedings:
The research proposes a versatile mobile platform that forms an incremental 3D map of the environment using an orthogonal pair of Light Detection and Ranging (Lidar) devices. The main advantages to the proposed technique are the high update rate, high output resolution, and effective progress visualization. The proposed technique has the potential to be utilized in infrastructure surveying applications such as geometric analysis, building maintenance, and energy modeling in unknown environments.
Laser scanned point clouds are relevant for geometric analysis, building maintenance and energy modeling, yet obtaining fully registered point clouds is a labor-intensive process when targets or common registration features are absent. Thus, we propose a versatile mobile platform that forms an incremental 3D map of the environment in real time using an orthogonal pair of LIDAR (Light Detection and Ranging) devices. The horizontal scanner aims to estimate the robot position and orientation with SLAM (Simultaneous Localization and Mapping) techniques whereas the vertical scanner recovers the building structure in the vertical plane. We also developed a real time point cloud visualization tool that allows an operator to track the mapping progress. The method was evaluated with walk-through laser scans of a complete building floor.





Proceedings:
Monitoring crane motion in real time is the first step to identifying and mitigating crane-related hazards on construction sites. However, no accurate and reliable crane motion capturing technique is available to serve this purpose. The objective of this research is to explore a method for real-time crane motion capturing and investigate an approach for assisting hazard detection. To achieve this goal, this research employed various techniques in four steps: 1) a sensor-based method that accurately, reliably, and comprehensively captures crane motions in real-time; 2) computationally efficient algorithms for fusing and processing sensing data (e.g., distance, angle, acceleration) from different types of sensors; 3) an approach that integrates crane motion data with known as-is environment data to detect hazards associated with lifting tasks; and 4) a strategy that effectively presents crane operator with crane motion information and warn them with potential hazards. The projects below present our effort toward these directions.




Crane Load Positioning and Sway Monitoring using Inertial Measurement Unit

Monitoring and controlling the position of the load is important to avoid struck-by accidents caused by crane load, especially in blind lifts. However, a reliable approach to position the load and monitor the load sway remains missing. This study proposes an orientation-based approach for tracking crane load position and monitoring load sway in daily lifting activities. This approach adopts an off-the-shelf inertial measurement unit (IMU) module for measuring load orientation, and an efficient algorithm for converting orientation measurements to load positions. Test results indicate that the proposed approach correctly converted orientation measurements to accurate load positions and reconstructed the load sway trajectory in both linear and circular sway motions. Enabling continuously monitoring of crane load motion, this approach augments the crane motion information obtained by typical crane motion capturing systems. This video shows the invited presentation in the Graduate Student Demonstration in the 2015 ASCE International Workshop on Computing in Civil Engineering in Austin, Texas.



Real-time Crane Motion Capturing System and Field Test on a 70 ton Telescopic Mobile Crane

Monitoring crane motion in real time is the first step to identifying and mitigating crane-related hazards on construction sites. However, no accurate and reliable crane motion capturing technique is available to serve this purpose. The objective of this research is to explore a method for real-time crane motion capturing and investigate an approach for assisting hazard detection. A prototype system was developed and deployed on a 70 ton telescopic mobile crane. The system was tested in multiple blind lift scenarios. The preliminary results show that this system is able to capture the crane body motion and the load motion in real time. This video demostrate the system capability of capturing crane motion in rea-time and visualizing virutal crane motion in crane cabin.

Journals:

  • Chen, J., Fang, Y., and Cho, Y. (2017). "Real-Time 3D Crane Workspace Update Using a Hybrid Visualization Approach." ASCE Journal of Computing in Civil Engineering, Volume 31, Issue 5, DOI: 10.1061/(ASCE)CP.1943-5487.0000698 [Full text]

  • Fang, Y. and Cho, Y. (2016). "Effectiveness Analysis from a Cognitive Perspective for a Real-time Safety Assistance System for Mobile Crane Lifting Operations." ASCE Journal of Construction Engineering and Management, Volume 143, Issue 4, April 2017, DOI: 10.1061/(ASCE)CO.1943-7862.0001258 [Full text]

  • Fang, Y., Cho, Y., and Chen, J. (2016). "A Framework for Real-time Pro-active Safety Assistance for Mobile Crane Lifting Operations." Automation in Construction, Volume 72, Part 3, December 2016, Pages 367-379, DOI: 10.1016/j.autocon.2016.08.025 [Full text]

Proceedings:

  • Chen, J., Fang, Y., and Cho, Y. (2017). “Mobile Asset Tracking for Dynamic 3D Crane Workspace Generation in Real Time.” Proceedings of the 2017 International Workshop on Computing for Civil Engineering (IWCCE), Seattle, WA, USA, June 25-27, DOI: 10.1061/9780784480830.016 [Full text]

  • Fang, Y. and Cho, Y. (2016). "Real-Time Visualization of Crane Lifting Operation in Virtual Reality." 16th International Conference on Construction Applications of Virtual Reality (CONVR), Hong Kong, Dec. 11-13 [Full text]

  • Fang, Y., and Cho, Y. (2016). “A Framework of Lift Virtual Prototyping (LVP) Approach for Crane Safety Planning.” International Symposium on Automation and Robotics in Construction (ISARC), Auburn, AL, July 18-21, 2016, DOI: 10.22260/ISARC2016/0036 [Full text]

  • Fang, Y. and Cho, Y. (2015). “Advance Crane Lifting Safety through Real-time Crane Motion Monitoring and Visualization”. The 6th International Conference on Construction Engineering and Project Management (ICCEPM). Busan, Korea, Oct. 11-14, 2015, [Full text]

  • Fang, Y. and Cho, Y. (2015). " Crane Load Positioning and Sway Monitoring Using an Inertial Measurement Unit." 2015 ASCE International Workshop on Computing in Civil Engineering, June 21-23, Austin, TX, DOI: 10.1061/9780784479247.087 [Full text]




Proceedings:

  • Kim, Y., Haas, C., Boehme, K, Cho, Y. (1999). “Implementing an automated road maintenance machine (ARMM): Lessons from the field.” 16th International Symposium on Automation and Robotics in Construction (ISARC), Madrid, Spain, page 22-24, DOI: 10.22260/ISARC1999/0071 [Full text]