Deep Learning Based Vehicle Pose Estimation and Truck Body Type Classification Using a Perspective Camera

Abstract

Recently, image-based metric measurement and development of traffic surveillance systems have attracted wide interests within academia and industry as computer vision and processing power have advanced. Utilization of camera vision is gaining attention in this realm, as it is unobtrusive. Despite remarkable progress in vision-based monitoring systems for vehicle detection and image-based metric measurements, most of the research have focused on images taken from the side, the rear, or the front of the vehicle in a structured environment. Furthermore, they typically adopt the conventional image processing techniques to extract physical information, which are not robust to lighting and weather variations. Research on how to extract physical information of vehicle from a perspective view is scant. This thesis investigates identifying, tracking, and classifying (ITC) transport vehicles using image-based photogrammetry system for lane pose estimation and wheel-based measurements with a single perspective camera. The research approach adopts a deep-learning-based technique for detecting the wheels on a vehicle as regions of interests (ROI), and extracting the tire-road contact points from the image. Next, a homography-based approach is used applied to extract metric measurements, such as vehicle pose and wheel-based measurements. The proposed approach can potentially have applications in law enforcement. This approach is more effective than traditional image processing-based approaches that analyze the color, edges and shape of the wheel only. The traditional image-processing-based approaches cannot adequately deal with the difficulties that arise due to different camera orientations and varying lighting conditions at real-world traffic inspection stations. The performance of our proposed approach was investigated and evaluated on a large number of images taken at different traffic inspection stations under different lighting conditions and weather differentials to demonstrate its efficiency and robustness. The results of the investigation are presented. An additional objective of the research was designing and developing a deep learning approach for identifying and flagging vehicles that have dynamic loads in their trailer. Loads, whose center of mass can shift or move during transport, fall into this dynamic load category. Current methods which are widely used in the industry are only capable of classifying trucks based on axels and lack the ability to directly classify trucks from their body types. Hence, this research project also investigated the possibility of using a machine learning-based approach for truck cab and body type classification. Finally, the proposed method is deployed and tested at one of International Road Dynamic Inc. (IRD’s) inspection facilities. Keywords: Deep learning, Image-based metric measurement, Homography from images, Vehicle lane pose estimation, Vehicle body type classification, Wheel-based metric measurement