Kedharnath Kuruba Basavaraj

• The development of Autonomous Vehicles (AVs) holds significant potential to transform society by reducing accidents, improving travel efficiency, and contributing to environmental sustainability. However, the advancement of AV technology is constrained by the challenges associated with generating large quantities of high-quality, annotated real-world data, which are costly and time-consuming to produce. This thesis addresses these challenges by exploring methods for synthesizing high-quality training data using simulation software, specifically focusing on automating the annotation process. It investigates the use of early fusion and late fusion approaches to optimize neural networks trained on synthetic data, aiming to enhance their generalization to real-world scenarios. The research leverages the Gazebo simulation tool to generate synthetic data, including RGB images and Light Detection and Ranging (LiDAR) point clouds, and employs automated labeling techniques based on transformation, rotation, and perspective projection. During the thesis, a cone detection model was built and trained n synthetic images and then tested on real images to evaluate its performance. A particular focus is placed on training the Red, Green, Blue with Depth channel (RGB-D) model using the early fusion technique, with a modified You Only Look Once v8 (YOLOv8) architecture, specifically its Nano and Small variants. Evaluations were conducted on real images from the Formula Student Objects in Context (FSOCO) dataset, and with each iteration, improvements in the model’s predictions and its ability to generalize to real images were demonstrated. The thesis specifically provides metrics for foreground and background cone classification, illustrating the effectiveness of the approach.