NCA2021 – Statistically Correlated Multi-task Learning for Autonomous Driving

Waseem Abbas, M. Fakhir Khan, Murtaza Taj, and Arif Mahmood

Abstract

Autonomous driving research is an emerging domain in computer vision and machine learning areas. Most existing methods perform Single Task Learning (STL) from one or more images while Multi-Task Learning (MTL) is more efficient due to the leverage of shared information between different tasks. However, MTL is challenging because different tasks may have different significance and varying ranges. In the current work, we propose an end-to-end deep learning architecture for Statistically Correlated MTL (SCMTL) using a single input image. Statistical correlation of the tasks to be learned is handled by including shared layers in the architecture. Later network separates into different branches to handle the difference in the behavior of each task. We observe that sharing of initial common features has increased the performance of SCMTL compared to STL. Training a multi-task model with varying task ranges may converge the objective function only with larger ranges. To this end, we explore different normalization schemes and empirically observed that the inverse validation loss weighted scheme has generated the best performance. In addition to estimating the control tasks including steering angle, braking and acceleration parameters, we also estimate the number of lanes on the left and the right side of the vehicle. To the best of our knowledge, we are the first to propose an end-to-end deep learning architecture to estimate this type of lane information. The proposed approach is evaluated on four publicly available datasets including Comma.ai, Udacity, Berkeley Deep Drive (BDD), and Sully Chen. We also propose a synthetic dataset GTA-V for autonomous driving research. Our experiments demonstrate the superior performance of the proposed approach compared to the current state-of-the-art methods. The GTA-V dataset and the lane annotations on the four existing datasets will soon be made publicly available via this web page.

Data

The datasets consists of 700,000 frames from a location similar to CA, USA. The dataset is collected during day time and it contains annotations for steering angle, acceleration, speed and brake. Furthermore, we also provide additional annotations for number of frames on left and on right side of the vehicle for 70,000 frames. Please also note that the data is for research and educational use only.

Download

Annotations: Coming soon …
Images: Coming soon …

Text Reference:

Waseem Abbas, M. Fakhir Khan, Murtaza Taj, and Arif Mahmood,
"Statistically Correlated Multi-task Learning for  Autonomous Driving, Neural Computing and Applications, 2021

Bibtex Reference:

@inproceedings{ResNet3DLeakyReLU2019,
   author   = "Waseem Abbas and
               M. Fakhir Khan and
               Murtaza Taj and                               
               Arif Mahmood",
   title     = "Statistically Correlated Multi-task Learning for Autonomous Driving",    
   booktitle = "Neural Computing and Applications",    
   month = "Aug ",
   year = "2021", }