Vision-based vehicle lateral localization has been extensively studied in the literature. However, it faces great challenges when dealing with occlusion situations where the road is frequently occluded by moving/static objects. To address the occlusion problem, we propose a highly robust lateral localization framework called multilevel robust network (MLRN) in this article. MLRN utilizes three deep neural networks (DNNs) to reduce the impact of occluding objects on localization performance from the object, feature, and decision levels, respectively, which shows strong robustness to varying degrees of road occlusion. At the object level, an attention-guided network (AGNet) is designed to achieve accurate road detection by paying more attention to the interested road area. Then, at the feature level, a lateral-connection fully convolutional denoising autoencoder (LC-FCDAE) is proposed to learn robust location features from the road area. Finally, at the decision level, a long short-term memory (LSTM) network is used to enhance the prediction accuracy of lateral position by establishing the temporal correlations of positioning decisions. Experimental results validate the effectiveness of the proposed framework in improving the reliability and accuracy of vehicle lateral localization.