CLIP-based road-marking detection with LLM-guided driving prompts
File(s)
Published version
Author(s)
Sheng, Shaofan
Formosa, Nicolette
Feng, Yuxiang
Quddus, Mohammed
Type
Journal Article
Abstract
The advancement of artificial intelligence (AI) has significantly improved the perception and decision-making abilities of autonomous vehicles (AVs), yet real-time and accurate road marking detection remains difficult under faded markings, nighttime scenes and adverse road–weather conditions. This paper presents RoadGPT, a vision–language pipeline that couples a CLIP-based detector (RoadCLIP) with a planner-facing, LLM-generated advisory layer. The model is trained and evaluated on 3,696 road marking images covering 14 UK Highway Code classes, comprising 2,576 real images (69.7 %) from Google Street View and 1,120 text-to-image synthetic images (30.3 %) that broaden rare appearances and degraded conditions. We use 2,968 images for training (2,072 real, 896 virtual) and 728 images for testing (504 real, 224 virtual), keeping the same 70:30 real–virtual ratio in both splits. On this test set, RoadCLIP attains 98.5 % precision, 97.9 % recall and 98.2 % F1 for non-lane markings, while lane-marking subclasses reach up to 89.8 % F1. The advisory layer transforms recognised markings into structured driving prompts and is assessed via semantic similarity using Sentence-BERT (all-mpnet-base-v2) cosine scores against Highway Code-based references, achieving 89.3 % similarity, alongside an external LLM-as-judge rating of 4.68/5 for accuracy, completeness, and concise effectiveness. The full camera-to-advisory path runs in real time at 135 FPS (batch size 1, 224 × 224) on an RTX 4070 under a unified timing protocol. A remaining limitation is that visually similar lane-marking classes and extreme low-light scenes still reduce discriminability compared with symbol-like, non-lane markings.
Date Issued
2026-02-01
Date Acceptance
2025-12-06
Citation
International Journal of Applied Earth Observations and Geoinformation, 2026, 146
ISSN
1569-8432
Publisher
Elsevier
Journal / Book Title
International Journal of Applied Earth Observations and Geoinformation
Volume
146
Copyright Statement
© 2025 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Publication Status
Published
Article Number
105012
Date Publish Online
2025-12-08