1 School of Computer Science and Technology, Tongji University, Shanghai 201804, China.
2 Department of Computer Science, City University of Hong Kong, Hong Kong 999077, China.
3 Key Laboratory of Road and Traffic Engineering of the Ministry of Education, Shanghai 201804, China.
4 College of Transportation Engineering, Tongji University, Shanghai 201804, China.
5 Zhejiang Fengxing Huiyun Technology Co., Ltd., Hangzhou 311107, China.
6 Department of Computer Science and Software Engineering, Concordia University, Montreal, QC H3G 1M8, Canada.

Drowsy driving is a leading cause of commercial vehicle traffic crashes. The trend is to train fatigue detection models using deep neural networks on driver video data, but challenges remain in coarse and incomplete high-level feature extraction and network architecture optimization. This paper pioneers the use of the CLIP (Contrastive Language-Image Pre-training) model for fatigue detection. And by harnessing the power of a Transformer architecture, sophisticated and long-term temporal features are adeptly extracted from video sequences, paving the way for more nuanced and accurate fatigue analysis. The proposed CT-Net (CLIP-Transformer Network) achieves an AUC (Area Under the Curve) of 0.892, a 36% accuracy improvement over the prevalent CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) end-to-end model, reaching state-of-the-art performance. Experiments show that the CLIP pre-trained model more accurately extracts facial and behavioral features from driver video frames, improving the model's AUC by 7% over the ImageNet-based pre-trained model. Moreover, compared with LSTM, the Transformer more flexibly captures long-term dependencies among temporal features, further enhancing the model's AUC by 4%.