Since we are focusing on driver drowsiness detection, out of the 468 points, we only need landmark points belonging to the eye regions. The eye regions have 32 landmark points (16 points each). For calculating the EAR, we require only 12 points (6 for each eye).
The previous sections covered all the required components for creating a driver drowsiness detection application. Now, we will start building our streamlit web app to make this application accessible to anyone using a web browser.
Vehicle drivers driving cars under the situation of drowsiness can cause serious traffic accidents. In this paper, a vehicle driver drowsiness detection method using wearable electroencephalographic (EEG) based on convolution neural network (CNN) is proposed. The presented method consists of three parts: data collection using wearable EEG, vehicle driver drowsiness detection and the early warning strategy. Firstly, a wearable brain computer interface (BCI) is used to monitor and collect the EEG signals in the simulation environment of drowsy driving and awake driving. Secondly, the neural networks with Inception module and modified AlexNet module are trained to classify the EEG signals. Finally, the early warning strategy module will function and it will sound an alarm if the vehicle driver is judged as drowsy. The method was tested on driving EEG data from simulated drowsy driving. The results show that using neural network with Inception module reached 95.59% classification accuracy based on one second time window samples and using modified AlexNet module reached 94.68%. The simulation and test results demonstrate the feasibility of the proposed drowsiness detection method for vehicle driving safety.
The contributions can be summarized as follows: (1) a vehicle driver drowsiness detection method using wearable EEG is proposed to alert and warn vehicle drivers under drowsiness conditions. (2) The method uses the neural networks with Inception module and modified AlexNet module to extract the feature of the EEG signals and then train and classify the EEG signals. (3) The early warning device is used to warn the status of vehicle drivers. If the vehicle driver is normal, the early warning device will show a white light. If the vehicle driver is judged as drowsy, the early warning will show a red light and sound an alarm.
From the above literature, it can be seen that EEG signals are widely used in driver drowsiness detection. However, researchers also found that EEG signals are very weak and susceptible to the background noise. Therefore, how to extract high-quality EEG signals under drowsy driving and how to accurately classify the EEG signals require further researches.
We visualize the relationship between driver drowsiness and brain position using network structure with Inception module. The visualization result is shown in Fig. 14. The horizontal axis of the image represents the sequence value. The vertical axis is the channel, which from top to bottom are Fp1, Fp2, C3, C4, T7, T8, O1 and O2. Figure 14a shows the beginning of the network training, and its weights are randomly generated, which looks chaotic. As the training progresses, the weight distribution gradually changes. After 20,000 iterations of the network, it can be seen from Fig. 14b that the basic white dots are concentrated in the bottom two rows, which means the EEG signals of O1 and O2 channels are most closely related to the drowsy state.
On the other hand, we compared the proposed method with the other state of the art methods. Lin et al.  proposed a one channel BCI system using Mahalanobis distance (MD) to detect the drowsiness in real time. Zhang et al.  used a support vector machine (SVM) classification algorithm and the fast Fourier transform (FFT) to determine the vigilance level. Li et al.  proposed a smartwatch-based wearable EEG system using support vector machine-based posterior probabilistic model (SVMPPM) for driver drowsiness detection. Punsawad et al.  developed a single-channel EEG-based device for real time drowsiness detection. Chai et al.  presented a two-class EEG-based classification using Bayesian neural network for classifying of driver fatigue. Wali et al.  used discrete wavelet packet transformation (DWPT) and fast Fourier transformation (FFT) to classify the driver drowsiness level. The comparison results are shown in Table 1.
For security reasons, the early warning system cannot be tested in a real environment, so we use the OpenBCI Cyton EEG detection system and the Arduino open-source electronic platform to verify the effectiveness of the above simulation. The early warning strategy experiment is shown in Fig. 20. When the driver is in awake state, first-class drowsiness state and second-class drowsiness state, the corresponding response of the early warning equipment is that the white light is on, the red light is on, and the buzzer sounds. The experimental results are consistent with the simulation state, verifying the reliability of the early warning strategy.
In this study, the vehicle driver drowsiness detection method using wearable EEG based on convolution neural network is presented. The EEG collection module, EEG signal processing module and early warning module formed a complete system which can be used in vehicle driving safety. The final experimental results show the great performance of the proposed method in vehicle driver drowsiness detection. Specifically, the equipment provides excellent classification efficiency, and the accuracy can reach 95.59% based on a one second time window samples using neural network with Inception module and reach 94.68% using modified AlexNet network module during simulation and tests. The proposed early warning strategy is also very effective. The simulation and test results demonstrate the feasibility of the proposed drowsiness detection system using EEG signals for vehicle driver driving safety.
Recent studies have shown that drowsiness is one of the major factors of road accidents that causes a large number of fatalities and monetary losses1,2,3,4. National Highway Traffic Safety Administration (NHTSA) announces that about 1.9% of total driving fatalities in 2019 (697 fatalities) were caused by drowsy drivers5. In another report, NHTSA estimated that in 2017, 91,000 police-reported crashes involved drowsy drivers that led to approximately 50,000 traffic injuries and 800 fatalities6. An assessment of the American Automobile Association (AAA) found that about 24% of drivers revealed been extremely drowsy while driving, at least once in the last month7. Furthermore, 14.5% of the drivers in the USA have been involved in at least one drowsiness-related traffic collision, according to a study carried out in 20088. Some studies also showed that the level of drowsiness in automated driving is significantly higher than in manual driving10,11,12. Given all this evidence, the estimation of driver fatigue is essential for road safety and also future intelligent transportation systems require a vigilant driver for take-over requests from automated vehicles failing to perform safely.
Generally, three types of data have been used in the literature to design driver drowsiness detection systems: (1) vehicle-based13,14, (2) vision-based15,16, and (3) physiological data17,18. The literature suggests that physiological data such as EEG may be more appropriate than other systems to detect the onset of driver drowsiness19,20 specifically because vehicle-based and vision-based systems can be too late in warning the driver in the early stages of drowsiness, when there might still be time to prevent the accident. Critical signs of drowsiness such as yawning and head-nodding often appear before lateral displacement of the car and other non-physiological signs. Vision-based systems, while convenient, also suffer from robustness limitations in different light conditions and their performance can be significantly degraded when the drivers wear glasses or sunglasses21,22. Furthermore, data privacy can also be another issue for vision-based drowsiness detection systems which should be more studied in future research works.
Alongside developing a real-time modeling solution to estimate driver drowsiness, we are interested in identifying neural biomarkers of drowsiness which may be useful to others studying drowsiness and needing reliable biomarkers. In the development of our modeling solution, we expand on the dynamical neural encoder-decoder modeling framework which has been successfully utilized in other applications such as extracting multi-dimensional auditory and visual stimulus-response correlations35, decoding neural recordings to predict speech36, reconstructing natural images using Bayesian decoder37, and decoding hidden cognitive states38.
Abstract-The major aim of this project is to develop a drowsiness detection system by monitoring the eyes; it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident. In such a case when drowsiness is detected, a warning signal is issued to alert the driver. This detection system provides a noncontact technique for judging different levels of driver alertness and facilitates early detection of a decline in alertness during driving. In such a case when fatigue is detected, a warning signal is issued to alert the driver. The system also has additional feature of slowing down the vehicle if driver fails to respond to the alarm and ultimately stops the vehicle.
The aim of this project is to develop a prototype drowsiness detection system. The focus is on designing a system that will accurately monitor the open or closed state of the drivers eyes in real-time. By monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident. Detection of drowsy involves a pattern of images of a face, and the observation of eye movements and blink rate. The analysis of face images is a popular research area with applications such as face recognition, virtual tools, and human identification security systems. This 2b1af7f3a8