How we built a Driver Status Monitoring system

Fatigue is an underestimated factor leading to car accidents, a secret killer in almost 30% of cases and a cause of serious injuries in 15% of cases.

Long hours of driving, monotonous landscape, sleepless night, shiftwork – no matter the cause, driver’s fatigue can be dangerous any time, day or night, even on short distance. Fatigue doesn’t always have to lead to falling asleep at the wheel. Even brief moments of deconcentrating can prove fatal.

Fatigue impairs driver’s performance almost as much as legally proscribed levels of alcohol intoxication. It causes vigilance and alertness to deteriorate, reaction time significantly extends, the ability to judge the situation on the road lowers.

Fortunately, no one falls asleep behind the wheel out of a sudden. There are several noticeable warning signs, like blinking more than usual, yawning, drifting on or over lane lines, eyes going out of focus or head lowering. The increasing fatigue level could be spotted, triggering the alarm and saving many human lives if only these symptoms were detected in time. But each human symptoms slightly differ, so it would be perfect if we can learn them and adjust to each driver individually during driving.

That’s how the idea for a Driver Status Monitoring system was born. Our Innovation Lab team has been working on this solution in recent months, and today we’d like to tell you more about how it was developed.

How does the Driver Status Monitoring system work?

The solution designed by our team uses a set of cameras that presently monitor drivers’ eyes/irises and face but also in the future will measure human condition such as pulse, blood oxidation and temperature without on-body sensors. Today, AI and proprietary solution algorithms for sensor fusion analyse the driver’s drowsiness level based on the classification in time – closed, mid-closed, mid-open, open.

The Mediapipe solution was used for the eyes and face recognition. Then Spyrosoft algorithms calculate the opening level based on the relative eye contour and iris area so we are independent of camera distance. The algorithms adjust the values to individual drivers who are recognised after several seconds. For each eye, the system takes the middle point of the upper contour and the middle point of the lower contour and calculates the distance between them. It also includes iris diameter to calculate the maximum eye opening level.

When the system detects that eyes are closed for at least 3 seconds or when no eyes are detected whatsoever (for example, when a driver isn’t looking at the road), the appropriate alarm is played through speakers. In the future, there may also be a possibility to stop the car or call help.

What happens with the data?

The data is stored in the non-volatile memory of the device. In case of an accident, the data about the driving time, driver’s identity, warnings and alarms can be retrieved from the so-called black box. The identification of a driver is possible by face recognition and ID that is assigned to each driver. Possible use of a pulse oximeter

To get additional information about a driver’s status, it’s possible to use a pulse oximeter to measure their blood saturation and pulse rate. For the sake of correct measurement, it’s necessary to place multiple pulse oximeters on the steering wheel and possibly on the gear stick, which will be very close to each other.

Future perspectives

As we are an NVIDIA partner in a perception program, we are transferring the application to run on NVIDIA Drive IX platform now which is dedicated to being used in the automotive.

In the future, we plan to integrate NVIDIA libraries to get extra functionalities and even better accuracy of our algorithms. The Driver Status Monitoring System can also be integrated with other sensors, the infotainment system or have its functionality extended with additional algorithms.

What are the advantages of this solution?

Driver Status Monitoring system can prove successful in decreasing the number of car accidents related to driver fatigue. What is more, by adding a pulse oximeter not only the fatigue level could be measured, but also a driver’s general health status, for example, heart attack signs. Thanks to wireless communication, medical staff could be immediately notified about the bad health status of the driver.

The solution can be integrated with any type and model of a car, which is especially important in light of the new EU regulation mandating automakers to install advanced safety systems in all new cars on the EU market. The regulation was passed in November 2019 and will be gradually implemented over the course of four years, starting from 2022. Also, the US government considers making Driver Monitoring System a necessity for new cars.

It’s also worth mentioning that the solution assures a high level of driver’s privacy by limiting the access to collected data, depending on the target markets, for example, private cars vs transport trucks.