Abstract
Statistics derived from a representative sample of traffic accidents state that approximately 20% of traffic accidents are due to drivers' drowsy driving. Additionally, drowsy driving is the reason for 22%-30% of severe traffic accidents resulting in death.
To address the dangers of highway drivers who doze off at the wheel, vehicle manufacturers as well as technology companies are unveiling sophisticated new driver drowsiness alert systems.
Many of these solutions are developed with technologies such as embedded systems, digital signal processing, and intelligent control. For designers wanting to enhance metrics such as real-time performance, accuracy, and reliability, however, technology that includes embedded soft-core processors and programmable logic can offer a more flexible, extensible foundation.
This article examines a prototypical real-time driver drowsiness tracking system based on a soft-core embedded processor, an FPGA, and a system-on-a-chip (SOC) architecture.
Are we keeping you awake?
Whether drifting into another lane or flat-out falling asleep at the wheel, drowsy drivers are one of the road's most dangerous hazards. The challenge lies in preventing fatigued motorists from continuing behind the wheel. Vehicle manufacturers and technology companies alike are committed to tackling this problem, devoting research and development resources to enhancing drowsy driver monitoring equipment.
Many of these systems utilize a mix of embedded, digital signal processing (DSP), and intelligent control technologies. For equipment designers who want to enhance metrics such as real-time performance, accuracy, and reliability, however, embedded soft-core processors and programmable logic can support a more flexible, extensible underlying foundation. Further, designing the equipment using a system-on-a-chip (SOC) architecture can simplify design complexity, debugging, and system maintenance.
A student design team from South China University of Technology devised a way to tap into the performance and design advantages of a soft-core, 32-bit embedded processor, an FPGA, and SOC design methodologies to create a prototypical real-time driver drowsiness tracking system. By using an FPGA in the embedded design, the team was able to take advantage of the logic rich resources of the device for complex system-level functions.
Keeping an eye on the driver
The design team's prototypical real-time tracking system evaluates whether a driver is distracted or dozing, and emits a voice alert when it detects extreme fatigue or another abnormal condition.
The system also monitors information such as vehicle speed and the driver's continuous driving hours. If a driver has been on the road for five hours or more and/or exceeds a certain speed limit, the system sends out warning signals. The system also records and stores valuable data – driving hours, speed, driver and vehicle status – that can be useful if a traffic accident occurs.
The tracking system begins working once the vehicle is turned on. A digital camera equipped with a color camera image sensing chip captures the driver's image, storing this data in external SRAM memory. An image-processing algorithm, implemented in hardware to support real-time performance, analyzes the visual data, performing several functions:
- Face-zone positioning: Shrinks the image zone for faster processing and allows for better facial feature analysis.
- Eye-zone positioning: Captures images of the driver's eye zone for additional processing.
- Eyes open/close judgment: Analyzes the open/close status and winking frequency of the driver's eyes to determine whether the driver is distracted or dozing.
An Altera Nios II embedded soft-core processor features a unique self-defined instruction mode. As such, the designers were able to apply self-defined logic to deploy their system in pure hardware mode to run the image processing algorithm. The team used HDL to develop hardware modules for each of the three algorithmic modes. Using the self-defined instruction mode in Altera's SOPC Builder system development tool, available in Altera's Quartus II design software, they defined the intellectual property (IP) of each algorithm as a special instruction.
Next, the designers attached each hardware module to the Avalon bus, to serve as peripherals of the embedded processor. Rather than writing logic modules independent of the embedded processor system, using self-defined logic speeds up the design flow and enhances system compatibility.
Since the pure hardware approach utilizes parallel processing and pipelined technology, it meets the system's real-time requirements, a vast improvement over DSP systems that use serial instructions for corresponding processing algorithms. This approach also improves accuracy and reliability.
The embedded processor calls the direct memory access (DMA) controller to transmit different image data to the three SRAM-based algorithm modules according to the following steps:
- Face-Zone Positioning Module: The embedded processor starts the DMA controller, and then transmits the driver image to the face-zone positioning module. The processor responds to the interruption signal from the module after it starts running, and writes the operating result from the module.
- Eye-Zone Positioning Module: The processor enters the face-zone positioning module results into the eye-zone positioning module. Then, it starts the DMA controller again, inputs face data, writes the operating module data, finds the line position of the smallest point, and judges the position of the driver's eyes.
- Eyes Open/Close Judgment Module: The processor enters eye data to the eyes open/close judgment module through the DMA controller. This module outputs the final eye-complexity calculation results to the embedded processor for analysis.
Using DMA transfers instead of the embedded processor saves nearly 75% of transmission time and leaves the embedded processor free to handle other functions during DMA transmission.
1. System architecture for the real-time drowsiness
tracking system for drivers.
(Click this image to view a larger, more detailed version)
