Portfolio

Drive Fatigue Monitoring System

The Truck Driver Fatigue Monitoring System is a real time, in cab computer vision application that watches a driver facing camera feed and flags fatigue related events yawning, prolonged eye closure, and head-drop patterns as they happen, rather than during a retrospective footage review. Unlike a post hoc safety audit tool, this system is built around a hard real time constraint: a drowsy driver needs an alert within seconds, not a score that updates on a dashboard someone checks later.

Architecturally, it extends the hybrid detection philosophy from the warehouse safety system into a three-stage pipeline: a custom-trained YOLO model handles fast, continuous localization (driver, hands, phone) at the edge; a facial recognition layer identifies the driver for personalized fatigue baselines and links detections to a known operator; and an LLM reasoning layer makes the semantic call is this a yawn, is this fatigue, does this rise to an alarm worthy episode. A persistence rule on top decides when a single detection becomes a confirmed "episode" worth alarming on. The system is one input source into a larger fleet management platform, which is what turns individual in cab alerts into fleet wide safety and compliance data.

The Challenge

Low light, in-motion imaging, night driving (as in the demo frame) sharply reduces contrast for eye state and mouth state cues that a daytime warehouse camera doesn't have to deal with.

False positive risk erodes trust a resting pose (hand on chin, glancing down) can visually resemble fatigue; over alarming on natural driving behavior teaches drivers to ignore or disable the system.

Alarm timing trade off the system must wait long enough to confirm a genuine fatigue episode (not a single blink) but still alert fast enough to matter at highway speed.

Connectivity and latency at speed a moving truck can't guarantee a stable connection for a cloud round-trip on every frame the way a fixed warehouse camera can.

Driver privacy continuous biometric adjacent monitoring of an individual's face raises consent and data retention questions that a general aisle activity camera doesn't.

Appearance and equipment variability facial hair, glasses, hats (as in the demo frame), and diverse cabin lighting all need to be handled without per driver calibration.

Our Solution

The system splits detection into three specialized stages rather than asking one model to do everything:

YOLO detection (edge, continuous): a custom trained model runs on every sampled frame to localize the driver, hands, and phone at low latency and low cost this is what keeps the system responsive even when the reasoning layer isn't invoked on every frame.

Facial recognition (identity): confirms who is driving, which lets fatigue baselines and thresholds be personalized per driver rather than applying one generic threshold fleet-wide, and lets alerts and compliance logs attach to a known operator rather than an anonymous detection.

LLM reasoning (semantic judgment): takes the YOLO detections plus frame context and makes the call a bounding box alone can't is this a yawn or a cough, is this genuine fatigue or a resting pose, does the pattern across recent frames rise to an alarm worthy episode.

A persistence and threshold layer sits on top of all three: an alarm only fires once fatigue indicators are confirmed across multiple consecutive checks above a confidence threshold (visible in the demo as the 91% confidence reading on the driver detection box) this is what turns a single yawn into a labeled "episode" rather than a nuisance alert.

Technical Architecture Highlights

┌───────────────┐ ┌──────────────────┐ ┌───────────────────┐ ┌──────────────────────┐

│ In-Cab Camera │ │ Stage 1: YOLO │ │ Stage 2: Facial │ │ Stage 3: LLM │

│ Feed │──▶│ Detection (edge) │──▶│ Recognition │──▶│ Reasoning Layer │

│ (continuous) │ │ driver/hands/phone│ │ driver ID + │ │ yawn confirmation, │

│ │ │ localization │ │ personalized │ │ episode/escalation │

│ │ │ │ │ baseline │ │ decision │

└───────────────┘ └──────────────────┘ └───────────────────┘ └──────────────────────┘

│

▼

┌───────────────────────┐

│ Persistence & Alarm │

│ Rule Engine │

│ (confidence + streak) │

└───────────────────────┘

│

┌───────────────────────────┴───────────────────────┐

▼ ▼

┌───────────────────────┐ ┌───────────────────────┐

│ In-Cab Alert │ │ Fleet Management │

│ (audio/visual) │ │ System Integration │

│ │ │ (dispatch, HR, │

│ │ │ compliance records) │

└───────────────────────┘ └───────────────────────┘

This page failed to load.

Refresh the page or re-upload the theme zip. If this persists, purge LiteSpeed/Cloudflare cache for HTML and theme assets.

Return home