Computer Vision & Deep Learning Safety

Drishti Kavach AI Enhanced Indian Railways Security System

Next-Generation Optical Automatic Train Protection (ATP) & Railway Clearance Perception Engine. Augmenting Indian Railways Kavach with real-time semantic track segmentation and vector geometric physical sabotage detection.

Explore System View Live Results → Watch Demo
Scroll
Optical Shield of Vision
01 — Executive Overview

Augmenting
Railway ATP

Traditional Automatic Train Protection (ATP) systems such as Indian Railways Kavach (TCAS) rely on fixed track circuits, axle counters, and RFID transponders to prevent Signal Passed at Danger (SPAD) and train collisions. However, they are inherently blind to unexpected physical track obstructions — including deliberate rail sabotage (placed girders, rods, fishplates), fallen storm trees, boulders, stalled level-crossing vehicles, and human trespassers.

Drishti-Kavach (Hindi: Shield of Vision) introduces an autonomous forward-looking optical perception envelope. By decoupling high-speed pixel-dense track gauge segmentation from localized physical hazard detection, and cross-referencing their spatial coordinates through polygon vector geometry, it provides instantaneous driver warning and automatic emergency brake triggering.

PyTorch 2.x Ultralytics YOLO11m BiSeNetV2 Shapely Geometry OpenCV 4.x Adaptive CLAHE / DCP NVIDIA Jetson / CUDA / MPS 850nm NIR Optics
>45
FPS Edge Pipeline
<22ms
End-to-End Latency
86.0%
BiSeNetV2 mIoU
8
Sabotage Classes
Threat Tier Geometric Spatial Condition Safety Action
CRITICAL Hazard Footprint Overlaps Drivable Track / Rails ATP Emergency Brake
WARNING Hazard Within Lateral Clearance Buffer Zone Driver Audio-Visual Alert
CLEAR Obstacle Outside Structural Clearance Envelope Telemetry Log Only
02 — Architecture & Workflow

Decoupled Perception
& Spatial Reasoning

Drishti-Kavach solves the latency-accuracy tradeoff through a decoupled dual-engine architecture: a lightweight bilateral segmentation network isolates the drivable rail gauge while a fine-tuned detector localizes hazards in parallel.

Step 01 OPTICAL
Optical Enhancement
Captures forward 1080p feeds under Daylight RGB or Active 850nm NIR. Applies Adaptive CLAHE and Dark Channel Prior (DCP) defogging for zero-light and fog conditions.
src/7_pipeline/weather_enhancer.py
Step 02 SEGMENT
BiSeNetV2 Segmentation
Lightweight bilateral network with Detail and Semantic branches generates pixel-level masks for Drivable Trackbed (89.5% IoU) and Steel Rails at 512×1024 resolution.
src/2_models/bisenetv2.py
Step 03 DETECT
YOLO11m Hazard Detector
High-precision 8-class railway hazard model trained on RailSem19 and UAV-RSOD datasets, identifying sabotage iron rods, boulders, fallen limbs, barrels, and vehicles.
RailDrishti_Det_YOLO11m.pt
Step 04 REASON
Vector Spatial Engine
Converts track masks into continuous Shapely polygons, projects obstacle ground-contact points, computes lateral safety clearance, and triggers real-time ATP telemetry.
src/5_spatial_reasoning/geometry.py
System Data Flow & Inference Pipeline
Forward Optics
RGB / 850nm
Night Vision
Pre-Filter
Adaptive CLAHE
& DCP Defog
Dual Engine
BiSeNetV2 +
YOLO11m
Vector Geometry
Shapely Polygon
Intersection
ATP Output
Emergency Brake
& Driver HUD
03 — Perception Taxonomy & Verification

Class Ontologies &
Benchmark Metrics

Evaluated across the 8,500-image RailSem19 benchmark and UAV-RSOD railway sabotage dataset under both daylight RGB and active 850nm NIR conditions.

Track Segmentation Classes BiSeNetV2
  • 0. Background Surrounding terrain, catenary, vegetation, sky
  • 1. Track Bed Ballast drivable gauge footprint (89.5% IoU)
  • 2. Rail Lines Running steel rails for track horizon vectoring (68.5% IoU)
Sabotage & Hazard Classes YOLO11m (8-Class)
  • Person & Vehicles Pedestrians, maintenance staff, cars, heavy trucks
  • Sabotage Iron Rods Placed girders, metal rods, foreign rails, fishplates
  • Natural Hazards Landslide boulders, rockfalls, fallen storm tree branches
  • Containers Chemical oil barrels, fuel jerrycans, sabotage containers
Subsystem Model Architecture Resolution Parameters Primary Metric Benchmark Score
Track Bed Segmentation BiSeNetV2 (Universal) 512 × 1024 ~3.49M Track Bed IoU 89.5%
Rail Ribbon Segmentation BiSeNetV2 (Universal) 512 × 1024 ~3.49M Rail Lines IoU 68.5%
Unified Semantic Segmenter BiSeNetV2 (Universal) 512 × 1024 ~3.49M Mean IoU (mIoU) 86.0%
Obstacle & Sabotage Detector YOLO11m (8-Class Custom) 1024 × 1024 ~20.1M mAP@50 60.8%
End-to-End Pipeline Speed Decoupled Dual-Engine 1080p Stream Total Engine Total Latency / FPS < 22ms (>45 FPS)
04 — Physical Prototyping & Empirical Model Selection

Whitefield Station Miniature Testbed
& Architecture Selection

To empirically stress-test our v1.0 perception engine before scaling to full-track datasets, we engineered a high-fidelity physical miniature scale reproduction of Whitefield Railway Station. This controlled testbed enabled rigorous comparative evaluation across multiple candidate vision models under simulated lighting and physical obstruction scenarios to select the optimal architecture blend.

Whitefield Station Miniature Scale Model - Full Junction Setup
01 / Physical Testbed
Whitefield Miniature Scale Junction

Custom physical fabrication recreating multi-track junctions, sleeper geometry, platforms, and switch turnouts for continuous sensor prototyping, lens calibration, and camera mounting optimization.

Whitefield Scale Model - Perspective Calibration & Track Layout
02 / Model v1.0 Stress Testing
Controlled Optical Obstacle Evaluation

Live deployment of our v1.0 perception model under simulated physical foreign object debris, ballast displacement, and rapid rolling stock transit to test detection robustness.

Candidate Model Comparison & Neural Architecture Selection
03 / Architecture Selection & Dataset Scaling
Empirical Model Selection & Full-Scale Training

Systematic multi-model evaluation across candidate backbones to pick out and benchmark which architecture fits best (balancing latency, memory footprint, and edge compute efficiency) before committing to full-scale training on massive railway datasets.

06 — Live Demonstrations

Real-Time Continuous
Inference Feeds

Locomotive Camera Test

We tried running our final built machine learning model on a locomotive camera test video, and this was the result — performing real-time track semantic segmentation, corridor clearance boundary estimation, and active rail collision zone prediction.

07 — Acknowledgement & Engineering Team

Team

Engineered by Team Git Commit at MVJ College of Engineering. Dedicated to advancing railway AI safety for India.

01 / 04
AL
Alvin Sonny
Team Lead
MVJ College of Engineering
02 / 04
GK
Gowri Krishnan Nair
Team Member
MVJ College of Engineering
03 / 04
AS
Ananya Sanjiv
Team Member
MVJ College of Engineering
04 / 04
GT
Ganesha Thejaswi V
Team Member
MVJ College of Engineering
Field Visit to South Western Railways
Field Work & Railway Outreach

South Western Railways (SWR) Field Visit

To validate our perception architecture against real-world operational environments, our team visited the South Western Railway (SWR) headquarters and divisional facilities. We formally demonstrated the Drishti-Kavach optical ATP engine to railway officials and submitted official proposals requesting permission to conduct live on-track sensor calibration, multi-spectral camera evaluation, and real-time obstacle detection tests across active railway stations, locomotive yards, and level crossings.

South Western Railways Live Station Testing Request MVJ College of Engineering
07 — Open Source Repository

Open
Source

All training routines, model checkpoints (BiSeNetV2 & YOLO11m), spatial reasoning engines, test scripts, and documentation are openly available on GitHub under the MIT License.

View on GitHub Presentation Slide Deck →
Repositorydrishti-kavach
LicenseMIT License
Perception ModelsBiSeNetV2 (3.49M) · YOLO11m (20.1M)
Geometry EngineShapely Polygon Intersection