Street Crossing Tracking from a moving platform Need to look left and right to find a safe time to cross Need to look ahead to drive to other side of road Must stay in crosswalk
Algorithm for Tracking Cars 1.Use image differencing method to extract motion regions 2.Noise filter using 3x3 median filter; effective for typical CCD sensor noise 3.Compute edges of motion regions using Canny edge detection 4.Use Mori’s “sign pattern” to find bottoms of cars [Mori 1994] 5.Find bounding boxes of moving objects 6.Use knowledge from prior frames to mark direction of travel of each bounding box
Mori Sign Pattern Tracking algorithm uses “Mori Scan” to reliably detect undersides of cars The Mori “sign pattern” for vehicle detection says: the shadow underneath a vehicle is darker than any other spot on the paved road The Mori result is invariant to lighting and holds for wet and dry roads Use of the Mori result obviates the need for explicit shadow detection and/or removal; previously, prominent shadow edges caused oversize bounding boxes
Mori Sign Pattern
Street Crossing Six frames of a tracking sequence
System Validation When it is safe to cross, a person monitoring the traffic scene presses a button A second button press means it is no longer safe to cross The time between button presses specifies a safe crossing window Use more than one person to compensate for individual risk tolerance Button press data is synchronized with the video data Compare system safety estimates to human safety judgments
System Validation When it is safe to cross, a person monitoring the traffic scene presses a button A second button press means it is no longer safe to cross The time between button presses specifies a safe crossing window Use more than one person to compensate for individual risk tolerance Button press data is synchronized with the video data Compare system safety estimates to human safety judgments
Crosswalk Traversal While crossing, devotes more processing to the right-looking video stream Uses a forward-looking camera to detect and stay on the marked (zebra striped) crosswalk Uses sonar to avoid pedestrians and stopped cars on the crosswalk Uses a laser range finder to detect the curb cut; driving over curbs is possible, but undesirable
Crosswalk Traversal While crossing, devotes more processing to the right-looking video stream Uses a forward-looking camera to detect and stay on the marked (zebra striped) crosswalk Uses sonar to avoid pedestrians and stopped cars on the crosswalk Uses a laser range finder to detect the curb cut; driving over curbs is possible, but undesirable
Related Work Automated driving systems: CMU’s Navlab project, Dickmanns’ autonomous Autobahn vehicle, DARPA Challenge Traffic scene monitoring systems that analyze traffic conditions Camera orientation and assumptions of existing vision-based, car-tracking systems do not apply to street crossing Robotic street crossing has not been done previously
Reasoning about Bounding Boxes Larger, lower (in the image plane) bounding boxes correspond to close cars Smaller, higher bounding boxes denote distant cars Tracking in real time using Phission so cars move very little from frame to frame Track individual cars over time to determine speed and travel direction Need to smooth results over time since CCD cameras produce noisy data
Research conducted under the auspices of Dr. Holly A. Yanco, the Robotics Lab, and the Computer Science Department. Contact for additional