Presentation is loading. Please wait.

Presentation is loading. Please wait.

GM-Carnegie Mellon Autonomous Driving CRL Structured Hough Voting for Vision- based Highway Border Detection 1 Zhiding Yu Carnegie Mellon University.

Similar presentations


Presentation on theme: "GM-Carnegie Mellon Autonomous Driving CRL Structured Hough Voting for Vision- based Highway Border Detection 1 Zhiding Yu Carnegie Mellon University."— Presentation transcript:

1 GM-Carnegie Mellon Autonomous Driving CRL Structured Hough Voting for Vision- based Highway Border Detection 1 Zhiding Yu Carnegie Mellon University

2 GM-Carnegie Mellon Autonomous Driving CRL Autonomous Driving: Not If, But When 2

3 GM-Carnegie Mellon Autonomous Driving CRL GM-CMU Collaborative Research

4 GM-Carnegie Mellon Autonomous Driving CRL Sensors Setup on SRX Platform Images from: Junqing Wei et al., “Towards a Viable Autonomous Driving Research Platform,” IEEE Intelligent Vehicles Symposium (IV), 2013

5 GM-Carnegie Mellon Autonomous Driving CRL Sensors: Price vs Information Price Information Radar Lidar Camera

6 GM-Carnegie Mellon Autonomous Driving CRL Computer Vision Applications  Object detection (pedestrian, vehicle, bicycle…)  Road parsing (lane/border detection, road segmentation, vanishing point estimation…)  Localization and tracking  Driver status monitoring  Many other applications……

7 GM-Carnegie Mellon Autonomous Driving CRL Motivation, Description and Goal 7 Goal –Development for future driving assistance system and autonomous driving system –Robust detection within 0.5 to 6 meters detection range. Achieve near 100% accuracy in daytime and over 90% in nighttime on the right most lane –Handling various scenarios including highway entrance and exit –Extend to the joint system with front view

8 GM-Carnegie Mellon Autonomous Driving CRL Concrete Barrier Guard Rail Soft Shoulder High-Level Idea: Learning based Method Guard Rail Soft Shoulder Concrete Barrier Lane Marking Densely Fired scanning windows Returned Voting Points Border / lane marking hypotheses Structured Hough Voting

9 GM-Carnegie Mellon Autonomous Driving CRL Overall 1592 training images: 1.Concrete Barrier (839 images) 2.Guard Rail (300 images) 3.Soft Shoulder (453 images) Overall 2638 testing images: Dataset Collection

10 GM-Carnegie Mellon Autonomous Driving CRL Training Patch Alignment Negative Samples: Positive Samples: ConcreteNaturalSteelLane Marker

11 GM-Carnegie Mellon Autonomous Driving CRL Filter Bank Patches that are discriminative to HOG Patches that are discriminative to filter banks Concatenated Filter Bank Feature Feature Extraction HOG Concatenated HOG Feature

12 GM-Carnegie Mellon Autonomous Driving CRL  Extract features from all training patches (based on previous page)  Perform Fisher discriminant analysis  Train an RBF kernel SVM  Scanning window detection (Deliberately having a lot of positive firing) Classification & Detection Guard Rail Soft Shoulder Concrete Barrier Lane Marking

13 GM-Carnegie Mellon Autonomous Driving CRL Hough Voting

14 GM-Carnegie Mellon Autonomous Driving CRL Structured Hough Voting: Intuitions  Basic philosophy: A model that assumes voting results are correlated rather than independent  Inter-frame structural info on hypotheses (Temporal smoothness)  Intra-frame structural info (Geometric relationship)  Multiple candidate hypotheses generation (Proposals with diversity) 1.Constrained Hough Voting on detected voting points (Detection + Tracking) 2.Arbitrary Hough Voting on detected voting points (Detection) 3.Constrained Hough Voting on image gradients (Pure Tracking)

15 GM-Carnegie Mellon Autonomous Driving CRL  Deals most of the frames where hypotheses from consecutive frames have strong correlation. Purpose of Candidate 1

16 GM-Carnegie Mellon Autonomous Driving CRL  Automatically corrects result through searching for “much better” voting configurations (This is the power of detection, avoids error from tracking) Purpose of Candidate 2

17 GM-Carnegie Mellon Autonomous Driving CRL  In the worst case where Type 1 voters fail, perform tracking by gradients from previous pose configuration. Purpose of Candidate 3

18 GM-Carnegie Mellon Autonomous Driving CRL Modeling under CRF: Background  A Conditional Random Field (CRF) discriminatively defines the joint posterior probability as the product of a set of potentials  The potentials are functions with hypotheses H i being the variables. They are modeled in such a way that a larger potential value generally indicates a better hypothesis configuration.  CRF inference seeks to find the joint hypothesis configuration H that maximizes H1H1 X1X1 H2H2 HNHN … Unary Potential Pairwise Potential X2X2 XNXN

19 GM-Carnegie Mellon Autonomous Driving CRL Modeling under CRF: Intuition  What are the hypothesis H i ?  E.g.: image pixel labels (FG/BG, Object Class, etc.), if it is a segmentation problem.  In our problem, H i is the Hough Voting hypothesis: H i = (r, θ ).  X is the observation of voting point coordinates and their weights.  The unary potential corresponds to the exponential of Hough voting weights: exp(v(H i )).  The pairwise potential corresponds to the inter-frame smoothness (tracking) constraint. H1H1 X1X1 H2H2 HNHN … X2X2 XNXN

20 GM-Carnegie Mellon Autonomous Driving CRL No Structural Information H bd,1 … Simplest Case: frame-wise independent Hough voting H bd,2 H bd,N X1X1 X2X2 XNXN H ln,1 … H ln,2 H ln,N X1X1 X2X2 XNXN

21 GM-Carnegie Mellon Autonomous Driving CRL Adding Inter-frame Structural Info. H bd,1 … Adding temporal smoothness: Hough voting constrained by neighboring frames H bd,2 H bd,N X1X1 X2X2 XNXN H ln,1 … H ln,2 H ln,N X1X1 X2X2 XNXN

22 GM-Carnegie Mellon Autonomous Driving CRL Adding Intra-frame Structural Info. H bd,1 … Adding Geometric Constraint: Hough voting constrained by both neighboring frames and intra-frame hypotheses H bd,2 H bd,N X1X1 X2X2 XNXN H ln,1 … H ln,2 H ln,N X1X1 X2X2 XNXN

23 GM-Carnegie Mellon Autonomous Driving CRL The Structured Hough Voting Model Candidate Hypotheses Generation Unit Mode Selection Potential Coupled Structure Potential

24 GM-Carnegie Mellon Autonomous Driving CRL The Structured Hough Voting Model

25 GM-Carnegie Mellon Autonomous Driving CRL Candidate Hypotheses Generation Unit

26 GM-Carnegie Mellon Autonomous Driving CRL  Use decision tree to guide the mode selection.  The mode selection basically forces the output to be one of the candidate hypotheses, but allows discrepancy with the decision tree prediction with a penalty. Mode Selection Potential

27 GM-Carnegie Mellon Autonomous Driving CRL  The coupled structure potential captures two most important relations between a border hypothesis and a lane hypothesis  Parallelism  Distance Coupled Structure Potential

28 GM-Carnegie Mellon Autonomous Driving CRL Inference  Conducting a whole inference each time given a new frame is computationally infeasible.  Relaxation: Initialize with the inferred state variable configuration of the previous t-1 frames and infer the current state variables, updating in an incremental way.  Inference procedure at t = 1: 1. Perform Hough voting for both border and lane marking 2. Perturbate hypotheses if geometric relationship violated (optional)  Inference procedure at t > 1: 1. Generate the 3 candidate hypotheses for both border and lane marking 2. Use decision tree to help selecting the best candidate 3. Perturbate candidate hypotheses if geometric relationship violated (optional) 4. Re-select the best candidate

29 GM-Carnegie Mellon Autonomous Driving CRL Experiments: Adding Coupled Structure

30 GM-Carnegie Mellon Autonomous Driving CRL Experiments: Qualitative Results Ground Truth and Baseline methods: 1.Ground Truth 2.Independent Hough voting in each frame using the fired detector voting points 3.Hough voting using the triggered detector voting points constrained by previous frame 4.Adding gradient tracking to Baseline 2. 5.Kalman filter. 6.Proposed Method

31 GM-Carnegie Mellon Autonomous Driving CRL Experiments: Quantitative Results

32 GM-Carnegie Mellon Autonomous Driving CRL Highway Entrance Detection and Lane State Tracking

33 GM-Carnegie Mellon Autonomous Driving CRL Summary  Proposed the Structured Hough Voting Model  The proposed model can be theoretically formulated under a CRF  Fast real-time feature extraction and online inference  Achieves very robust and good performance under challenging scenarios and low quality inputs from production camera

34 GM-Carnegie Mellon Autonomous Driving CRL Thank You! Q & A


Download ppt "GM-Carnegie Mellon Autonomous Driving CRL Structured Hough Voting for Vision- based Highway Border Detection 1 Zhiding Yu Carnegie Mellon University."

Similar presentations


Ads by Google