Download presentation
Presentation is loading. Please wait.
Published byAnne Quinn Modified over 9 years ago
1
Estimating Camera Pose from a Single Urban Ground-View Omnidirectional Image and a 2D Building Outline Map Tat-Jen CHAM (with Arridhana Ciptadi, Wei-Chian Tan, Minh-Tri Pham, Liang-Tien Chia) Center for Multimedia & Network Technology (CeMNet) School of Computer Engineering Nanyang Technological University, Singapore
2
Estimating Camera Pose from a Single Urban Ground-View Omnidirectional Image and a 2D Building Outline Map Tat-Jen CHAM (with Arridhana Ciptadi, Wei-Chian Tan, Minh-Tri Pham, Liang-Tien Chia) Center for Multimedia & Network Technology (CeMNet) School of Computer Engineering Nanyang Technological University, Singapore
3
Urban Landmarks Those easy to recognize Those that aren’t
© Anirudh Koul © kevincole © qureyoon
4
“Back-to-Basics” Map Reading!
An image or images taken from a single location, at probe time A plan-view outline map Won’t consider GPS GPS reception bad in high- rise urban areas GPS can be jammed or spoofed
5
Related Work and Differences
Appearance-based matching in urban areas Robertson & Cipolla BMVC04, Yeh et al. CVPR04, Zhang & Košecká 3DPVT06 General wide-baseline stereo / multi-view (but not targeted for searching through significant-sized datasets) Bay et al. CVPR05, Mičušík et al. CVPR08, Schindler et al. 3DPVT06, Schmid & Zisserman IJCV00, Werner & Zisserman ECCV02 Key differences here: No prior appearance information Only a 2D plan-view geometric map available No stereo / multi-view Images are taken from single location
6
A Geometric Matching Paradigm
Assume buildings are vertical planar extrusions Match building corners in map vertical corner lines in rectified image Significant building corners Not façade details / painted edges Geometric Signature
7
2D Geometric Image Features
Basic Lines (2D)
8
2½D Geometric Image Features
David Marr’s bottom-up visual perception framework Image Primal Sketch 2½D Sketch 3D model Augmented Lines (2D + adjacent 3D normals)
9
2½D Geometric Image Features
David Marr’s bottom-up visual perception framework Image Primal Sketch 2½D Sketch 3D model Elemental Planes (2D + fixed depth ratios of vertical boundaries)
10
2½D Geometric Image Features
David Marr’s bottom-up visual perception framework Image Primal Sketch 2½D Sketch 3D model Structural Fragments (piecewise 3D structures with unknown scales)
11
Geometric Signatures – Uniqueness Analysis Under Ideal Conditions
Structural Fragments (3D structure with unknown scale) Strong match Elemental Planes (2D + fixed depth ratios) Basic Lines (2D) Augmented Lines (2D + 3D normals) Poor match
12
Overview of Localization Method
BOTTOM-UP Query image Extract vertical corners + normals Recover elemental planes with 3D normals Link into plan-view structural fragments (modulo similarity) Calibration from vanishing points TOP-DOWN 2D map Camera pose Geometric hashing lookup for correspondence candidates Voting-based estimate of optimal camera pose
13
Estimation of Quasi-Manhattan Vanishing Points
Use EM algorithm (Schindler et al. 3DPVT 2006) Details in paper Image rectification 3D verticals become || to image y-axis
14
Vertical Corner Line Hypothesis (VCLH)
Hypotheses for corners of buildings Based on heuristics 3 Categories: Uni-Normal Augmented Line Bi-Normal Augmented Line Basic line
15
Elemental Planes Elemental Plane:
2 VCLHs connected by groups of collinear horizontal edges Same plane normals on linked sides Invariant Depth Ratio:
16
Structural Fragments Structural fragment
Sequence of adjacent elemental planes sharing bi-normal VCLHs Full 3D structure (modulo scale)
17
More Examples Elemental Planes Structural Fragments
18
Matching with Structural Fragments
Exhaustive testing: Correspondence structural fragment of l planes l linked building edges Best-fit matching with error Consensus support C from other VCLHs Vote in pose-space accumulator array Vote score: Complexity: O(n), n = # of building corners in map 8s per search on Matlab
19
Matching Example with Structural Fragments
Inconsistent matches Consistent matches
20
Experiments – Dataset I
Bronx neighborhood of Woodstock Google Street View images (total 212) 53 unique locations, 4 images per location (shown in quads) Manually created building outline plan view map 111 buildings with 885 corners
21
Experiments – Dataset II
Singapore government housing (HDB) estate Self-collected images (total 120) 30 unique locations, 4 images per location Manually created building outline plan view map 20 mega buildings with 659 corners
22
% of test probes where correct pose is better than this rank
Matching Results Compare probe signature to signatures at 3600 grid locations, and sort matching scores Find rank of ground truth % of test probes where correct pose is better than this rank Selectivity of 0-10% Match ranks
23
Dataset II Example Correct Matches
Example results for matching 3D models are only used for visualizing results
24
Observations This is a start to solving a challenging problem
difficult even for humans Results are mixed: Selectivity is very high 57-70% of correct poses within top-1% selectivity (36 out of 3600) But need to be higher to be end-usable Yet in ideal conditions signatures appear very discriminative Main challenges False VCLH negatives (some) building corners not detected due to poor resolution, etc. False VCLH positives (many) Windows / other façade features often misdetected as corners Architectural designs are seldom perfect extrusions Overhangs, balconies, fire escapes, etc.
25
Concluding Remarks Geometric features can be powerful for discriminating locations Do not always have to rely on prior appearance data Intelligent extension to geometric 2½D features 2D 2D+normals 2D+depth ratios 3D (mod scale) Informal test in ideal conditions show excellent discriminating power Key challenge lies in more robust image analysis Needs robustness to noise and minor deviations from map Future Work Use existing results to bootstrap more advanced (and costly) registration techniques E.g. top-down bundle adjustment working directly on raw image intensities, rather than detected edgels
26
Credits Joint work with Arridhana Ciptadi Wei-Chian Tan Minh-Tri Pham
Clement Liang-Tien Chia Thanks Teck-Khim Ng Zahoor Zafrulla Rudianto Sugiyarto Research Sponsor Project Tacrea Grant Defence Science & Technology Agency (DSTA), Singapore
27
Thank You
28
Scene Assumptions Quasi-Manhattan World
Vertical direction is orthogonal to all horizontal directions Horizontal directions need not be orthogonal to each other Vertical Extrusion Model Each building is a vertical extrusion of a ground-plane cross section Implies buildings have simple vertical planar facades
29
Case 1: Matching Basic Line VCLHs
Possible Procedure: 3 pairwise basic line correspondences between image and map unique basis for pose RANSAC Randomly select any 3 pairs of line correspondences Verify with other lines Exhaustive search computational cost: n is the number of building corners in a map
30
Case 2: Matching Augmented Line VCLHs
Possible Procedure: 2 pairwise augmented line correspondences for unique pose Pose basis: 2 positions + 1 normal Other excess normals used as quick compatibility test RANSAC Verify with other lines only when compatibility tests passed Exhaustive search computational cost: k<1 is the combined factor for reduced cost of quick compatibility test * unknown % of bases
31
Case 3: Matching with Elemental Planes
Possible Procedure: Exhaustive testing: Correspondence = 1 elemental plane to 1 map building edge Basis for computing pose Voting-based pose estimation Vote in pose-space accumulator array Each vote score from consensus support of other VCLHs Computational cost: m = n is the number of map building edges
32
Geometric Hashing Geometric hashing explored as a rapid shortlisting stage Get correspondence candidates between detected structural fragments and map building contours Offline Preprocessing Phase: For each building contour in map, use each contour edge as basis Other corners of the building are transformed into the canonical frame Each corner logs (building_id, basis) in related hash bin Online Lookup Phase: For each structural fragment in image, select an elemental plane as basis For other VCLHs in the structural fragment, look up hash bins and vote for building and basis. Building + basis with high votes are correspondence candidates
33
Dataset I Results – Geometric Hashing
Use large hash bins to minimize missing correct matches (false negatives) Shortlisting selectivity Speed Without hashing: 8 seconds per image on Matlab With hashing: 4 seconds (actual hashing lookup <1s) # of links per structural fragment 3 4 5 Avg # of correspondence candidates 212.98 91.06 113.33 Selectivity (out of 885) 24.1% 10.3% 12.8%
34
Geometric Signatures – Uniqueness Analysis Under Ideal Conditions
How discriminative are geometric signatures comprising 2½D features? Compute matching score: Signature at an arbitrary spot to other signatures at all locations Under ideal noise-free conditions All building corners are perfectly detectable Generate probe signatures directly from plan-view map Informal analysis Expect uniqueness to be dependent on specific location and design of buildings
35
Geometric Signatures – Uniqueness Analysis Under Ideal Conditions
Structural Fragments (3D structure with unknown scale) Strong match Elemental Planes (2D + fixed depth ratios) Basic Lines (2D) Augmented Lines (2D + 3D normals) Poor match
36
Reduction to 1D Image Matching
Vertical lines parallel to image y-axis Problem reduces to: matching points in a 2D space to a 1D point signature 2D camera extrinsic parameters: 2D position of optical center Angle of optical axis Signatures normalized to unit focal length
37
Camera Calibration from Vanishing Points
Manhattan world 3 v.p.’s 3 orthogonal directions Also computes focal length and principal point Quasi-Manhattan world Ground-vertical v.p. orthogonal to other v.p.’s Assume principal point at image center Obtain 3D v.p. directions w.r.t. camera reference frame In reality, we integrate calibration as part of the iterative v.p. estimation process
38
Potential Future Directions
Exploit localized architectural design “language”? priors to improve geometric feature detection in poor quality images predict occluded parts of higher order geometric features that form the local architectural “vocabulary” Investigate if reasonable to have prior distribution that buildings close by have similar geometric designs
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.