Reconstruction the 3D world out of two frames, based on camera pinhole model : 1. Calculating the Fundamental Matrix for each pair of frames 2. Estimating the Essential Matrix using the calibration information of the camera. Extracting the Transformation between the frames out of the Essential Matrix 3. Calculation of first-order triangulation Laboratory of Computer Graphics & Multimedia Reconstruction Results: Simulation Scenarios: Collision directionSame direction Collision directionSame direction (2) 3D Reconstruction Matches Fundamental Matrix Estimating transformation between frames Triangulation 3D Reconstructed points (4) Collision Detection Estimate dynamic points scattering Is there collision ? Static Points Static Points Static Points Static Feature Points N שחזור העולם התלת - ממדי על פי הנקודות הסטטיות בלבד Reconstruction of the Dynamic points N-1 Static Points Static Points Static Points Dynamic Feature Points N שחזור העולם התלת - ממדי על פי הנקודות הסטטיות בלבד Estimating Fundamental Matrix by the Static points N-1 Project goal: Designing an algorithm for recognition of possible collision trajectories by vehicles, using a video taken from a camera directed toward the rear of the direction of driving Presented by: Adi Vainiger & Eyal Yaacoby, under the supervision of Netanel Ratner SIFT vs. ASIFT Though slower (~50x) then SIFT, ASIFT was chosen due to accuracy reasons and fining more features. (1) Feature Detection & Matching Matches Feature Detection & Image Descriptors Frame 1 Matching Interest Points Frame 2 Feature Detection & Image Descriptors In this section we find interest points and their descriptors then match them between the two frames. This stage was implemented using the algorithm ASIFT. (2) 3D Reconstruction System outline: Frame i-N Frames (3) Recognition and Differentiation Between Static and Moving Objects (4) Collision Detection Alert Frame i (1) Feature Detection and Matching The system takes a video from a camera, with an angle to the direction of the movement. For each window of time (~2.5 seconds) in the video, the system looks at pairs of frames a second apart. Each such pair of frames is processed at stages 1 and 2. After there are enough reconstructions the algorithm performs stages 3 and 4. (1) Feature Detection and Matching (2) 3D Reconstruction (2) 3D Reconstruction (1) Feature Detection and Matching Introduction: Driving is a task that requires attention distribution. One of its many issues is identifying possible collision trajectories by vehicles from behind. Thus, there is a need for a system that automatically recognizes vehicles that are about to collide with the user, and warns him/her. Our solution is an algorithm that uses a video feed from a single simple camera, recognizes moving vehicles in the video and predicts whether they are about to collide with the user. Part A of this project focuses on the algorithm itself, without taking into account real-time constraints. (3) Recognition and Differentiation Between Static and Moving Objects Dynamic Feature Points Reconstructions Matching Variance Calculation for each point Static Feature Points N-1 3D Reconstructed points 3D Reconstructed points 3D Reconstructed points Matching the reconstructions for each point. Differentiation of moving points from static points is based on the normalized variance of the reconstructed matches of each point. High Variance Dynamic Point Reconstruction Low Variance Static Point Reconstruction L0w ambiguity High ambiguity We normalize the variance by angle and distance from camera, as the ambiguity correlates well with them On a collision course, the lines between the camera centers and the object are almost parallel. Thus, the reconstructions will be very distant from one another, as shown in results We estimate whether the dynamic points are moving towards the camera, using their scattering thorough out the reconstructions. Collision detection Results: *Scenario 4 is a collision scenario and the rest are non-collision scenarios. Ideal results for synthetic environment : 2% false negatives 12% false positives Real movie results: 3D reconstruction of the world example Static & moving object differentiation *Red points – high variance --> dynamic points. Green points – low variance --> static points Conclusions : On the synthetic environment, the system produces good results. When turning to real movies, we had several issues: Matching features of dynamic objects (due to rolling shutter) did not work, and the classification did not work well. However, under certain conditions, we still get valuable results. Further research should allow much better results. We believe that a tracking algorithm can solve most of the issues that we saw. Our thanks to Hovav Gazit and CGM Lab for the support