Presentation is loading. Please wait.

Presentation is loading. Please wait.

3D Reconstruction Using UAVs

Similar presentations


Presentation on theme: "3D Reconstruction Using UAVs"— Presentation transcript:

1 3D Reconstruction Using UAVs
Binocular Stereo Structure From Motion Introduction to Drones Chia-Hung Yeh 葉家宏 Dept. of E.E. National Sun Yat-sen University Chia-Yen Chen 陳佳妍 Dept. of C.S. and I.E. National University of Kaohsiung Making 3D Models with A Drone

2 Binocular Stereo

3 Binocular Stereo Similar to how human eyes perceive depth
Acquire a pair of images using two cameras separated by a known base distance, or a single moving camera

4 Binocular Stereo Finds matching points and calculates depth values from disparities Conjugate pair is two points in different images that are the projections of he same point in the scene Disparity is the distance between points of a conjugate pair when the two images are superimposed

5 Binocular Stereo P2 Scene object points P1 Blue + Red lines
Conjugate pair Disparity for P1 Baseline distance b

6 Binocular Stereo P2 Scene object points P1 Disparity for P2
Baseline distance b

7 Binocular Stereo Larger disparity => Smaller distance
Disparity for P2 Disparity for P1 Disparity for P1 > Disparity for P2 Means P1 is closer than P2

8 Binocular Stereo f: focal length P1 (x,y,z) z xR xL f Disparity for P1
Baseline distance b

9 Binocular Stereo Accuracy of depth computation can be enhanced by increasing the baseline distance, b However, the scene seen by both cameras decreases

10 Binocular Stereo Stereo analysis Left image Right image

11 Binocular Stereo Depth map Gray values indicate distances from camera
Depth with texture map

12 Binocular Stereo What are the relative distances of the blocks?

13 Binocular Stereo What are the relative distances of the blocks?

14 Binocular Stereo Input images Output disparity map or depth map Left
Right

15 Binocular Stereo Early satellite images
Binocular images of the pentagon

16 Binocular Stereo orthogonal map
Prof. Gimelfarb at CITR :

17 Binocular Stereo What are the limits of binocular stereo? Occlusion
Some scene points are obstructed in an image Cannot find the conjugate pair Resolution Disparity values are integers Higher accuracy achieved at cost of smaller overlapping regions

18 Binocular Stereo Consider this P2
Note the order of the points’ projections on the imaging plane P1

19 Stereo Matching Correspondence problem
Detection of conjugate pairs in stereo images Still a challenging problem For each point in the left image, find the corresponding point in the right image Use similarity to determine correspondence

20 Structure from Motion

21 Structure from Motion Extension from binocular stereo
Match corresponding features and measure distances between features on the camera image plane The Scale Invariant Feature Transform is key to matching corresponding features despite varying distances Scale Invariant Feature Transform SIFT (Lowe, 1999) allows corresponding features to be matched even with large variations in scale and viewpoint and under conditions of partial occlusion and changing illumination

22 Structure from Motion Given: 𝑚 images of 𝑛 fixed 3D points
𝒙 𝑖𝑗 = 𝑷 𝑖 𝑿 𝑖 , 𝑖=1,…, 𝑚, 𝑗=1,…,𝑛 Problem: estimate 𝑚 projection matrices 𝑷 𝑖 and 𝑛 3D points 𝑿 𝑖 from the 𝑚𝑛 correspondences 𝒙 𝑖𝑗

23 Structure from Motion When we have the matching locations of multiple points on two or more photos, there is usually just one mathematical solution for where the photos were taken. Therefore, we can calculate individual camera positions, orientations, focal lengths and relative positions of corresponding features in a single step known as “bundle adjustment”. This is where the term Structure from Motion comes from. Scene structure refers to all these parameters; motion refers to movement of the camera

24 Structure from Motion Next, a dense point cloud and 3D surface is determined using the known camera parameters and using the SfM points as “ground control”. All pixels in all images are used so the dense model is similar in resolution to the raw photographs. This step is called “multiview stereo matching” (MVS) Photographs from a range of angles and distances can be used, with no a priori knowledge of locations or orientations Enables “unstructured” image acquisition from the ground, from legacy air-photosets, or from cheap, unmanned aerial platforms

25 Introduction to Drones

26 What are Drones? Drones are more formally known as unmanned aerial vehicles (UAV, 無人航空載具). Drones can be remotely controlled or can fly autonomously through software-controlled flight plans. Drones are often associated with military but they are also used for many non-military applications, such as search and rescue, surveillance, weather monitoring, etc. Drones can be as large as a full-sized plane or small enough to fit in the palm of your hand. By one estimate, more than 1500 different types of drones are manufactured today for the military, commercial and civilian markets. 靶機(Target Drone)是作為空中靶標的無人駕駛飛機。一般而言,靶機是用來鑑定地對空飛彈、空對空飛彈、航空機砲和高射砲等武器的效能,也可提供地對空飛彈、戰鬥機和高射砲練習射擊使用。靶機的種類很多,有的以屆退役的飛機和老舊飛彈改裝而成,有的是專門為某一特殊用途所設計、製造。 MQ-9 Reaper(MQ-9死神偵察機)是一款由通用原子航空系統為美國空軍所開發的無人機。MQ-9是由MQ-1掠奪者式改進而成的進階機種,但尺碼更大、載重更重,具有長滯空時程、高海拔監視的能力,有別於前代機種原本是開發作為偵察用途但轉用於主動攻擊目標,MQ-9是第一款專門設計作為攻擊用途的無人機。 Off-the-shelf: 現成的

27 The Evolution of Drones
1935 Reginald Denny mass produced radio-controlled target drones. He manufactured nearly fifteen thousand drones for the US Army during the World War II. 1994 The MQ-9 Reaper is the first attack drone designed for long-endurance, high-altitude surveillance. 2013 The DJI Phantom is the first quadcopter that requires no DIY assembly and is sold at an affordable price. 1960 The Ryan Firebee was one of the first jet-propelled drones, it was used for reconnaissance missions during the Vietnam War. Early 1990s Drones innovations are originally focused on target practice to the military. The Austrian balloons The earliest recorded use of an unmanned aerial vehicle for warfighting occurred on August 22, 1849, when the Austrians attacked the Italian city of Venice with unmanned balloons loaded with explosives. World War I The first pilotless aircraft were built during and shortly after World War I. Leading the way, using A. M. Low's radio control techniques, was the Ruston Proctor Aerial Target of 1916. 早期無人機多為軍事用途 隨著電子陀螺儀、GPS 、遙控無線電和飛行控制系統的發展,使無人飛行載具成長迅速。 Drones can be as large as a full-sized plane or small enough to fit in the palm of your hand. By one estimate, more than 1500 different types of drones are manufactured today for the military, commercial and civilian markets. 靶機(Target Drone)是作為空中靶標的無人駕駛飛機。一般而言,靶機是用來鑑定地對空飛彈、空對空飛彈、航空機砲和高射砲等武器的效能,也可提供地對空飛彈、戰鬥機和高射砲練習射擊使用。靶機的種類很多,有的以屆退役的飛機和老舊飛彈改裝而成,有的是專門為某一特殊用途所設計、製造。 MQ-9 Reaper(MQ-9死神偵察機)是一款由通用原子航空系統為美國空軍所開發的無人機。MQ-9是由MQ-1掠奪者式改進而成的進階機種,但尺碼更大、載重更重,具有長滯空時程、高海拔監視的能力,有別於前代機種原本是開發作為偵察用途但轉用於主動攻擊目標,MQ-9是第一款專門設計作為攻擊用途的無人機。 Off-the-shelf: 現成的

28 Why are Non-military Drones so Popular?
Before drones came along, the only way to get aerial footage was to hire a helicopter. With drones, anyone can use it to capture stunning aerial footage. Aerial Footage is Breathtaking There are no limitations on who can buy and fly a drone. Drones continue to become more affordable, accessible and easier to use. Drones are Easy to Acquire They are like toys for adults, which can go thousands of feet into the air with a camera . Drones are Just So Cool We can get creative with the use of our drones. There are many cool things you can do with drones.

29 What can we do with Drones?
Journalism, Filming and Photography To film sport events like skiing and snowboarding. For film makers to capture innovative shots. To be used in real estate market. Search and Rescue Efficiently and safely locate victim Drop supply 3D Reconstruction 3D scanning of large buildings and landmarks. Mapping difficult-to-access locations like eroded coastline or mountain tops. Each of our aircraft have the capability of “looking down” with high resolution video cameras to survey and search an area. Via on-board video recorder or FPV (first person view) technology, we can survey extensive terrain in real time. (Imagine flying INSIDE the cockpit). Once a subject of interest is located, the exact location can be relayed back to the ground crew. Onboard video can also be downloaded for post-flight inspection. Average flight times for these all electric drones is usually between minutes giving the pilot enough time with several batteries to scan an area miles in diameter using a zig-zag sweep pattern or expanding spiral search pattern. One drone can search an extensive area in minutes that might take an entire group of ground personnel to cover in hours. Besides locating victims, a multi-rotor could potentially be used to “drop in supplies” to an otherwise unreachable location. For example a drone might be utilized to lower a walkie talkie, gps locater, med supplies or water to a stranded victim before rescue crews are able to extract them.

30 What can we do with Drones?
Agriculture Monitor farms Collect crop data Manage crop and livestock Conservation Track endangered species Environmental monitoring Monitor illegal dumping. Shipping/Delivery Drones can be used to deliver small packages, pizzas, medicines, beverages in short distances. Emergency call for distribution of medication and aid. By Daniel Kelly on November 5, 2014 Unmanned aerial drones have a lot of uses in environmental monitoring, from aiding conservationists to keeping track of agriculture. But the National Oceanic and Atmospheric Administration is using them for another purpose, according to TreeHugger: whale tracking. Drones are a good tool for the effort because they can circle high above whales and keep out of earshot. The high-altitude approach is good for taking snapshots of whale girth, a marker of a healthy diet, without disturbing them. Researchers with NOAA have so far used drones to successfully study killer whales off the coast of Canada. Using a custom-built hexacopter, they’ve captured 30,000 images over just 60 flights. Scientists say the technology will help them identify unhealthy whale conditions earlier, as well as aid in forming future conservation plans. Top image: An orca jumping out of the water near East Poiont, Saturna Island. (Credit: Miles ritter, via Flickr/CC BY 2.0)

31 可穩定停懸於空中,操控靈活,適合精細量測
Types of Drone 類型 定翼型 (Fixed-Wing) 旋翼型 (Multi-Rotor) 示意圖 滯空時間 可達60分鐘 約10~25分鐘 飛行範圍 起飛場地 需跑道供起飛降落 小型平台即可垂直起降 飛行高度 約100m 可達500m 載具尺寸 翼展1400mm 軸距350mm 特點 因浮力好,飛行時間長,適合大範圍量測 可穩定停懸於空中,操控靈活,適合精細量測 定翼型即常見的飛機,機身有一對固定的機翼,使其產生浮力滑翔於空中 旋翼型以直升機為代表,機身有至少一組得主旋翼向下產生推力,使機體能滯留於空中

32 Pros and Cons of Different Types of Drones
Typical Uses Price ($AUD) Multi-Rotor Accessibility Ease of use VTOL and hover flight Good camera control Can operate in a confined area Short flight times Small payload capacity Aerial Photography and Video Aerial Inspection $5k-$65k for pro drones Fixed-Wing Long endurance Large area coverage Fast flight speed Launch and recovery needs a lot of space No VTOL/hover Harder to fly, more training needed Expensive Aerial Mapping, Pipeline and Power line inspection $25-$120k for pro drones Single-Rotor Long endurance (with gas power) Heavier payload capability More dangerous Aerial LIDAR laser scanning Fixed-Wing Hybrid VTOL and long-endurance flight Not perfect at either hovering or forward flight Still in development Drone Delivery TBD, in development (Vertical Take-Off and Landing) This article appears in the Australian DRONE magazine, issue 3 (June 2016). It is a great read for hobbyists and professionals alike, with both fun and informative articles, and all the latest gear — check it out:

33 Potentials and Outlooks
Perceptual smart machine age includes more familiar technologies such as autonomous vehicles, commercial UAVs and smart dust and will be the most disruptive class of technologies over the next 10 years.

34 Cameras on Drones At first, no cameras Attach small cameras, GoPros
Built-on cameras, with 4K resolution Drones able to carry SLR cameras

35 Cameras on Drones Live HD video feed direct from the camera 30fps Full camera control Video stored on SD card UHD: 4096x2160p 24/25, 3840x2160p 24/25/30 FHD: 1920x1080p 24/25/30/48/50/60 HD: 1280x720p 24/25/30/48/50/60

36 GIS Drone Drone acquisition & generation of raster data
Flight planning Flight(s) Image download & processing Generation of orthophotos / digital surface models Extraction of vector data GPS recording Geodatabases Manual data collection Digitisation Web services Drone point cloud Import & layering of vector & raster data GIS software Analysis & decision making Progress tracking, feature identification, flood simulation, project planning etc.

37 GIS Drone Skycatch releases mobile apps built on the DJI SDK, COMMANDER iOS, compatible with the DJI Phantom 3. The app creates high-resolution maps and 3D models in just a few taps.

38 Exercise for the Next Class
Find recent applications or technologies that use UAV Less than 3 years old Article from reliable source Excluding the examples that I have given Prepare minutes presentation on the selected topic What kind of UAV used? Advantages over conventional methods Difficulties encountered Future aspects Please include your references


Download ppt "3D Reconstruction Using UAVs"

Similar presentations


Ads by Google