1. 1 Architecture based modeling of buildings Filip Van den Borre

2. 2 Overview Objective GeoAutomation data set Fit ground plane model Fit facade planes Style grammar rjMCMC Future work

3. 3 Objective Model buildings Without human interaction Input: images Semantic model: knowledge about the structure of the building and his elements. Use architectural grammar: set of rules that can be used to describe the building model Repeated elements -> one enhanced texture for each

4. 4 Objective Procedural Modeling of Buildings. [Mueller et al.] SIGGRAPH 2006 CGA shape: shape grammar for the procedural modeling of CG architecture. low cost -> computer games, movies,... efficient unprecedented level of detail

5. 5 Example PRIORITY 1: 1: footprint ; S(1r,building height,1r) facades T(0,building height,0) Roof(hipped,roo f angle){ roof } PRIORITY 2: 2: facades ; Comp(sidefaces){ facade } 3: facade : Shape.visible(street) ; Subdiv(X,1r,door width*1.5){ tiles | entrance } : 0.5 ; Subdiv(X,door width*1.5,1r){ entrance | tiles } : 0.5 4: facade ; tiles 5: tiles ; Repeat(X,window spacing){ tile } 6: tile ; Subdiv(X,1r,window width,1r){ wall | Subdiv(Y,2r,window height,1r){ wall | window | wall } | wall } 7: window : Scope.occ(noparent) != none ; wall 8: window ; S(1r,1r,window depth) I(win.obj) 9: entrance ; Subdiv(X,1r,door width,1r){ wall | Subdiv(Y,door height,1r){ door | wall } | wall } 10: door ; S(1r,1r,door depth) I(door.obj) 11: wall ; I(wall.obj) PRIORITY 3:...... 14:

6. 6 My mission PRIORITY 1: 1: footprint ; S(1r,building height,1r) facades T(0,building height,0) Roof(hipped,roo f angle){ roof } PRIORITY 2: 2: facades ; Comp(sidefaces){ facade } 3: facade : Shape.visible(street) ; Subdiv(X,1r,door width*1.5){ tiles | entrance } : 0.5 ; Subdiv(X,door width*1.5,1r){ entrance | tiles } : 0.5 4: facade ; tiles 5: tiles ; Repeat(X,window spacing){ tile } 6: tile ; Subdiv(X,1r,window width,1r){ wall | Subdiv(Y,2r,window height,1r){ wall | window | wall } | wall } 7: window : Scope.occ(noparent) != none ; wall 8: window ; S(1r,1r,window depth) I(win.obj) 9: entrance ; Subdiv(X,1r,door width,1r){ wall | Subdiv(Y,door height,1r){ door | wall } | wall } 10: door ; S(1r,1r,door depth) I(door.obj) 11: wall ; I(wall.obj) PRIORITY 3:...... 14:

7. 7 Data of GeoAutomation Spin-off company of Visics System of precise mapping measurements by means of mobile mapping Vehicle with 10 cameras 3D feature points clouds Geo-referenced by reference to ground control points Gravity vector g || z-axis x, y, z in meters

8. 8 Different cameras

9. 9 Point cloud of Sint-Truiden

10. 10 Close-up

11. 11 Preprocessing Feature points located at great distances from the cameras have a low precision They create noise with respect to other points When only the correct points are retained, there are not enough of them to provide proof for possible models Remove the feature points if distance > 20 m

12. 12

13. 13

14. 14 Split data up into voxels No relation between feature points at a certain distance from each other Easier to access & analyse the data Voxel size has the same order of magnitude as the minimal size of the structures we want to find in the first steps (In the first steps we want to locate the ground and the facades without the door and windows structures).

15. 15

16. 16 Fit ground level Assumption: visual point with the smallest z-value = point on the ground plane If (lowest point ground plane) then Point is an outlier There are no feature points on the ground plane Carry out a median filtering to correct errors (lowest point ground plane)

17. 17 Fit ground level

18. 18 Fit wall planes Assumption: Walls // z-axis Remove ground points Find median point (x m,y m ) in each voxel Give each point a gaussian weight with μ= ||p i -m|| and σ = 1 meter Weighted Singular Value Decomposition (SVD) on each voxel

19. 19 Fit wall planes Problems: Corner (two walls coming together) No walls Plane-shaped objects like cars, vans, fences,... Street furniture: trees, (traffic) lights, traffic signs,...

20. 20 Fit wall planes Solutions: The separate walls of a corner are found by looking at the walls in the neighboring voxels Errors caused by plane-shaped objects will be removed by projecting the images on the planes or by using car-detectors,... The singular values give us an idea of how plane – alike a given point cloud is Results with S 1 /S 2 < thresholds are removed

21. 21 Fit wall planes Facade planes all other objects No feature points behind the facade plane Count all points after a plane If (#points ~ 0) -> facade plane

22. 22 Fit wall planes

23. 23 Fit wall planes

24. 24 Textured model

25. 25 Textured model

26. 26 Textured model

27. 27 Textured model

28. 28 Extra information Rectified Images Projected feature points with depth information

29. 29 Style grammars Use shape grammar of Mueller: CGA shape Problem: too flexible -> very large search space Solution: stricter rules Use intuitive subdivision of buildings / semantics Use style dependent grammars

30. 30 Style grammars (example) Building SPLIT VERTICAL Facade Facade (height~2.5m) REPLACE FirstFloor Facade (height>>2.5m) SPLIT HORIZONTAL FirstFloor (height=gauss(2.5m,0.5m)) Upperfloors Upperfloors REPEAT HORIZONTAL Floor (height=gauss(2.5m,0.5m)) FirstFloor SPLIT VERTICAL FirstFloor DoorTile (height=gauss(1m,0.2m)) FirstFloor FirstFloor SPLIT VERTICAL FirstFloor WindowTile (width=unifor(0.2m,3m)) FirstFloor DoorTile SPLIT HORIZONTAL Door (height=gauss(2m,0.2m)) Wall WindowTile SPLIT HORIZONTAL Wall Window (height=gauss(1m,0.5m)) Wall

31. 31 rjMCMC Find solution for P(Ø | S, D) Ø: rules and parameters of shape grammar S: given shape grammar D: input data (images and 3D) P(Ø | S, D) is complex and the dimension vary depending on the used rules Work of Ripperda and Dick shows us that we can use rjMCMC for this

32. 32 Future work Join facade planes to a 3D shape Find smaller 3D structures like doors & windows Find description/model based on rules Perform in-painting of hidden structures based on the resulting model and the images...