1. Department of Geoinformation Science Technische Universität Berlin 2009/07/29 On the Automatic Reconstruction of Building Information Models from Uninterpreted 3D Models Thomas H. Kolbe Director of the Institute for Geodesy and Geoinformation Science Berlin University of Technology Joint work with Claus Nagel & Alexandra Stadler {kolbe | nagel | stadler}@igg.tu-berlin.de Academic Track of Geoweb 2009 Conference, Vancouver

2. 2 T. H. Kolbe – Semantische 3D-Stadtmodelle für geschäftliche Kommunikation Department of Geoinformation Science 23/1/2008 Building Information Models & IFC Building Information Model (BIM)  digital representation of the physical and functional characteristics of a constructed site or facility  comprehensive information source on a facility aiming at collaborative usage  intended to be used along the entire lifecycle of a facility  key feature: models have well-defined semantics Industry Foundation Classes (IFC)  ISO standard for semantic building models  diverse crafts/themes; incl. billing of material and costs  supported 3D geometry types: CSG, Sweep, B-Rep, etc.

3. 3 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 BIM Application #1: Energy Assessment Image: ThermoRender, Nemetschek North America

4. 4 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 BIM Application #2: Space Management Image: Space planning created with Onuma Planning System

5. 5 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 BIM Application #3: Structural Analysis Image: Autodesk Robot Structural Analysis Brochure

6. 6 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Problem Statement  BIM models are typically prepared for newly planned buildings only  But: applications should also be usable with existing buildings  Acquisition method required for BIM models for existing buildings manual acquisition is expensive  automation required  Challenges: What are appropriate data sources? Which problems have to be faced and how could they be overcome concerning the interpretation / reconstruction?

7. 7 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Starting Point: 3D Geometry / Visual Models Photogrammetric models Airborne laser scan models CAD and planning models Visualization models  Preprocessed sensor data from LIDAR / Photogrammetry, i.e. point clouds or surface patches  Visual models / surface based models (‘polygon soups’) From CAD or computer graphics From CAD or computer graphics Characteristics of input data:  Pure geometry (and radiometry)  Geometry can be unstructured or structured according to visualization purposes; it can also be incomplete  Topological errors (permeations, overshoots, undershoots)  No semantic information

8. 8 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Goal: Reconstruction of BIM models Reconstructed BIM models  explain (most of) the observed geometrical entities in the ‘best’ way  are composed of fully classified and attributed entities like Walls, Slabs, Roofs, Spaces, etc. thus, they are semantically rich and structured semantics follow the IFC standard  have volumetric, parametric geometries (CSG) required in order to make the models editable by CAD tools  should make hypotheses about 3D components with respect to invisibility / unobservability

9. 9 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 IFC Example

10. 10 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Surface-based modeling of Interior Space

11. 11 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Two-stage Reconstruction Process Automatic reconstruction of BIM models from 3D geometry models faces a high level of complexity  Unstructured, uninterpreted geometry  semantic classification handled in Stage 1: graphics model  semantically enriched boundary model  Accumulative  generative modelling paradigm handled in Stage 2: semantically enriched boundary model  building information model with volumetric, parametric components

12. 12 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Urban Models and their Applications

13. 13 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 CityGML  OGC Standard for virtual 3D city models spatial and thematic disaggregation / semantic modeling  LOD4: Building model including interior space  Modeling paradigm: BRep + explicit semantics  Close to photogrammetric / lidar observations In fact, closer than IFC models using CSG

14. 14 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Stage 1: Graphics model  CityGML = classification stage  Purely geometric graphics models (e.g., KML) are converted to semantically enriched boundary models (e.g., CityGML)

15. 15 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Underneath the surface…  Visual models are explicitly built for visualisation  only visible parts are trustworthy Wireframe model dormers are extruded through the whole building Textured visualisation visualisation does not reveal over- lapping building and dormer bodies

16. 16 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Strategies for Geometry Handling (I)

17. 17 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29  Geometry remains unchanged  Merely attach semantic information to polygons  Transform ‘polygon soup‘ into structured geometry aggregates  No effect on coordinate values  New geometry is generated according to target model and fitted to observations  May result in topological changes (e.g. closing of volumes)  New geometry is generated according to target model and fitted to observations  After geometric/semantic structuring keep original geometry in final result Strategies for Geometry Handling (II) A Keep original geometry B Structure geometry C Replace geometry D Additional requirements on the target model

18. 18 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Level-of-Detail (LOD) Concept How to decide on the appropriate target model LOD?  Conclusions about target model LOD according to input model granularity  E.g., no window setoffs or molded roof structures  ≤ LOD2  User specifies target model LOD  Attention: input model may not fulfill requirements of the chosen LOD  Specification of one basic target LOD (automatic recognition or user input)  Build LOD series covering all lower LODs (probably using generalization)  Explicit linkage between multiple LOD representations Automatic LOD recognition User input Build a LOD series

19. 19 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Stage 2: CityGML  IFC Reconstruction of component-based volume model from a surface model  Instantiation and rule-based combination of volumetric building objects (walls, roofs, …) which most likely explain the input model CityGML model is seen as the observation, IFC model will be the interpretation result  Key aspect: Semantic information as a priori knowledge Both CityGML and IFC provide semantic models of the built environment Allows for reducing the search space of potential IFC elements Complexity results from the fact that  CityGML and IFC follow different modeling paradigms  Building components are only observable in parts or not observable  typ. only observable parts are contained in input 3D model  From each component two or more surfaces may be observable Represented as individual semantic entities in CityGML

20. 20 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Differing Modeling Paradigms V olumetric, parametric primitives representing the structural components of buildings BIM (e.g., IFC) Constructive Solid Geometry Accumulation of observable surfaces of topographic features 3D GIS (e.g., CityGML) Boundary Representation

21. 21 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Matching between CityGML and IFC Entities  n CityGML entities may represent one IFC element  n CityGML entities may result in m competing IFC elements  Further 1:1 and 1:m relations possible  High combinatorial complexity  Generation of IFC element hypotheses from CityGML entities Semantic information as a priori knowledge Evaluation of geometric-topological relations between CityGML entities

22. 22 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Instantiation of IFC Elements (I) Instantiation of CSG primitives which best fit the spatial properties of all matched CityGML entities  Man-made objects often deviate from the idealized CSG shape  Parameter estimation has to obey contextual constraints Unary: usually impair the best fit of a single element Mutual: aim at aligning elements  affect parameters of many elements  Conversion of B-Rep to CSG in general is ambiguous Building components are only observable in parts  CSG primitives cannot be derived from closed volumes  Competing hypotheses  Requires additional a priori knowledge / assumptions

23. 23 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Instantiation of IFC Elements (II)  Purely geometric-topological constraints on IFC primitives cannot prevent unreasonable element hypotheses E.g., IfcRoof elements at the bottom of a building  Both CityGML and IFC do not explicitly qualify objects and inter- object relations in order to ensure sensible configurations Based on UML, XSD, and EXPRESS Focus on generic notion of ‘objects’ and ‘associations’  Reconstruction requires a framework providing enhanced model expressiveness Physical, functional, semantic / logical object constraints Rules for structural valid element configurations What makes a ‘valid’ building?

24. 24 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Interpretation Strategy (I) How to express / formalize knowledge about 3D building models?  CityGML and IFC data models do not provide (formal) constraints on object instances  A more expressive formal representation is required specifying how complex objects are aggregated in a logically / semantically sound way  Formal Grammars are becoming applied increasingly often for this purpose Formal grammars originate from computational linguistics Definition of different classes of grammars by N. Chomsky; later extended by D. Knuth (attributed grammars)

25. 25 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Example of a Formal Grammar (in EBNF)

26. 26 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Interpretation Strategy (II)  Requirements on the grammar Words / Objects have attributes (attribute grammar) Geometric Shapes (shape grammar / split grammar) Stochastical aspects (a priori probabilities) Combination of all grammar types is required Further requirements:  Need of an evaluation / objective function In order to determine the ‘best’ interpretation from all possible hypotheses meaningful definition of ‘best’ interpretation -Using probability theory: the most likely model under the given data Avoid overfittings

27. 27 Automatic reconstruction of building information models from uninterpreted 3D models Department of Geoinformation Science 2009/07/29 Conclusions  Reconstruction of BIM models is a specific instance of the general 3D object recognition problem  What makes it especially difficult? Gap between observed surfaces and volumes to be reconstructed (BRep  CSG ambiguities) High structural and semantic complexity of BIM models Uncertainty, unobservability, and errorneous observations Definition of an objective function / measure to compare the appropriateness (probability?) of BIM model hypotheses  Which aspects help in this process? Two-stage strategy allows for a step-wise interpretation and extraction of semantic information (divide-and-conquer) IFC and CityGML are target models with well-defined semantics