Automatic Annotation of Urban Environments to Support POL Behavior Automatic Annotation of Urban Environments to Support Pattern of Life Behavior Daniel J. Hershey David M. McKeown TerraSim, Inc. One Gateway Center, Suite Fort Duquesne Blvd. Pittsburgh, PA March 29, 2012 GameTech 2012 Orlando, FL

Automatic Annotation of Urban Environments to Support POL Behavior Introduction Pattern of Life (POL) in Urban Environments How can we create more compelling simulations of human activity in urban environments? How can we avoid manual scripting or programing the background activities that are common to urban life? As we develop new urban environments for simulation and training, do we have to start with a clean slate? In this presentation we will address these issues and show how urban environments can be automatically generated to support advanced POL activity.

Automatic Annotation of Urban Environments to Support POL Behavior What Is Pattern of Life Anyway? AI entity behaviors that autonomously perform daily background “life” activities within an environment. –Driving to work and parking a car, walking from car to office, enter office building. –Leave office building for lunch, walk to lunch place, meet with others, go into restaurant. –Leave restaurant in group, individually return to place of work. –Leave work, walk to car from office, drive car from parking garage. Using spatial cues, this part of the simulation should be autonomous and tailored to the specific environment

Automatic Annotation of Urban Environments to Support POL Behavior Information Required to Support POL Locations and the properties of places in the environment Spatial organization of the urban terrain for appropriate routing of pedestrians and vehicles Properties need a rich set of spatial attribute descriptors. –Types of buildings based on function: residential, commerial, administrative, parking lots, etc. –Types of open areas based on human use: parks, soccer fields, stadiums, lakes, open space. etc. This information can be generated from: –the geospatial source data used to build the 3D environment –or automatically generated based upon urban density statistics. –or interactively added to source data prior to 3D generation.

Automatic Annotation of Urban Environments to Support POL Behavior Urban Environments Urban spatial organization varies widely depending on area of the world However, there are common themes of human geography and, in particular, daily human activity. Currently background POL activity is manually scripted, when it is available at all. AI must infer spatial relationships and content from the game geometry without benefit from other semantic information. Our goal is to provide a correlated attributed model that various AI systems and Game Developers can use directly.

Automatic Annotation of Urban Environments to Support POL Behavior Generation of Annotated Environments for POL Make minimal assumptions about available source data –Road centerline and building footprints are generally available for most modeling and simulation tasks Derive all other POL features from input source data –Use a combination of computational geometry and image processing –Minimize human interaction to post processing checks Make annotated urban environment generation an integral part of runtime database creation process.

Automatic Annotation of Urban Environments to Support POL Behavior Annotated Urban Environments Process Flow

Automatic Annotation of Urban Environments to Support POL Behavior Some Key Urban Annotation Primitives Pedestrian RoutingVehicular Routing Sidewalk areas Crosswalk areas Buildings Building entrances Road areals Road centerlines Road intersection area Intersection points Parking areas Parking curb cutouts

Automatic Annotation of Urban Environments to Support POL Behavior Building an Integrated Spatial Network Pre-computed relationships vs. derived spatial relationships crosswalk 42 connects sidewalk crosswalk 42 crosses road surface 84 sidewalk 37 is adjacent to parcel 54 parcel 54 contains buildings building 101 as entrances at road 14 intersects road 37 at intersection 99 Support entity reasoning and navigation at a high level of spatial abstraction

Automatic Annotation of Urban Environments to Support POL Behavior Software Support for Annotated Environments Develop an application programming interface Use API support to abstract (hide) underlying spatial data content and organization. Levels of API support: Access to each thematic layer: basic geometry and attribution Access to all layers: specify and limit to spatial area of interest Access to functional and spatial relationships between urban objects within and across layers

Automatic Annotation of Urban Environments to Support POL Behavior Generation of an Urban Semantic Description

Automatic Annotation of Urban Environments to Support POL Behavior Complex Urban Environment (360K buildings)

Automatic Annotation of Urban Environments to Support POL Behavior Building 2D Footprint Layer (Input Data)

Automatic Annotation of Urban Environments to Support POL Behavior Building Parcel Source Layer (Input Data)

Automatic Annotation of Urban Environments to Support POL Behavior Road Centerline Source Layer (Input Data)

Automatic Annotation of Urban Environments to Support POL Behavior Road Junction Areas (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Street Islands (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Sidewalks (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Pedestrian Crosswalks (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Parking Areas (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Parking Access Points (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Building Entrances (Derived)

Automatic Annotation of Urban Environments to Support POL Behavior Example Area 1

Automatic Annotation of Urban Environments to Support POL Behavior Example Area 1 (Phototexture)

Automatic Annotation of Urban Environments to Support POL Behavior Example Area 1 (Source Data Layers)

Automatic Annotation of Urban Environments to Support POL Behavior Example Area 1 (Source and Phototexture)

Automatic Annotation of Urban Environments to Support POL Behavior Example Area 1 (POL Derived Layers)

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 2 (Angled Road)

Automatic Annotation of Urban Environments to Support POL Behavior Phototexture Detailed Area 2 (Angled Road)

Automatic Annotation of Urban Environments to Support POL Behavior Source Data Detailed Area 2 (Angled Road)

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 2 (Angled Road)

Automatic Annotation of Urban Environments to Support POL Behavior POL Generated Detailed Area 2 (Angled Road)

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 2 (Angled Road) in VBS2

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 2 (Angled Road) in VBS2

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 3 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior Phototexture Detailed Area 3 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior Source Data Detailed Area 3 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 3 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior POL Generated Detailed Area 3 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 3 in VBS2 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior Detailed Area 3 in VBS2 (Median Road)

Automatic Annotation of Urban Environments to Support POL Behavior Parking Area

Automatic Annotation of Urban Environments to Support POL Behavior Parking Area (Zoom)

Automatic Annotation of Urban Environments to Support POL Behavior Parking Area and Cutout in VBS2

Automatic Annotation of Urban Environments to Support POL Behavior Parking Area and Cutout in VBS2

Automatic Annotation of Urban Environments to Support POL Behavior Road T-Junction

Automatic Annotation of Urban Environments to Support POL Behavior Road T-Junction Zoom

Automatic Annotation of Urban Environments to Support POL Behavior Road T-Junction in VBS2

Automatic Annotation of Urban Environments to Support POL Behavior Road T-Junction in VBS2

Automatic Annotation of Urban Environments to Support POL Behavior Preliminary Results: Annotated Environments Working with SimCentric and Bohemia Interactive VBS with SimCentric Urban Ambience “add-ons” Use TerraTools 4.0 to generate correlated: –VBS2 maps and multi-tile.pbo –POL annotated data layers POL data layers are loaded into Urban Ambience Current default behaviors include: –Limiting walking to sidewalks and avoid street areas –Crossing roads within crosswalk areas –Entering and exiting buildings are door locations. –Milling activity within defined park and open areas

Automatic Annotation of Urban Environments to Support POL Behavior Entity Routing Without Annotated Environment

Automatic Annotation of Urban Environments to Support POL Behavior Entity Routing Without Annotated Environment

Automatic Annotation of Urban Environments to Support POL Behavior Entity Routing With SimCentric “Urban Ambience”

Automatic Annotation of Urban Environments to Support POL Behavior Entity Routing With SimCentric “Urban Ambience”

Automatic Annotation of Urban Environments to Support POL Behavior Crossing and Loitering Sequence

Automatic Annotation of Urban Environments to Support POL Behavior Crossing and Loitering Sequence

Automatic Annotation of Urban Environments to Support POL Behavior Crossing and Loitering Sequence

Automatic Annotation of Urban Environments to Support POL Behavior SimCentric UPOL Editor (Office Buildings - Blue)

Automatic Annotation of Urban Environments to Support POL Behavior SimCentric UPOL Editor (Restaurants - Purple)

Automatic Annotation of Urban Environments to Support POL Behavior SimCentric UPOL Editor (Residential - Pink)

Automatic Annotation of Urban Environments to Support POL Behavior Ongoing work Refine generation of annotated urban environments –Additional attention to structure semantic attribution (human) –Add segment definitions for roads (vehicular navigation) –Add traffic signs and lights using derived intersection areas –Add urban clutter (hydrants, garbage bins, newspaper box, etc.) –Work with additional urban datasets Work with Serious Game and AI middleware developers to flesh out requirements for Annotated Urban Environments

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh Source Data – (Buildings)

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh Source Data – (Roads)

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh Source Data – (Parks)

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh Source Data – (Drainage)

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh Annotated Environment – (All Layers)

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh POL Environment in VBS2 – Overview

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh POL Environment in VBS2 – Overview

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh POL Environment in VBS2 – Overview

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh VBS2 – Gateway Center

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh VBS2 – Gateway Center

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh in VBS2 – Street View

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh in VBS2 – Clutter Models

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh in VBS2 – Clutter Models

Automatic Annotation of Urban Environments to Support POL Behavior Pittsburgh in VBS2 – Bus Shelter

Automatic Annotation of Urban Environments to Support POL Behavior Conclusions Annotated urban environments can be efficiently produced as an integral part of the serious game generation process They can be used to support improved AI behaviors particularly to support background urban “pattern of life” We believe they will have particular impact in making urban environments look less sterile and in supporting significantly more compelling training scenerios. Questions or comments?

Automatic Annotation of Urban Environments to Support POL Behavior TerraSim, Inc. One Gateway Center, Suite Fort Duquesne Blvd. Pittsburgh, PA Visit us at: