cesiumjs.org Patrick Sean

cesiumjs.org Patrick Cozzi @pjcozzi Sean Lilley @lilleyse
Welcome, thanks everyone for coming! My name is Patrick Cozzi and this is Sean Lilley. We are 3D developers at AGI, the company that started Cesium. Today, we're going to talk about what I believe will be the biggest thing in 3D geospatial since Cesium, and will likely be bigger than Cesium itself: 3D Tiles. We are at the start of a new era of streaming 3D geospatial data, and I hope this talk spurs your creativity. cesiumjs.org Patrick Cozzi @pjcozzi Sean Lilley @lilleyse

Today Motivation Examples Principles Tiling algorithms Tile payloads
Declarative styling Future work Bentley ContextCapture Image from Bentley ContextCapture 3D Tiles export,

Sean Lilley Cesium Developer

Patrick Cozzi Started Cesium Books on Virtual Globes, WebGL, OpenGL
Teach Computer Graphics Contribute to Open Standards and Formats Open-Source Geospatial Community Service

1. Motivation Image from the NASA International Space Apps Challenge 2013,

1.1 A brief personal history
2011 – Streaming imagery 2008 – HLOD thesis 2014 – Point clouds 2016 – Almost done 2015 – 3D Tiles birth

2008 – MS Thesis: Visibility Driven Out-of-Core HLOD Rendering
I was interested in rendering massive 3D buildings in 2005 even before grad school at the University of Pennsylvania. I remember mentioning it in my grad school application. My interest came after implementing a quadtree for view frustum culling for VPF (Vector Product Format) for AGI’s STK desktop app, and reading the book Real-Time Rendering. Fast-forward a few years, and I started a master’s thesis on rendering 3D buildings using Hierarchical Level of Detail (HLOD). As work progressed, it became clear that the approach was applicable to lots of massive models, not just 3D buildings, hence the final generic thesis title. Screenshots Far left: Pompeii Top right: colorized nodes Bottom Left: with view frustum culling Right: with view frustum and occlusion culling MS Thesis:

2011 – Streaming imagery in Cesium
When we started Cesium, it was just a textured ellipsoid. When we added streaming imagery, I helped with the design (left), but Dan Bagnell actually implemented it. Kevin Ring and Scott Hunter then re-implemented it when they added streaming terrain. The diagram here doesn’t tell the whole story of today’s implementation, but it is surprisingly not completely wrong. The screenshot to the right is the earliest screenshot of Cesium’s streaming imagery that I know of. Note that it predates the Cesium logo and even Cesium brand, “Geoscope” is in the url. Screenshot from

2014 – Point cloud prototype
In 2014, Tom Fili and I prototyped some point cloud streaming in Cesium using HLOD and Layered Point Clouds LPC… LPC: Actual screenshot here is from 3D Tiles, not our prototype.

2015 – 3D Tiles We never actually finished or released the point cloud prototype, but the experience was useful. We shifted our attention to streaming 3D buildings. Given all our experiences along the way – my initial MS thesis, imagery, terrain, point clouds – it was obvious that that a core, but flexible, approach could be taken, and 3D Tiles were born. Screenshot of CyberCity 3D buildings for Canary Wharf. Demo:

1.2 Industry trends The motivation for 3D Tiles was more than “scratching an inch.”

Ease of distribution 2011: WebGL – plugin-free, cross-browser, cross-platform, cross-device. Enables widespread distribution. 2011/2012: Cesium – open-source, early WebGL adopter. Facilitated widespread Cesium adoption. 2014: Google Earth plugin deprecated. Accelerated Cesium adoption. Top-left image from Bottom image from

Cost and availability of supply
Open data policies Crowdsourcing Open data policies – federal, state, and local governments. Terrain, imagery, point clouds, vector data, etc. Even things like bike thefts and red-light cameras. Crowdsourcing – OSM, OpenTreeMap. Companies like Mapbox contributing. Government data, e.g., NYC buildings, often integrated. Ease of data acquisition Cost of data acquisition nationalmap.gov.au

Cost and availability of supply
Ease and cost of data acquisition fairbanksfodar.com Phantom 2 Vision+ Ease of data acquisition – individuals can fly planes, fly drones, and capture point clouds with their phones. Cost of data acquisition – A startup can launch imaging satellites. It no longer takes a 900 million dollar satellite. planet.com For more examples, see

easy distribution + large, low cost supply = disruption
Industry trends easy distribution + large, low cost supply = disruption (pardon the business-ish speak)

Why haven’t we disrupted 3D yet?
Industry trends Why haven’t we disrupted 3D yet? Something is missing…

Massive heterogeneous
3D geospatial datasets and other 3D engines

3D geospatial datasets and other 3D engines Enter 3D Tiles. 3D Tiles are an open specification for streaming massive heterogeneous 3D geospatial datasets. That is a mouthful, but each of those words is important. 3D Tiles provides the flexibility needed for the overlay but diverse 3D use cases. I alluded to this at Web3D in 2014: And presented some of the technical approaches at MIT a bit later in 2014: Open specification for streaming massive heterogeneous 3D geospatial datasets

3D geospatial datasets and other 3D engines // Complete Cesium code example to load a 3D Tiles tileset var viewer = new Cesium.Viewer('cesiumContainer'); viewer.scene.primitives.add(new Cesium.Cesium3DTileset({ url : 'path-to-tileset.json' })); The Cesium API is easy to use; just point it to the 3D Tiles tileset that you create. There’s more API to interactive with individual features in the tileset and use declarative styling.

Every building in Manhattan to every satellite in space
1.1 million buildings in New York City Point cloud of the Church of St Marie at Chappes, France. High resolution terrain and imagery Trees in Philadelphia 3D vector data Being open source helped Cesium achieve wide-spread adoption across many industries with many use cases. Cesium is now used for every building in Manhattan to every satellite in space...literally. Cesium, of course, does the classic streaming of high-resolution terrain and imagery, which extends naturally enough from 2D. However, our users want to fully take advantage of 3D and stream millions of buildings, perhaps even textured buildings, millions or billions of points, models that can be instanced like trees, and fully 3D vector data, like corridors and polygons, polylines, and points that conform to terrain.

Foreshadowed at Web3D 2014 I suggested this at Web3D 2014 in the Geospatial information on the Web Workshop organized by Mike McCann. See

3D Tiles in the ecosystem
KML, GML, COLLADA, LAS… WMTS, … 3DPS convert to 3D Tiles for streaming massive datasets preserve attributes individual 3D models can be payload in a 3D Tiles tile data discovery payload can be 3D Tiles 3D Tiles complement existing 2D standards WMS logo from CityGML logo from massive 3D datasets with attributes and styles optimized for runtime streaming and other 3D engines

2. Examples

3D Tiles Showcases

Federal Office of Topography swisstopo
1.5 million buildings with 3D Tiles Buildings clamped to terrain Separate terrain and imagery layers 22.2 GB KML/COLLADA converted to 1.5 GB 3D Tileset (0.97 GB gzipped) KML + COLLADA Source data Uncompressed: 22.2 GB Compressed: 4.95 GB 3D Tiles: Uncompressed: 1.50 GB Compressed: 0.97 GB Work in progress. Data thanks to the Federal Office of Topography swisstopo!

NYC: CityGML and OSM ~1.1 million buildings Mouse over highlight
OSM extract from Mapzen: CityGML from NYC: CityGML source: GB, 20 files, 1,083,437 buildings 3D Tiles: GB across, 7,419 files Without position quantization ~1.1 million buildings Mouse over highlight Dynamic styling with per-building attributes Separate imagery layer 12.84 GB CityGML converted to 1.85 GB 3D Tileset (727 MB gzipped) CityGML OSM extract .json / COLLADA .json / COLLADA See

Per-building attributes Display Style
Stream 100K+ buildings Mouse over highlight Per-building attributes Display Style Combine with terrain, imagery, and vector data Washington, DC: Over 135,000 structures Use additive refinement so the “most important” buildings are streamed first and never replaced. Works well for a large number of simple models – would not work for complex models. 3D Tiles provides the flexibility to create an optimized data structure based on the input dataset. .csv / COLLADA See

Per-building attributes Display Dynamic Styling
Textured buildings Mouse over highlight Per-building attributes Display Dynamic Styling Combine with terrain and vector data - awesome video on their website 5D Smart CityFeatures Video: Above: Chicago Annotation and measurement tool using Cesium entities. See

Derive LOD from CityGML LOD Full access to CityGML semantics
Textured buildings Translucent windows Derive LOD from CityGML LOD Full access to CityGML semantics Point Clouds Combine with terrain and imagery ~540K full textured LOD2 buildings. Create a three-level tree (TMS levels 15-17) with different LODs and texture resolutions. Screenshot: Berlin Dataset: Berlin 3D city model 539,481 fully textured LoD2 buildings 890 km² digital terrain model Approx. 370 LoD3 buidings and 3 LoD4 buildings (interior details) City trees (SolitaryVegetationObject), bridges, S-Bahn stations, … Data volume 16.6 GB CityGML file (XML unzipped) 19.4 GB texture files 3D Tiles / B3DM files for the entire model Three TMS layers with different LoDs and texture resolutions Level 15: LoD2 only and low-res textures (2.54 GB) Level 16: LoD2 only and mid-res textures (4.22 GB) Level 17: LoD2-4 and high-res textures (19.5 GB) Demo (with just open data): Just presented Using glTF for streaming CityGML 3D City Models here at Web3D

Top right: select wall surface of the CityGML building
Bottom left: show building CityGML attributes Screenshot: Berlin

ContextCapture ContextCapture reality mesh export
Buildings, terrain, trees, etc. Publish data and app to the Web Overlay vector data See See

High-resolution terrain, buildings, and imagery Runtime annotation
TIN See

Separate tileset each for buildings, trees, windmills, etc.
Wind park planning Separate tileset each for buildings, trees, windmills, etc. 3DPS for data discovery Separate tileset each for buildings, trees, windmills, etc. Fraunhofer IGD: Institute for applied research in Visual Computing which focuses among others on the visualization and management of geoinformations Applications: Screenshot: Visualize plannings of e.g. windparks to local communities and to stakeholder of the infrastructure project Visualize huge city models Smart City plattforms, etc. Problem: Currently the citymodels and all other objects are added as gltf models Very basic (self implemented) streaming capabilities 3D-Tiles is currently implemented to have: Proper streaming capabilities Access to metadata Instancing of objects (trees) Visualization pointclouds

3. Principles Right image from

How much geospatial data
How much geospatial data? Terabytes is not uncommon NVIDIA GeForce 980 GTX – 4 GB Out-of-core algorithms are required.

A few examples 2006 – Google: 70 TBs of compressed satellite imagery in BigTable Oregon Department of Geology and Mineral Industries Lidar Program Data: 290 billion points 2012 – Microsoft: 302 TBs of imagery 2014 – MapBox adds 1 million NYC buildings to OSM Google - Oregon - Microsoft - MapBox -

2D tiling has limited use in 3D
OK for imagery OK-ish for terrain What about 3D buildings, point clouds, vector data, and massive models? In 2D, tiling is usually done with a quadtree that is uniformly subdivided in longitude and latitude, e.g., TMS. This makes sense for imagery that is evenly distributed. 2D Level of detail is always a top-down view where only one level is visible at a time. Image from virtualglobebook.com

2D tiling for terrain and imagery

2D tiling has limited use in 3D
Sub-optimal subdivisions for non-uniform datasets No 3D subdivision, e.g., for point clouds Doesn't easily handle objects that overlap two tiles 3D requires multiple LODs in the same view Need error metric for LOD selection Need to avoid cracking visual artifacts In 3D, 2D tiling is OK for imagery and OK-ish for terrain, but * Leads to sub-optimal subdivisions for non-uniform datasets * Doesn't provide fully 3D subdivision like what is needed for point clouds * Doesn't easily handle, for example, when a 3D building overlaps two tiles In 3D, things also become complicated because multiple LODs can be visible in the same view; with higher LODs near the viewer and lower LODs farther away. We need an error metric to select the right tiles and ways to avoid visual artifacts, called cracking, where tiles from different LODs meet.

The foundations for 3D tiling were created by
Graphics research Movie industry Game industry 3D Tiles brings these techniques to geospatial

Hierarchical Level of Detail (HLOD)
visit(tile) { if (computeSSE(tile) <= pixel tolerance) render(node); } else foreach (child in tile.children) visit(child); HLOD is the foundation for streaming all geospatial dataset types. [Cozzi08] - [Ulrich02] - Images: [Cozzi08], Code: [Ulrich02]

Hierarchical Level of Detail (HLOD)
Easy to add culling. Exploit spatial coherence to discard entire sub-trees. View frustum Occlusion / horizon / fog Backface Need tight bounding volume Can also exploit frame-to-frame temporal coherence. Tight bounding volume is also better for SSE. [Ring13.a] [Ring13.b] Horizon culling: [Ring13.a], [Ring13.b]

3D Tiles use case Massive datasets Heterogeneous datasets
Tiling, LOD, cache management culling, batching, instancing, compression Heterogeneous datasets Data fusion, common algorithms Non-uniform datasets Require a variety of spatial data structures Interaction and styling Per-feature properties/attributes/metadata and uniquely style individual features When we look at these use cases, some components are very different, but many things are similar. All of these use cases have massive datasets that are too big to fit into memory or download in their entirety so they require view-dependent streaming, with techniques such as: * Tiling to create a spatial data structure for use with Hierarchical Level Of Detail (HLOD) * These includes selecting the right LOD for a given view, efficiently culling tiles that are out of view, and managing a client-side tile cache. * Features, e.g., individual models, should be batched together or instanced so they are rendered efficiently with WebGL. * And the tile payload should come across the wire compressed in such a way that it can be efficiently decompressed. Many of these use cases have non-uniform data, e.g., buildings are not individually distributed throughout a city and not all 3D building models have the same number of triangles. So we would like the ability to support a variety of spatial data structures in a general manner. With the exception of terrain and imagery, being able to interact with and style individual features is key for most users. We want to know what 3D building or tree is clicked, perhaps display per-feature metadata, and then style each feature based on the metadata. These different types of data bring a lot of value when combined, e.g., buildings (batched models) and trees (instanced models) on terrain (TIN).

3D Tiles Spec Software ecosystem
Spatial data structure defined in JSON Tile formats: binary with embedded JSON Ready to render Declarative styling In progress on GitHub Software ecosystem Cesium implementation in progress Several significant adoptions Concretely, what are 3D Tiles? Two things: a spec and an open-source implementation in Cesium. The spec is in Markdown on GitHub. It defines a spatial data structure in JSON. The spatial data structure defines how the tileset is organized. It also defines tile payload formats for a few types of tiles such as batched 3D models and point clouds. The 3D model tile formats use glTF, the GL Transmission Format, which is an open-standard from Khronos for runtime 3D models. Khronos is the group behind many graphics formats such as WebGL, OpenGL, and COLLADA. I contributed significantly to the glTF spec. By utilizing glTF, we are able to take advantage of the glTF loader in Cesium and the growing ecosystem of open-source glTF pipeline tools. Finally, the spec defines a declarative styling language, which is basically JavaScript expressions that have access to per-feature attributes.

3D Tiles Spatial data structure supports many tiling approaches
Server/tool can decide what is optimal for a dataset Client/runtime traverses generic spatial data structure Using techniques that are common in the movie and game industry, 3D Tiles define a general spatial data structure that can be used to create many different hierarchical data structures. The key insight is that the client is generic and the same code can traverse any of these data structures because traversal relies on the property that a tile's children are inside the tile's bounding volume. This is called spatial coherence. A converter/server can now tile up a dataset in a way that is optimal for that dataset. For example: * In the top-right, there is a kd-tree with non-uniform splits, perhaps based on the cost of rendering features in each tile * In the middle, there is an octree perhaps for rendering a point cloud * In the bottom right, there is a loose non-uniform quadtree, which is a subdivision strategy we use for one type of 3D buildings dataset. There is not a one-size-fits-all subdivision strategy; the converter/server does what it thinks is optimal for the dataset, then the client handles it in a general fashion.

Tilesets and tiles tileset.json Tiles
Defines spatial layout for the tileset Points to external tiles Tiles Contain the tile’s actual content Binary + embedded JSON Now I'm going to go into more detail about the 3D Tiles format. The design is separated into two parts. First is the tileset.json file which defines the spatial layout of the tiles and points to the external tile files. The tiles themselves store the actual data and are represented as a binary format. We can also apply optional compression methods to the tiles, such as Web3D Quantized Attributes which can encode position data with fewer bits by storing the deltas relative to a center position. On the side is an example of what a tileset might look like as a folder structure. In this example, the tileset is for Seattle. The tileset.json references all the b3dm files, which are small portions of the city. And this is just one example, your tileset layouts may look different. Other compression methods: Open3DGC, oct encoding

3D Tiles Now that I've covered the design and formats, I'm going to talk about the possibilities of using 3d tiles for creating spatial data structures.

tileset.json Tileset metadata
{ "properties": { "Height": { "minimum": 1, "maximum": 241.6 } }, "geometricError": , "root": { "boundingVolume": { "region": [ , , , , 0, 241.6 ] "geometricError": , "content": { "url": "0/0/0.b3dm", "boundingVolume ": { /* ... */ } "children": [..] Tileset metadata Geometric error if tileset is not rendered Bounding volume of root tile Geometric error if only root tile is rendered url to tile content Bounding volume of content Children of this tile

Bounding volumes 3D Tiles allows multiple bounding volumes so you can pick the right one to tightly fit your data Spheres: 599 tiles OBBs: 307 tiles See See

Spatial coherence Child’s content must be in parent’s bounding volume
Does not imply child’s bounding volume is in parent’s bounding volume See

Spatial data structures
Grid Buildings in Cambridge arranged in a grid layout. Each red box is a tile. The outer red box is the bounding volume for the tileset. But no random access to grid cells like most grid approaches; could be an explicit tiling scheme; see future work section at the end of the slides

Quadtree Tight-fitting Quadtree We can be flexible with the bounding volumes by finding tight bounding volumes around the child tiles. Bottom left tile tightly wraps around its content (blue buildings).

Adaptive loose-ish Quadtree Another helpful subdivision technique is the loose quadtree, where neighboring tiles can overlap in order to avoid splitting models along tile edges. The green building in the middle would have been split in half, but the left bounding volume extends into the right bounding volume to avoid that. Overlapping neighbor tiles is just fine by the spec.

Octree K-d tree While most of the subdivision techniques we’ve presented have been 2D based, you can also define 3D subdivisions.

Geometric error How many “meters wrong” is the image of a tile’s children (and their sub-trees) are not rendered? At runtime, used to compute screen-space error based on view parameters

Geometric error: terrain example
[Ring13.c] - [Klein96] - [Garland95] - From [Ring13.c] “Ideally, epsilon is the Hausdorff distance of the tile relative to the original, full-detail terrain in the region covered by the tile. Imagine we look at all of the points on the surface of the simplified mesh. For each of those points, we measure the distance from that point to the closest point on the original mesh. Then we do the same in reverse. For each point on the original mesh, we find the distance to the closest point on the simplified mesh. The greatest of those distances is the Hausdorff distance, and is an accurate measure of error in the simplified representation.” “Then, we follow the method described by Michael Garland in his paper “Fast Polygon Approximation of Terrains and Height Fields.” Initially, only the corner vertices are active. The rest are inactive. We compute a Delaunay triangulation of the simple mesh containing these four vertices, and then determine which of the two triangles contains each of the remaining, inactive vertices. If a vertex is on a triangle edge, it is assumed to be contained by both triangles. Then, we compute the distance from each vertex to the triangle that contains it and insert each triangle into a priority queue, sorted by the distance to its most distant vertex. We select the most distant vertex in the highest-priority triangle, activate it, and redistribute the vertices in the original triangle to the three new triangles that result from inserting the vertex” “The process repeats until the most distant vertex is closer to its containing triangle than the maximum geometric error allowed at this level. The error metric used here is not the Hausdorff distance. We’re only measuring the distance from the original vertices (not triangles!) in the original mesh to the triangles in the simplified mesh. This shortcut saves us a lot of processing time, and so far I haven’t seen any objectionable visual artifacts as a result of this choice.” Images: [Ring13.c], Algorithm: [Garland95]

Refinement: replacement vs. additive
Replacement O(nlogn) tileset Additive O(n) tileset visit(tile) { if (computeSSE(tile) <= pixel tolerance) render(node); } else foreach (child in tile.children) visit(child); visit(tile) { render(tile); if (computeSSE(tile) > pixel tolerance) foreach (child in tile.children) visit(child); } Replacement is like downloading a low-resolution image, then higher-resolution images. Additive is like downloading the most important pixels in an image first. When you zoom in, pixels may be rendered that are not needed. The two refinement choices are replacement and additive refinement. Replacement refinement is the traditional 2D approach where lower LOD tiles are replaced by new higher resolution tiles, and because of that we need to wait for all the child tiles to download first. Additive refinement instead renders new tiles on top of existing tiles, which allows for child tiles to render right as they are loaded in. LPC: [Gobbetti04]

Content bounding volume
Used to optimize culling in additive refinement. The tile’s BV encloses the content of all descendent tiles. The tile’s content BV encloses just the tile’s content.

Additive refinement http://youtu.be/1useSwpuM8w
Zoom in and root tile with the "most important" buildings appear. Freeze frame to take a closer look. "Important" buildings are tall and/or have a large footprint. Lots of other heuristics could be used too; it doesn't matter to the runtime engine. Zoom in a bit closer and some of the root node's children appear. Each tile is a single WebGL draw call, has a bounding volume used for view frustum culling, and has a geometric error that is used to know when to stop refining down the tree. No individual building LOD This is additive refinement; internal tiles, not just leaf tiles are rendered This is a non-uniform loose-ish quadtree subdivision

Tileset of tilesets 3D Tiles also supports a tileset of tilesets. If the root tile is your global data set, each smaller tileset.json might be a different city. The tilest is only streamed in once it is in view.

Tile Formats Small, fast to download
Compression Fast to decode, create WebGL resources “Ready to render” with batching or instancing Include per-feature properties/attributes/metadata Utilize glTF Benefit from glTF’s open-source ecosystem

glTF GL Transmission Format
Efficient open-standard runtime 3D model format by Khronos Khronos also publishes COLLADA, WebGL, OpenGL, … JSON + binary geometry/textures/animations Strong brand and open-source community Converters Validators Pipeline tools 3D engine importers Open-source glTF pipeline tools: khronos.org/gltf

3D Tiles tile formats Batched (.b3dm) Instanced (.i3dm)
Point Cloud (.pnts) Vector (.vctr) Composite (.cmpt) Add your own!

Tile formats Batched (.b3dm) Left: Vricon
Right: Canary Wharf by CyberCity3D. Note that the geometry is batched, but each building is accessible for interaction/styling/metadata.

Tile formats Instanced (.i3dm)
Left: 57,714 trees using instanced tiles (i3dm) rendered with WebGL instancing. OpenTreeMap data: Right: Berlin by virtualcitySYSTEMS

Tile formats Point Cloud (.pnts)

Tile formats Vector (.vctr)
Work in progress. Data thanks to the Federal Office of Topography swisstopo!

Tile formats Composite (.cmpt) Image from virtualcitySYSTEMS.
Composite tile allows storing multiple tiles in one tile so pipeline tools can decide between one tileset for all source datasets, or one tileset per dataset type (or any combination).

Declarative styling { "show" : "${Area} > 0", "color" : {
"conditions" : { "${Height} < 60" : "color('#13293D')", "${Height} < 120" : "color('#1B98E0')", "true" : "color('#E8F1F2', 0.5)" } Here's an example of declarative styling. Just a few lines of code styles these buildings in Seattle. Each building's height maps to a color ramp where the tallest buildings are also translucent. For illustration, it also only shows the buildings with an area greater than zero. Finally, in the top right a building was highlighted on mouse over - that is not part of declarative styling, but is an important use case for interactivity.

4. Tiling algorithms I’m going to go more in depth about the different tiling algorithms to chose from when generating 3D tilesets and what algorithms make sense for which kinds of data.

Discrete Level of Detail
This first example isn’t even much of a tiling algorithm as it is a way to present different levels of detail in a tileset. Many game developers are familiar with the concept of discrete level of details for 3D models. This concept can be easily applied to 3D tiles.

Discrete LOD { "asset": { /* ... */ }, "geometricError": 500, "root": { "transform": [ /* ... */ ], "boundingVolume": { "box": [ /* ... */ ] }, "geometricError": 100, "refine": "replace", "content": { "url": "dragon_low.b3dm" }, "children": [ "geometricError": 10, "url": "dragon_medium.b3dm" "geometricError": 0, "url": "dragon_high.b3dm" } ] The tileset is straightforward. You have a hierarchy of 3 tiles, each referencing a mesh with a different level of detail. The root node is the lowest LOD, followed by the medium and high LODs. Once the screen space error is met, the tile is just replaced with the higher LOD tile. Sample tilesets:

Discrete LOD Quick and dirty. Easy Popping artifacts
Just decimate model at different LODs Popping artifacts Future work: alpha blending or vertex morphs Downloads duplicate data at each LOD True of replacement refinement in general Future work: just send deltas Not appropriate for massive models LOD selection based on screen-space error (from geometric error). Not appropriate for massive models since the entire full-resolution model is in memory when zoomed in close, but a great starting point for learning about massive models.

Hierarchical Level of Detail

HLOD with replacement refinement
{ "asset": { /* ... */ }, "geometricError": 1024, "root": { "boundingVolume": { "region": [ /* ... */ ] }, "geometricError": 512, "refine": "replace", "content": { "url": "0/0/0.b3dm" }, "children": [{ "geometricError": 256, "content": { /* ... */ }, "children": [ /* ... */ ] }, { // ... ] } Geometric error is the same as each level and half that of the parent in this example, but that is not at all required. Siblings can have different geometric errors, and children geometric error just needs to be <= parent’s. In this example the date is split into a uniform quadtree which results in the evenly split geometric errors.

Hierarchical LOD is utilized by massive models. While not shown in this picture, the root tile would be a highly decimated version of the original 3D model. Then as you zoom in the child tiles begin to load, which now have more detail. Ultimately the union of all the leaf nodes is the original 3D model. This screenshot is early work into our massive model implementation of splitting up a 3D model into chunks.

Same tileset but only rendering the tile bounding volumes that define the spatial hierarchy

And the blue boxes represent the bounding volumes of the content of each tile.

Appropriate for massive models Tile with quadtree, octree, … Union of leaf tiles is original model Root tile is drastically decimated model Interior tiles are decimated subset of model with decreasing geometric error farther down the tree

Popping artifacts Future work: alpha blending or vertex morphs Cracks – often use skirts to fill Downloads duplicate data at each LOD

HLOD with additive refinement
Point Clouds – using the same Cesium runtime code as 3D buildings (the Octree and Quadtree and just a 3D Tiles tree to Cesium) and a different tile format – points instead of glTF. Zoom out, freeze frame, then zoom in to show just root tile. Unfreeze, then show colorized tiles. Like 3D buildings, this is using additive refinement.

{ "asset": { /* ... */ }, "geometricError": 1024, "root": { "boundingVolume": { "box": [ /* ... */ ] }, "geometricError": 512, "refine": "add", "content": { "url": "0/0/0.pnts" }, "children": [{ "geometricError": 256, "content": { /* ... */ }, "children": [ /* ... */ ] }, { // ... ] }

Additive refinement applied to a city tileset. The largest buildings are part of the root tile.

Then as you get closer new buildings are added on top of the existing buildings.

And so on.

Here’s a bird’s eye view of the same scene. Here you can see the white bounding volume containing several child tiles in red. Non-uniform overlapping quadtree.

Does not download duplicate data Children can be rendered before all siblings are downloaded Does not require decimation Great for lots of simple models, e.g., many building datasets Can render too much when zoomed in close This is contrasted with the Rio demo which uses additive refinement instead.

LOD with imposters

LOD with imposters using custom shader for glTF, could also be .vctr
{ "asset": { /* ... */ }, "geometricError": 100, "root": { "boundingVolume": { "region": [ /* ... */ ] }, "geometricError": 10, "refine": "replace", "content": { "url": "trees_as_3D_imposters.i3dm" }, "children": [ "geometricError": 0, "url": “trees_as_3D_models.i3dm" } ] using custom shader for glTF, could also be .vctr

LOD with imposters Different tree levels have different data
In general, can switch between any representation Point billboard 3D model Point polyline polygon Extruded footprint building with rooftop textured building Low-resolution textured building Highly textured building

Tight-fitting quadtree

Here’s the same concept but with buildings in a city. Interior tiles are empty for culling.

{ "asset": { /* ... */ }, "geometricError": 12, "root": { "transform": [ /* ... */ ], "boundingVolume": { "region": [ /* ... */ ] }, "refine": "replace", "children": [ "geometricError": 0, "content": { "url": “trees.i3dm" } }, // ... ] non-leaf tiles have the same geometric error no content, used for culling leaf tiles have content

Interior tiles accelerate culling Tight subdivision avoids wasted space and deep trees possible with a traditional quadtree

Grid Root tile with many children No interior tiles,
{ "asset": { /* ... */ }, "geometricError": 30, "root": { "transform": [ /* ... */ ], "boundingVolume": { "region": [ /* ... */ ] }, "refine": "replace", "children": [ "geometricError": 0, "content": { "url": “tile1.b3dm" } }, // ... ] Root tile with many children No interior tiles, no accelerated culling

Separate tilesets

Separate tilesets Independently tile optimally for the input dataset
Only stream layers that are visible Alternatively use composite tiles for a heterogeneous tileset Reduces number of requests Less optimal subdivision

https://youtu.be/PgX756Yzjf4
Request Volumes

https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/101
Request Volumes { "asset": { /* ... */ }, "geometricError": 200, "root": { “viewerRequestVolume": { “sphere": [ /* ... */ ] }, "boundingVolume": { "sphere": [ /* ... */ ] "geometricError": 0, "refine": "add", "content": { "url": "points.pnts" } When the viewer is inside this bounding volume, the tile is rendered

Tile expiration

Tile expiration Re-request points.pnts every five seconds {
"asset": { /* ... */ }, "geometricError": 346.4, "root": { "expire": { "duration": 5 }, "boundingVolume": { "sphere": [ /* ... */ ] "geometricError": 0, "refine": "add", "content": { "url": "points.pnts" } Re-request points.pnts every five seconds

https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/99
Tile expiration Client re-requests expired tile and sub-tree Work in progress Duration or explicit date Store in tile so it can change

5. Tile payloads

Tile Format Principles
Same spirit as glTF Utilize open-source glTF ecosystem Fast to download Binary blob (little endian) with embedded JSON (utf-8 string) Compression Fast to decode, “ready to render” Flatten node hierarchy, texture atlases, pre-batch, vertex cache optimize Pre-triangulate Minimize WebGL draw calls Preserve per-feature properties Often served pre-gzipped

Compression Optional quantization for positions
Store x, y, z each as uint16 Stays compressed in GPU memory Decode for free in the vertex shader ~1.5m error in 100,000m scale Tiles get smaller farther down the tree glTF WEB3D_quantized_attributes extension Future Context-aware, e.g., Open3DGC Variable bit based on bounding volume YCoCg for color? What do you suggest? See See See YCoCg, see

Compression Optional oct-encoding for normals and texture coordinates
For lighting: oct16P – 2 bytes for 3D normal Mean error: , max error: degrees For rotation (i3dm): oct32P – 4 bytes for 3D normal Mean error: , max error: degrees Stays compressed in GPU memory Decode for cheap in the vertex or fragment shader Figure from

Tile formats Batched (.b3dm) Instanced (.i3dm) Point Cloud (.pnts)
Vector (.vctr) Composite (.cmpt)

Batched (.b3dm) Pre-batched binary glTF Buildings, terrain, …
Per-feature batchId to identify features in a batch Single draw call per material

Batched (.b3dm) Pre-batched binary glTF
Per-feature batchId to identify features in a batch Example vertex buffer: Positions for all features Normals for all features batchId 0 batchId 1 ... batchId n Points to Batch Table

Batched (.b3dm) At runtime, in a vertex or fragment shader, use batchId to lookup color/show in a texture Fast styling – just change the texel

Batched (.b3dm) batchId points into Batch Table to access per-feature properties, which can be used for Styling Display Web services position: [xyz, xyz, xyz, ..., xyz, xyz, xyz, ...] normal: [xyz, xyz, xyz, ..., xyz, xyz, xyz, ...] batchId: [0, 0, 0, ..., 1, 1, 1, ...] Recall, property min/maxes are stored in tileset.json { "id" : ["unique id", "another unique id"], "displayName" : ["Building name", "Another building name"], "yearBuilt" : [1999, 2015], "address" : [{"street" : "Main Street", "houseNumber" : "1"}, {"street" : "Main Street", "houseNumber" : "2"}] }

Instanced (.i3dm) Single model with slight variations, called instances Trees, traffic lights, bolts, … Efficiently rendered with WebGL instancing

Instanced (.i3dm)

Instanced (.i3dm) Feature table contains per-instance:
Positions (optionally quantized) Rotations (optional, optionally oct-encoded) Scale (optional) batchId (optional)

Instanced (.i3dm) Feature table semantics reference binary data
Example: POSITION_QUANTIZED points to INSTANCES_LENGTH array of uint16[3] Also working on adding general-purpose binary accessors (like glTF accessors) to the Batch Table

Instanced (.i3dm) glTF Use composite tile for multiple models
url to external glTF to share among tiles embed in tile to minimize number of requests Use composite tile for multiple models Also working on adding general-purpose binary accessors (like glTF accessors) to the Batch Table

Point Cloud (.pnts) Still working on this, should stabilize in 1-2 months Follow

Point Cloud (.pnts) Feature Table (not shown in diagram) will contain
Positions (optionally quantized) Color (optional, RGB or RGBA, compression) Normals (optional, optionally oct-encoded) Scale (optional) batchId (optional) Normals for hidden surface removal, lighting, etc. Color compression TBA

Point Cloud (.pnts) At runtime, generate vertex shader for
Point size attenuation Point culling Show/color/size based on declarative styling – executing on the GPU in parallel! Lighting, hidden surfaces, … Edge-preserving blur Future plans, not doing this yet.

Vector (.vctr) Still working on this, should stabilize in 2-3 months
Follow Triangulation only add indices, not vertices – small payload increase Work in progress. Data thanks to the Federal Office of Topography swisstopo!

Vector (.vctr) Feature and Batch Tables like .i3dm and .pnts
Polygons, polylines, points (labels/billboards) + extrusions, Cesium-style geometries Feature and Batch Tables like .i3dm and .pnts Pre-triangulate, ready to render Triangulation only add indices, not vertices – small payload increase

Composite (.cmpt) Combine heterogeneous tiles into one tile Examples
Composites of composites too Examples Buildings (b3dm) + trees (pnts) Multiple tree species (i3dm) Image from virtualcitySYSTEMS. Composite tile allows storing multiple tiles in one tile so pipeline tools can decide between one tileset for all source datasets, or one tileset per dataset type (or any combination).

Composite (.cmpt) Users turn layers on/off a lot?
Separate tilesets Also allows more optimal subdivision Users look at heterogeneous datasets Composite tiles Fewer requests Likely less optimal subdivision

6. Declarative styling

Principles Declaratively describe per-feature: Write expressions using
Show/hide Color (color.rgb) Translucency (color.a) Write expressions using Subset of JavaScript Augmented for styling Access per-feature properties (from the Batch Table) Applying new styles should be fast

Cesium styling API // Applying a style to a tileset var viewer = new Cesium.Viewer('cesiumContainer'); viewer.scene.primitives.add(new Cesium.Cesium3DTileset({ url : ' path-to-tileset.json' })); tileset.style = new Cesium.Cesium3DTileStyle({ "show": true, "color" : "rgb(100, 255, 190)" });

Styling language Built-in Color type with CSS-like constructor functions "color" : "color('#BAA5EC')" // 3 or 6-digit hex, optional alpha "color" : "color('cyan', 0.5)" "color" : "rgb(100, 255, 190)" // also rgba() "color" : "hsla(0.9, 0.6, 0.7, 0.75)" // also hsla()

Styling language Access to per-feature properties using ${}
"show" : "${ZipCode} === '19341'" "color" : "(${Temperature} > 90) ? color('red') : color('white')"

Styling language Built-in RegExp type for string compares
"show" : "(regExp('^Chest').test(${County})) && (${YearBuilt} >= 1970)"

Styling language Use conditions to create color ramps and other functions Execute like an if…else series "color" : { "conditions" : { "(${Height} >= 100.0)" : "color('#0000FF')", "(${Height} >= 70.0)" : "color('#00FFFF')", "(${Height} >= 50.0)" : "color('#00FF00')", "(${Height} >= 30.0)" : "color('#FFFF00')", "(${Height} >= 10.0)" : "color('#FF0000')", "(${Height} >= 1.0)" : "color('#FF00FF')" }

Styling language Create non-visual expressions with meta.
Evaluate per feature: "meta" : { "description" : "'Hello, ${featureName}.'", "volume" : "${height} * ${width} * ${depth}" } var feature = scene.pick(/* screen x, y position */); if (Cesium.defined(feature)) { var meta = tileset.style.meta; console.log("Description: " + meta.description.evaluate(feature)); console.log("Volume: " + meta.volume.evaluate(feature)); }

Cesium styling API Change per-feature properties at runtime
The style is automatically reevaluated // 3D Tiles style: "color" : "(${Temperature} > 90) ? color('red') : color('white')" // In JavaScript using the Cesium API: feature.setProperty('Temperature', 91); // Can also get properties, and get/set color/show feature.getProperty('Temperature'); // 91 feature.color; // Cesium.Color.RED

Cesium styling API Allows custom styles, e.g., to implement another styling language var style = new Cesium.Cesium3DTileStyle(); style.show = { evaluate : function(feature) { return true; } }; style.color = { evaluateColor : function(feature, result) { return Cesium.Color.fromBytes(255, 255, 255, 255, result); tileset.style = style;

7. Future work We need your input!

Explicit tiling schemes
For well known tiling schemes, e.g., quadtree Allow random access, e.g., z/y/x Avoid large tileset.json and tilesets of tilesets Inflexible, requires same tile format throughout tile { "asset": { /* ... */ }, "tilingScheme": "quadtree", "format": "b3dm", "boundingVolume" : { "region": [ /* ... */ ] // could default to global } Tile payloads would may need an extra section for geometric error (if it isn’t implicit) and children contents’ BVs. github.com/AnalyticalGraphicsInc/3d-tiles/issues/92

github.com/AnalyticalGraphicsInc/3d-tiles/issues/102
Time-dynamic tiles Similar to streaming video “keyframe animation for massive models” Introduce a new composite tile (.cmpt) Each tile in temporal composite also has a time stamp Temporal composite (or tileset.json) can point to previous/next tiles in the time series, perhaps like a skip list Future future work: only store deltas time interval times 0..n tiles 0..n previous/next tiles github.com/AnalyticalGraphicsInc/3d-tiles/issues/102

Download multiple children in one request
Fewer files, fewer requests Especially useful for replacement refinement when all sibling tiles need to be downloaded Client could use prefetch strategies Similar to glTF buffer and bufferView "tiles": { "level1-id" : "0/0/0.b3dm" }, // ... "content": { "tile": "level1-id", "byteOffset": 0, "byteLength": 1024 } github.com/AnalyticalGraphicsInc/3d-tiles/issues/9

New tile formats Finish vector (.vctr)
Consider OSM tile, higher-order surfaces Generate geometry at runtime from OSM constructs, e.g., roofs Compute/memory trade-off Terrain Create your own tile format! Client-side many constructs could be pre-triangulated and then cached or instanced. Terrain – perhaps based on github.com/AnalyticalGraphicsInc/3d-tiles/tree/master/TileFormats/OpenStreetMap

Optimize additive refinement
Add spatial data structure to each tile Each tile in the nested spatial data structure has (offset, count) Only render subset of tile that is visible Natural extension to content.boundingVolume github.com/AnalyticalGraphicsInc/3d-tiles/issues/11

Expand declarative styling
Expand standard library, e.g. Boolean inShadow(); JulianDate simulationTime; Refer other features, e.g., color : ramp(palette, distance(${thisFeature.location}, ${anotherFeature.location})) github.com/AnalyticalGraphicsInc/3d-tiles/issues/2

Package tileset in a single file
Could be as simple as a database with tileset.json and rows with key: relative path to tile value: tile contents github.com/AnalyticalGraphicsInc/3d-tiles/issues/89

How can we work together?

Consider 3D Tiles or Cesium as a platform for your research.
Tiling algorithms New tile formats Compression algorithms Rendering optimizations Tell us about it!

We can showcase your work with 3D Tiles or Cesium
We can showcase your work with 3D Tiles or Cesium. Sarah Chow, The Cesium website has over 75 demos

Software Developers and Geospatial Data Wranglers
We’re hiring! Software Developers and Geospatial Data Wranglers cesiumjs.org/jobs

Join us for the Cesium BOF
Tuesday, 10:30am-12:00pm. Room 203B Cesium State of the Union. Patrick Cozzi, Cesium 3D Tiles and OGC‘s 3D Portrayal service standard. Ralf Gutbell, Fraunhofer Making Drone Data Useful in Cesium. Chris Cooper, Propeller Publishing Reality Meshes to Web from Bentley ContextCapture using Cesium 3D Tiles. Makai Smith, Bentley Baking AO in the glTF Pipeline. Gary Li, Cesium Visualizing Geostationary Satellite Orbits in Cesium. Ed Mackey, Analytical Graphics, Inc.

https://github.com/AnalyticalGraphicsInc/3d-tiles
Check out 3D Tiles Contact Thanks to Patrick Cozzi @pjcozzi

cesiumjs.org Patrick Sean

Similar presentations

Presentation on theme: "cesiumjs.org Patrick Sean"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

cesiumjs.org Patrick Sean

Similar presentations

Presentation on theme: "cesiumjs.org Patrick Sean"— Presentation transcript:

Similar presentations

About project

Feedback