Presentation is loading. Please wait.

Presentation is loading. Please wait.

Spatial Database Engine Keith T. Weber, GISP GIS Director Idaho State University.

Similar presentations

Presentation on theme: "Spatial Database Engine Keith T. Weber, GISP GIS Director Idaho State University."— Presentation transcript:

1 Spatial Database Engine Keith T. Weber, GISP GIS Director Idaho State University

2 Todays Topics What is SDE? Why use SDE? SDE Data Structure How is data stored within SDE? DEMO: Meet ArcSDE Professional GDB Enterprise workflow: Versioning and Replication

3 What is SDE? A spatial database engine that works on an RDBMS. Helps to serve geospatial data to clients via a network SDE

4 Why use SDE? Advantages: –Data loss/integrity degradation through versioning –Centralized data management –Enterprise GIS –Geo-spatial data is immediately usable

5 Why use SDE? (contd) Disadvantages –Data management role –RDBMS administration –Capital expenditure

6 To Use SDE…or Not To Use SDE… What will help make this decision? –ROI –TCO –Is this the correct technology for the problem?

7 ArcGIS Data Structures ArcGIS Vector Objects Shape files Coverages Raster Objects Grids Images GDB

8 The GDB Can store tables (just information), vector feature classes, and raster layers

9 Layers and Layer Files All GIS Datasets are considered LAYERs in ArcMap. A LAYER FILE is a file that you save in ArcMap to retain customized settings. –This file refers to the LAYER (shape file, coverage, grid, or feature class) –It displays the data with your saved visualization settings, textual annotation, etc.

10 WorkspacesWorkspaces Arc/Info –Info folder –Geodata sets (coverages, grids, TINs) Collection of ArcView shape files Geodatabases

11 CoveragesCoverages Tic Bnd Arc AAT, PAT

12 GeoDatabasesGeoDatabases Personal File-based ArcSDE Personal ArcSDE Professional (or Enterprise)

13 Personal Geodatabases Uses the MS Access Jet Database engine –Note: Do not open/edit these with MS Access Limitations –2GB (Access) –Only vector feature classes are actually stored inside the Access database –4 users but only one editor –Does not support versioning

14 File-based Geodatabase fGDB Stores vector and raster layers in the file/folder structure. Limitations –Multi-user (max = 10) –1 Editor (no versioning) –Max size is 1 TB RDBMS

15 ArcSDE Personal Uses MS SQL Server Express Limitations –4 GB –Supports versioning/replication but only one editor

16 ArcSDE Professional Geodatabases Uses DB2, Oracle, Informix, SQL Server, etc. No software size limits and unlimited number of users Can accommodate vector and raster data

17 Given all these differences, there are really many similarities

18 Geospatial Data Storage (Vector) Geo-spatial data are stored as Feature classes Non-spatial data are stored as stand- alone tables Vector data is handled by DB2s Spatial Extender. SDE is a broker.

19 Geo-spatial Data Storage (Raster) Two methods –Stand-alone raster data set –Mosaic ArcSDE is not the best solution to store raster GIS data for the Enterprise –Size considerations –Performance issues Raster data is handled by SDE

20 Internal Data Storage Within the DB2 RDBMS –All data is stored within table spaces – referred to by Configuration Keyword. –A Configuration Keyword points to a set of two table spaces: Attribute table space Coords table space

21 Loading Vector Data into a GDB PART 1: Stand-alone feature classes

22 The Spatial Index Grid Uniform grid of square tiles –Like grid reference on a street map Each feature (lakes) referenced by one or more tiles –Envelope of feature determines tiles occupied Spatial Index Key records occurrences of features in tiles –Empty tiles not stored

23 Loading Vector Data into ArcSDE PART 2: Feature classes within a Feature Data Set –First, you need a Feature Data Set –What is a Feature Data Set?

24 A Feature Data Set is: Required to implement Full Topology! What?!

25 Full Topology The spatial relationship among feature classes participating in a topology layer –Must belong to a feature dataset Feature classes share geographic reference system, and spatial domain. –More realistic representation of data

26 A Feature Data Set then… Is an organizational tool used to ensure that all feature classes within it use a common: –Geospatial reference system –Spatial domain

27 Understanding the Spatial Domain Low-precision GDB Based upon LONG INTEGER (32- bit) –What is the domain range of a LONG? High-precision GDB Based upon 64-bit Integer –Covers a geographic reference systems Horizon

28 Fitting the World into a LONG If we express the X,Y coordinates in the familiar Latitude/Longitude system… By whole degrees, we would use: –Latitude (180 units) –Longitude (360 units) This is only % of the 4B space

29 Problems with this approach Resolution to 1 degree is terrible Wastes the capacity of LONG INTEGER

30 What if we use Decimal Degrees? Hold on! Decimals cannot be stored in an INTEGER data type Lets just shift the decimal place to the right by multiplying the coordinate by a scaling factor –e.g., 10 preserves one decimal place, 100 preserves decimal places etc.

31 Fitting the World into a LONG (revisited) By using a scaling factor of 1M, the world would fit nicely into a 3.6B space (theres even a bit left over!) What is the spatial resolution of 1/1M th of a degree? –Approximately 1/10 th of a millimeter!

32 More about the High- Precision GDB Can be pGDB, fGDB, or SDE GDB Uses 64-bit integer to encapsulate the spatial horizon What? –64-bit numbers have a range of 18,446,744,073,709,551,616 –Thats 18 quintillion! –

33 The Spatial Horizon? Essentially, its a spatial domain large enough the contain the entire earth at high- precision

34 Applying this to ArcGIS –Rule #1, use the high-precision GDB model whenever possible. –Why not always?

35 Hints and Tips Optimize the spatial domain by using high-precision GDB Feature dataset If not, set up your low-precision Feature dataset to –Allow for spatial growth –Allow for improved instrumentation

36 ArcSDE Professional Demo –Import a vector data set into ArcSDE

37 The Future…

38 Think about it… Object-relational databases have native geospatial support ArcGIS for Server can make geospatial data available to the Enterprise Do we need an ArcSDE middle-ware? –ArcGIS Spatial Data Server

39 Questions…Questions…

40 Geodatabases in an Enterprise Workflow Keith T. Weber, GISP GIS Director, ISU GIS Training and Research Center


42 Lets Get Started GIS is… –Data-driven –Powerful –Dynamic

43 GIS Data Life Cycle EditionBackupEditValidate Update Metadat a Create Data Change Happens!

44 The Bottleneck Distributing the new edition

45 The Solution Networks and the Internet

46 A New Problem is Born MY version

47 GIS Grows Up! RDBMS –Keep the benefit of network connectivity –Eliminate the problem of MY version –Eliminate the bottleneck –And, change the cycle of events

48 GIS Data Life Cycle Edition Version and/or Replicate Edit Validate: Synchronize or reconcile and post Update Metadata Create Data Change Happens!

49 Backup vs. Versioning Backups and archiving are still critical steps for the enterprise. BUT, not part of the GIS Life Cycle any longer

50 In the Beginning… Backups were made in case we really messed up Edits were made to the original Copies of the clean new edition were distributed

51 Today…Today… The original [parent] is versioned [a child is born] Edits are made to the child, not the parent Clean edits are copied [synchronized or posted] to the parent.

52 Benefits Of This Approach Brainstorm!!! –Minimize downtime –Processes completed within the RDBMS

53 The Role of Backups Data retention and deletion Legal requirements

54 GIS Data Life Cycle…Today Edition Version and/or Replicate Edit Validate: Synchronize or reconcile and post Update Metadata Create Data Change Happens!

55 Questions/Discussion?Questions/Discussion?

56 REPLICATION AND VERSIONING Presentation and Discussion


58 What is Replication? Duplication Copying Mirroring

59 True Replication… Does not need ArcGIS Every RDBMS can be replicated natively However, using ArcGIS to perform the replication –Is easy –Supports GIS workflows better

60 Why Replicate? Enable disconnected editing for: –Performance/load balancing –Network load reduction –Publishing data to subscribers

61 Network Load Reduction The network is a primary bottleneck in a High-Capacity Enterprise Note: capacity refers to how many concurrent users a system can support without loss of performance

62 How Do I Replicate? We will cover this with the hands-on exercise As an overview… –Version the database –Replicate the database –Edit/update –Synchronize changes with the parent

63 So Replication is Versioning No… but replication uses a versioned database

64 What is Versioning? One database Parent edition (tables) remains live/usable Child edition(s) simultaneously edited Roll-up is seamless

65 Versioning: Principal Concepts Edits are stored in Supporting Tables Geographic changes (linework) are stored in Supporting Vector Tables Attribute changes are stored in Supporting Delta Tables.

66 Delta Tables A = Add (insert) D = Delete U = Update (delete existing then add)

67 A Tree is Formed As versions are created and changes are made, a tree grows –Q: What kind of tree? –A: A State Tree

68 Sort of an Upside-down Tree

69 The State Tree Default: state 0 Arthurs Court sub-division [Another] sub-division Tree Trunk Branche s

70 Multiple Versions Multiple versions are allowed –Versions can be based upon location (north edits, south edits), projects (sub- divisions), or other logic decided upon my the GIS Manager. Batch reconcile and post are supported

71 The Day of Reconciliation Arthurs Court sub-division edits have been completed –Time to reconcile –This process looks for conflicts –Once all conflicts have been resolved… –Reconciliation is complete

72 PostPost To roll-up the edits back to the trunk of the state tree we Post

73 ConsiderationsConsiderations Performance can degrade with active databases –Workflow itself can generate unnecessary versions –Delta tables will become large over time –DBMS statistics may need to be refreshed or reviewed by the DB Admin

74 The Cure For many of these ArcGIS-centric performance issues is compressing the database –Moves common rows from delta tables into base tables –Reduces depth of the state tree by removing states no longer needed

75 Compression Example

76 Questions/Discussion?Questions/Discussion?

77 Hands-On Exercise Practice both replication and versioning

78 Your Assignment Complete the exercise handouts –Connecting to and using SDE on DB2 –Practice both replication and versioning Read the PDFs in the SDE exercise folder Visit the URL link for Spatial Data Server and explore this topic

79 Key Concepts SDE is an engine layer residing between a spatially-enabled RDBMS and the GIS desktop. SDE enables Enterprise GIS SDE reduces data management responsibilities. Understand Enterprise workflow

Download ppt "Spatial Database Engine Keith T. Weber, GISP GIS Director Idaho State University."

Similar presentations

Ads by Google