2009.foss4g.org
Spatial Database Tips & Tricks Paul Ramsey pramsey@opengeo.org
Housekeeping Copy workshop from DVD Download from http://xx.xx.xx.xx/xxx/spdb-003.zip Install or not Ignore me or not Examples also at http://xx.xx.xx.xx:8080/spatialdbtips
Impatient People They try to install without reading instructions When you see an error box during PostGIS install, click “Ignore” Remember to create “medford” (not “postgis”) database during PostGIS install
Impatient People
It’s not dead, it’s just resting…
Motivation Spatial databases are powerful Godlike, really You do not need “GIS software” Your database is “GIS software” You do not need “spatial middleware” See above
Standard Database Has data types varchar integer real date
Spatial Database Has spatial data types point linestring polygon multipoint multilinestring multipolygon
Standard Database Has one-dimensional indexes b-tree hash
Spatial Database Has spatial indexes r-tree quad-tree grid
Find intersecting shapes…
Start with all boxes,
find intersecting boxes,
then find intersecting shapes.
Standard Database Has functions Work against standard types lower() round() substring() trim() dayofweek ()
Spatial Database Has spatial functions Work against spatial types ST_Area(geometry) ST_Distance(geometry,geometry) ST_Intersects(geometry,geometry) ST_DWithin(geometry,geometry,radius) ST_Union(geometry,geometry)
Open Geospatial Consortium (OGC) Simple Features for SQL (SFSQL)
Spatial Locator
No intersection operation No centroid point L O C A T O R No buffer operation No union operation No intersection operation No centroid point No area or length calculation
S P A T I A L Linear referencing system (LRS) support Spatial analysis and mining functions and procedures (SDO_SAM package) Geocoding support (SDO_GCDR package) GeoRaster support Topology data model Network data model
SFSQL compliant New release, not as many features No coordinate reference system transforms Windows only Grid index
SELECT * FROM the_table WHERE ST_Intersects( the_geom, ST_GeomFromText('POINT(0 0)',0) ); SELECT * FROM the_table WHERE the_geom.STIntersects( geometry::STGeomFromText('POINT(0 0)',0) );
PostgreSQL / PostGIS SFSQL compliant Open source (GPL) Proprietary / open source clients “geographic” coordinates require care
ST_Distance(‘POINT(0 0)’,’POINT(1 1)’) What units? ST_Distance_Spheroid() ST_Distance_Sphere() Indexes are not sphere aware Spherical distance functions defined on points only
Installation
Installation
Installation
Installation
Data We are going to be installing data from PostgreSQL backup files, because it is fast and easy to get everything in the right place and indexed. But data you download from government will usually be in some other format.
Data Shape files .shp, .shx, .dbf, .prj shp2pgsql Other? FME ogr2ogr
Tomcat JNDI configured JSLT installed PostgreSQL JDBC Connection to “medford” database JSLT installed <c:> <sql:> GeoServer installed Connections to “medford” tables Styles for “medford” tables
Tomcat
Workshop
#0 - Base Map with WMS
+ =
+ =
http://localhost:8080/geoserver/
<body onload="init()"> <div id="map"></div> </body>
var lon = -122.8450; var lat = 42.3438; var zoom = 18; var map; function initMap() { map = new OpenLayers.Map( 'map' , {controls:[new OpenLayers.Control.MouseDefaults(), new OpenLayers.Control.LayerSwitcher(), new OpenLayers.Control.PanZoomBar()], numZoomLevels:20}); var gmap = new OpenLayers.Layer.Google( "Google Streets" // the default ); var gsat = new OpenLayers.Layer.Google( "Google Satellite", {type: G_SATELLITE_MAP} map.addLayers([gmap, gsat]); map.setCenter(new OpenLayers.LonLat(lon, lat), zoom); }
// Initialize WMS layer from our local // GeoServer var bwms = new OpenLayers.Layer.WMS( "Medford Buildings", "http://localhost:8080/geoserver/wms?", { "transparent":"true", "layers":"medford:buildings", "format":"image/png" }, { "reproject":"true" } ); // Add WMS layer to our map map.addLayer(bwms);
#1 – Click to Query
#1 – Click to Query
// Tie the map click event to our query function map. events // Tie the map click event to our query function map.events.register("click", map, queryDatabase );
function queryDatabase(e) { // Read the map coordinates from the click event var lonlat = map.getLonLatFromViewPortPx(e.xy); // Read the table we are going to query from page var table = document.getElementById("table").value; // Construct the query URL var url = "01-click-query.jsp"; url += "?lon=" + lonlat.lon; url += "&lat=" + lonlat.lat; url += "&table=" + table; // Load the URL into an iframe document.getElementById("query").src = url; }
01-click-query.jsp? lon=-122.8451943397522& lat=42.344141057680226& table=medford.taxlots
01-click-query.jsp <%@ taglib uri="http://java.sun.com/jsp/jstl/sql" prefix="sql" %> <%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %> <%@ page contentType="text/html" %> <sql:query var="rs" dataSource="jdbc/medford"> . . . . . . . </sql:query>
select st_geometrytype(the_geom) as geometrytype, st_area(the_geom) as area, * from ${param.table} where st_contains( the_geom, st_transform( st_setsrid( st_makepoint(${param.lon},${param.lat}), 4326), 2270))
SQL
SRID?!? Location = Coordinate + SRID Here = (-121.92, 37.37) + EPSG:4326 ST_Transform() to change SRID Store data in an efficient SRID Transform inputs to your storage SRID Transform outputs to your display SRID
#2 – Click to Analyze
02-click-analyze.jsp? lon=-122.8451943397522& lat=42.344141057680226& radius=200
select count(*) as "Number of Lots", round(avg(st_area(the_geom))::numeric/43560, 1) || ' acres' as "Average Lot Area", '$' || avg(impvalue)::integer as "Average Improvement Value", '$' || avg(landvalue)::integer as "Average Land Value", '$' || avg(impvalue + landvalue)::integer as "Average Total Value", avg(yearblt)::integer as "Average Year Built" from medford.taxlots where st_dwithin( taxlots.the_geom, st_transform( st_setsrid( st_makepoint(${param.lon},${param.lat}), 4326), 2270), ${param.radius} )
ST_DWithin()?!? Indexed distance query ST_Distance(g1,g2) < r is not indexed ST_DWithin(g1,g2,r) is equivalent to g1 && ST_Expand(g2,r) AND ST_Distance(g1,g2) < r
#3 – Click and Join
zoning taxlots
Everything is related to everything else, but near things are more related than distant things. - Waldo Tobler
Spatial relationships are a universal key for joining otherwise disparate data. zoning table customer table taxlot table census table road table stream table
select count(*) as num_lots, sum(st_area(taxlot.the_geom)) as total_lot_area, zone.zoning as zoning, sum(taxlot.landvalue) as total_land_value, sum(taxlot.landvalue) / as value_per_ft
from medford.taxlots taxlot join medford.zoning zone on ( st_contains( zone.the_geom, st_centroid(taxlot.the_geom) )
where st_dwithin( taxlot.the_geom, st_transform( st_setsrid( st_makepoint( ${param.lon}, ${param.lat}), 4326), 2270), ${param.radius} ) group by zone.zoning order by total_lot_area desc
#4 – Click and Union
ST_AsGeoJSON() select st_asgeojson(the_geom) from medford.streets where st_npoints(the_geom) < 6 limit 1;
GeoJSON Geometry {"type":"MultiLineString", "coordinates":[[ [4289753.869,253537.254], [4290375.489,253518.361] ]]}
ST_As*() Standard PostGIS ST_AsText() – defined by OGC (SFSQL) ST_AsBinary() – defined by OGC (SFSQL) ST_AsGML() – defined by OGC (GML) PostGIS ST_AsKML() – defined by Google (OGC) ST_AsSVG() – defined by W3C ST_AsGeoJSON() – defined by community
FeatureCollection Feature Geometry Properties Feature Geometry
<%@ taglib uri="http://java. sun <%@ taglib uri="http://java.sun.com/jsp/jstl/sql" prefix="sql" %> <%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %> <%@ page contentType="text/x-json" %> <sql:query var="rs" dataSource="jdbc/medford"> ${param.sql} </sql:query> {"type":"FeatureCollection", "features":[ <c:forEach var="row" items="${rs.rows}" varStatus="rowStatus"> {"type":"Feature", "geometry":<c:out value="${row.st_asgeojson}" escapeXml="false" />, "properties":{ <c:forEach var="column" items="${row}" varStatus="columnStatus"> <c:if test="${column.key != 'st_asgeojson'}"> "<c:out value="${column.key}" escapeXml="false" />": "<c:out value="${column.value}" escapeXml="false" />" <c:if test="${! columnStatus.last}">,</c:if> </c:if> </c:forEach> }} <c:if test="${! rowStatus.last}">,</c:if> ]}
select st_asgeojson( st_transform(the_geom,900913) ) from medford.taxlots where st_dwithin( the_geom, st_transform( st_setsrid( st_makepoint(-13676108, 5212594), 900913), 2270), 100
// Make a fresh vector layer, pulling features our URL json_layer = new OpenLayers.Layer.Vector("GeoJSON", { strategies: [new OpenLayers.Strategy.Fixed()], protocol: new OpenLayers.Protocol.HTTP({ url: json_url, format: new OpenLayers.Format.GeoJSON() }) }); // Add our vector layer to the map map.addLayer(json_layer);
#5 – Arbitrary SQL
DEMO
#6 – Walk a Network
medford.storm_drains node_fm node_to
medford.storm_drains id node_fm node_to 91058 D372W25CN0169 91061 D372W25CN0168 D372W25CN0167 id node_fm node_to 91062 D372W25CN0167 D372W25CN0166
DEMO
select d.node_fm, d.node_to, d.pipe_id from medford.storm_drains d, medford.storm_drains e where st_dwithin(d.the_geom, e.the_geom, 5) and e.node_to = 'D371W28CN0134' and e.gid != d.gid and st_distance( st_endpoint(st_geometryn(e.the_geom, 1)), st_startpoint(st_geometryn(d.the_geom, 1)) ) < 5;
e.node_to = 'D371W28CN0134'
ST_DWithin()
ST_DWithin()
e.gid != d.gid
st_distance( st_endpoint(st_geometryn(e st_distance( st_endpoint(st_geometryn(e.the_geom, 1)), st_startpoint(st_geometryn(d.the_geom, 1)) ) < 5;
st_distance( st_endpoint(st_geometryn(e st_distance( st_endpoint(st_geometryn(e.the_geom, 1)), st_startpoint(st_geometryn(d.the_geom, 1)) ) < 5;
ST_GeometryN() Convert To MULTILINESTRING((0 0, 1 1, 2 2))
In conclusion…
Standards are good… SFSQL KML WMS GML WKT GeoJSON WFS WKB
Special middleware is unnecessary…
SQL
2009.foss4g.org