1. Spatial Disaggregation – A Primer
Tom D’Avello – NRCS-NSSC-GRU
contact:
Travis Nauman – NRCS-NSSC-GRU, WVU
contact:

2. Overview Define ‘Disaggregation’ Approaches and Tools Summary
West Virginia
Illinois
Arizona
Summary
Literature list for your reference

3. What is spatial disaggregation?
The next opportunity for the NCSS
Add value to SSURGO
“The process of separating an entity into component parts based on implicit spatial relationships or patterns” – (Moore, 2008)
Getting more detail
Spatially refining maps to reflect the level of detail for current needs
Corresponding increased resolution of attributes
Trying to meet new types of demands

4. What is spatial disaggregation?
Mapping of components within map units
Usually complexes or associations for Order 2 & 3 soil surveys (SSURGO)
STATSGO2 effort
Alaska (Moore, 2008)
New needs served
modeling community
maintenance and improvement of the product is a primary charge of NCSS
The description of what is mapped and where it occurs has been done well. This is the process of showing where different soils occur. A better defined spatial representation of soil patterns leads to questions and research towards why the pattern occurs as mapped, and how they behave as a system.
Modeler's - carbon, water table, hydric soils, tile drainage, drainage management systems, available water etc

5. What is spatial disaggregation?
Ultimately, it is a refined segmentation of the landscape
Along with the spatial, the attributes are equally important
Map units have multiple parts with attributes
Example: Ponded parts of a larger map unit
Related to SDJR
Scope driven!
Area of Interest
Can be relevant to one, some or all map units.
The landscape is used comprehensively here, i.e., the soilscape – all soil forming factors in natural, meaningful classes

6. Purpose of the demonstration
Demonstrate case studies across varying physiographic regions
Get feedback from soil scientists on their assessment of current soil maps
Investigate different digital techniques
Evaluate results
Develop materials and guidelines for application by soil scientists

7. West Virginia early efforts
Gilpin-Laidig
Pineville-Gilpin-Guyandotte
Other
SSURGO Map Units
Gilpin
Pineville
Laidig
Guyandotte
Dekalb
Component Soils
Craigsville
Meckesville
Cateache
Shouns

8. General disaggregation workflow
Goals
Scope
What data is accessible to help
Choose method
Implement
Validate Quality
(evaluate and iterate earlier steps as needed)
Assuming 1 -3 are constant for typical project td

9. Current workflow in West Virginia
Goals
Soil series map on field scale grid
Scope
All map units in Pocahontas and Webster Counties, WV
What data is accessible to help
~30-meter DEM (NED), Landsat Geocover (Fed. MDA, 2004), lithology, SSURGO
Choose method
SSURGO-derived expert rule training sets & classification tree ensemble (100 trees run on random subsets)
Implement
Run analysis with Access (SQL), GIS, and Python (or R)
Validate Quality
Independent pedons for ground truth
Assuming 1 -3 are constant for typical project td

10. 1. & 2. Goals and scope Scope is key
define what needs to be disaggregated
Universal vs within map unit(s) (Local)
Local model
(confined to existing map unit)
Keep original lines
Universal model
uses original survey to create but lines not used for final
Local
Local and Universal/Global may work as terms - TD
Universal
Figures courtesy of Dave Hoover, NSSC

11. Components (not explicitly mapped)
3. What data: SSURGO
Components (not explicitly mapped)
Inclusion
Legend
Map Unit
Horizon
Geomorphology
Parent material
Landscape attributes
Horizon attributes
Soil physical properties
Soil chemical properties
Component
SSURGO database diagram in graphic form
Notes: mention that surveys of different vintage has to be migrated and forced into this format despite being mapped with potentially different procedures, talk about inside-mapunit data for use in disaggregation

12. 3. What data: SSURGO Most work done on SSURGO or equivalent scale maps
Raster (grids) used for modeling
to match environmental data
West Virginia data

13. 3. What data: environmental
Raster grids
Sometimes other polygon layers converted (e.g. geology)
Characterize variation within polygons using data that infer soil forming factors
SSURGO lines over landforms
(Schmidt & Hewitt, 2004)
SSURGO lines over Landsat
SSURGO lines over DEM
Examples from West Virginia

14. 4. Method: model techniques
Training Data
Match environmental data to components of interest
Use representative areas or pedon locations
Model Types
Expert landscape rules
Hardened or fuzzy
Statistical models
Area to Point Interpolations (Goovaerts, 2011)
Dekalb series training areas in WV
Example Classification Tree Model

15. 5. & 6. Implement & Validate Create raster disaggregation map
Validate with ground truth data
Different methods available
 validation
Spatial Support
match type
nearest
60-m radius
exact
26%
39%
like soil
45%
66%
any tree
57%
73%
WV example: universal model for Webster and Pocahontas Counties

16. Historical survey of Webster County, WV
These folks were pretty good
Milton Whitney
Curtis Marbut
Hugh Bennett
Nice map, too!
Soil survey efforts have a long history. The disaggregation process is a continuation of improving the product if possible to satisfy a need for more detail.

17. Peoria, Illinois investigation
Goals
Components or phases within Sable and Ipava units
Scope
All Sable and Ipava map units within Peoria County
What data is accessible to help
3-meter DEM (NED), SSURGO
Choose method
Expert rule training sets & classification trees
Implement
Run analysis with R, ArcGIS and ArcSIE
Validate Quality
Local soil scientist review.

18. Peoria, Illinois investigation 1. Goals
Identification of Non-ponded and ponded phases in Sable units
Identification of poorly drained components in Ipava units
Ipava and Sable soils are extensive in occurrence

19. Peoria, Illinois investigation 2. Scope - study site
The project area is within MLRAs 95B, 108A,
108B, 108C and 115C
~900,000 acres of Sable
~1,186,000 acres of Ipava
Why here?
Availability of high resolution DEMs
Representative setting for Sable and Ipava
Good test for developing procedures to complete for entire extent of units when LiDAR coverage is complete
Ipava and Sable soils are extensive in occurrence

20. General setting 2. Scope - study site
Typical cross-section and qualitative description of Sable and Ipava soils
soil
slope
profile
tangential
wetness
position
sinks
Ipava
Low
Plane
High
Broad summit/Talf
Some
Sable
Lowest
Concave-plane
Highest
Dip on talf
Yes
The setting is flat – talf – dip -rise

21. Variables developed 3. Data - all derived from 3m DEM with ArcGIS/ArcSIE/SAGA GIS
Altitude above channel network
Curvature at numerous neighborhoods
Horizontal distance to flow channel
Maximum curvature –numerous neighborhoods
Minimum curvature –numerous neighborhoods
Multi-resolution ridge top flatness index
Multi-resolution valley bottom flatness index
Profile curvature –numerous neighborhoods
Relative position-numerous neighborhoods
Sink depth and Depression cost surface
Slope
Tangential curvature –numerous neighborhoods
Topographic position index
Vertical distance to flow channel
Wetness index
These are commonly derived by SS and GIS staff with CCE software

22. Exploratory Data Analysis 4. Method
An extensive sample with soil series as a response was developed
Classification Tree in R to determine explanatory variables

23. Purpose of evaluation 4. Method
Spatial data needs to be the driver for modeling effort
Efficient determination of explanatory variables
Efficient determination of thresholds for variables
Practical tools are needed to assist soil scientists in this effort
Do a cursory assessment of all of the major soils to determine how strong the soil-landform relationships are for the given set of potential variables (“predictors”). Provides an indication of similarity among soils in terms of data parameters. E.g. two soils that share geographic settings will not be separable using DSM techniques. However, is “hole mapping” more effective in these cases?

24. Results from classification tree 5. Implement
Input variables
Important variables
Altitude above channel network
Horizontal distance to channel
Minimum curvature 120m neighborhood
Multi-resolution ridge top flatness index
Profile curvature 150m neighborhood
Relative position 90m neighborhood
Relative position 60m neighborhood
Relative position 30m neighborhood
Sink Depth
Slope 30m neighborhood
Topographic position index
Wetness index
Altitude above channel network
Relative Elevation (aka Relative position)
Sink Depth
Developed 20+ datasets – 12 showed promise from qualitative review – 3 were
identified through classification tree as explanatory variables in this example

25. Results from classification tree
5. Implement -Ipava and Sable independently
Classification trees have the advantage of being a “White Box” technique – everything is displayed and can be interpreted readily as opposed to other “Black Box techniques like AI
The “first” node (lower left), indicates 87% classified as Ipava, 13% misclassified
The “second” ( next one to the right) indicates 69% classified as Ipava and 31% misclassified
The “third” ( next one to the right) indicates 64% classified as Sable and 36% misclassified
The “fourth” ( lower right) indicates 76% classified as Sable and 24% misclassified

26. Results from classification tree
5. Implement – walk through the splits
Altitude above channel network
>= 0.25
< 0.25

27. Results from classification tree
5. Implement – walk through the splits
Relative position
>= 0.595
< 0.595

28. Results from classification tree
5. Implement – walk through the splits
Sink depth
>= 1.472
< 1.472

29. Results from classification tree
5. Implement - Results of rules applied for Sable and Ipava
Do the rules determined from the classification tree match SSURGO? Another FYI, and to identify potential outliers and improve concepts for the geographic setting

30. Results from classification tree
5. Implement Rule base compared with SSURGO for Sable
Do the rules determined from the classification tree match SSURGO? Another broad assessment, and to identify potential outliers and improve concepts for the geographic setting. This looks good for Sable, especially given the low relief and limited tools available to soil scientists at the time it was mapped.

31. Ponded vs. non-ponded Sable 6. Validate Local - using depression depth
Blue – likely depression/ponded
Red -Yellow – no depression
Depression depth is one potential derivative that indicates ponding and possibly ponding duration

32. Ponded vs. non-ponded Sable 6
Ponded vs. non-ponded Sable 6. Validate Local - using depression cost surface
Blue – likely depression/ponded
Red -Yellow – no depression
Depression cost surface is another potential derivative that indicates ponding and possibly ponding duration

33. Ponded vs. non-ponded Sable 6. Validate Local - using 3m USGS NED
Zonal statistics indicate 41% of the area mapped as Sable is ponded
Based on selected thresholds
Verification and tuning of threshold values is ongoing
High resolution DEMs have artifacts and some areas identified as depressions may be due to noisy data. Changing the threshold effects estimate, which must be verified with ground truth. Spatial representation provides a perfect medium to supply to the greater field staff for verification across the extent of mapping.

34. Ponded vs. non-ponded Sable
6. Validation/Data Local - using 10m USGS NED
Zonal statistics indicate 17% of the area is ponded
Bigger legend
Area “missed” with
coarser 10m DEM
Test with 10m just to answer inevitable question, “how does a 10m DEM perform?” Horizontal resolution is too large to match small changes in relief in gentle terrain

35. Ponded vs. non-ponded Ipava 6. Validate Local - using 3m USGS NED
Zonal statistics indicate 9% of the area is ponded

36. Future effort for Peoria County
Populate component table
- based on verified and validated thresholds
Rename map unit phases if needed
What is reasonable to improve product?
Accept line work and split components within existing map units?
- A working copy in preparation for phase II of data recorrelation
makes this feasible

37. Arizona – arid example Goal Scope Data Method Implement
match environmental classification of soil forming factor raster layers to soil types.
Scope
Entire soil survey: Organ Pipe Cactus National Monument (ORPI)
Data
Used DEM and ASTER imagery to represent topography, vegetation, and geology
Method
Unsupervised classification (clustering)
Implement
Erdas Imagine and ArcGIS
Validate (evaluation)
Contingency tables (Chi2 Cramer’s V) to MUs; found separation of components in most complexes in field recon. (Nauman, 2009)

38. Arizona – arid example

39. Arizona More methods trials are planned for northeast AZ
Initial spatial data is being compiled
Model runs by late 2013

40. Summary Webster and Pocahontas, West Virginia Peoria, Illinois
ORPI, Arizona
Goal
Soil series map of entire area
Components/phases within Sable and Ipava units
Match environmental raster patterns to MUs
Scope
Full extent of both surveys (Webster and Pocahontas)
Sable and Ipava map units within Peoria Co.
Entire ORPI survey area
Data
DEM (NED 30m), Landsat, geology
DEM (NED 3m)
DEM (NED 30m), ASTER
Method
SSURGO component rules and classification trees
Expert rules and classification trees
Clustering (ISODATA)
Implement
Access (or SQL), ArcGIS, and Python (or R)
ArcGIS, ArcSIE, R
ArcGIS, Erdas Imagine
Validate
Independent set of pedons
Expert review
Compare w/ SSURGO, expert review
Highlights
Series map, harmonized surveys, maintained accuracy
Picked out fine scale depressions
Detected components in complex MUs

41. Summary
Disaggregation is a process that is defined by a need for more detail
Needs a directed scope
Tremendous amount of new data and computing abilities to incorporate
Disaggregating classic soil surveys
improves the detail of final maps without loss of accuracy and with no new data
more realistic representation of soil distribution (continuous – background probabilities)
Can use new field data in future to re-model for easy update (doing this in WV)

42. Next Steps Match disaggregated data to ESDs
Further disaggregate to ESD state and transition models
Would better match imagery because management (e.g. pasture vs forest) is more easily detected with remote sensing.
Could map at state and/or community level for direct use in conservation planning
National Range and Pasture Handbook, 2003
Currently submitting article for peer review documenting WV case study
Nauman, T., J.A. Thompson. (In prep). Semi-Automated Disaggregation of Conventional Soil Maps using Knowledge Driven Data Mining and Classification Trees

43. Resources – Available Training NRCS offers the following courses which provide an introduction to some of these techniques – check AgLearn
Spatial Analysis workshop (distance learning)
Introduction to Digital Soil Mapping (distance learning)
Digital Soil Mapping with ArcSIE (conventional class)
Remote Sensing for Soil Survey Applications (conventional class)

44. Literature
Bui, E., B. Henderson, and K. Viergever Using knowledge discovery with data mining from the Australian Soil Resource Information System database to inform soil carbon mapping in Australia. Global Biogeochemical Cycles 23. Bui, E.N. and Moran, C.J., Disaggregation of polygons of surficial geology and soil maps using spatial modelling and legacy data. Geoderma, 103(1-2): Bui, E.N., A. Loughhead, and R. Corner Extracting soil-landscape rules from previous soil surveys. Australian Journal of Soil Research 37: de Bruin, S., Wielemaker, W.G. and Molenaar, M., Formalisation of soil-landscape knowledge through interactive hierarchical disaggregation. Geoderma, 91(1–2): Goovaerts, P., A coherent geostatistical approach for combining choropleth map and field data in the spatial interpolation of soil properties. European Journal of Soil Science, 62(3): Häring, T., Dietz, E., Osenstetter, S., Koschitzki, T. and Schröder, B., Spatial disaggregation of complex soil map units: A decision-tree based approach in Bavarian forest soils. Geoderma, 185–186(0): Kerry, R., Goovaerts, P., Rawlins, B.G. and Marchant, B.P., Disaggregation of legacy soil data using area to point kriging for mapping soil organic carbon at the regional scale. Geoderma, 170: Li, S., MacMillan, R. A., Lobb, D. A., McConkey, B. G., Moulin, A., & Fraser, W. R Lidar DEM error analyses and topographic depression identification in a hummocky landscape in the prairie region of Canada. Geomorphology, 129(3), McBratney, A.B., Some considerations on methods for spatially aggregating and disaggregating soil information. Nutrient Cycling in Agroecosystems, 50(1-3): MDA, Federal Landsat Geocover TM 1990 & ETM Edition Mosaics Tile N TM-EarthSat-MrSID. USGS, Sioux Falls, South Dakota.

45. Literature
Moore, A Spatial Disaggregation Techniques for Visualizing and Evaluating Map Unit Composition. NRCS 2008 National State Soil Scientist’s Workshop. Florence, Kentucky. ftp://ftp-fc.sc.egov.usda.gov/NSSC/NCSS/Conferences/state/2008/moore.pdf Nauman, T.W., Digital Soil-Landscape Classification for Soil Survey using ASTER Satellite and Digital Elevation Data in Organ Pipe Cactus National Monument, Arizona. MS Thesis. The University of Arizona. Nauman, T., J.A. Thompson, N. Odgers, and Z. Libohova Fuzzy Disaggregation of Conventional Soil Maps using Database Knowledge Extraction to Produce Soil Property Maps, In B. Minasny, et al., (eds.) Digital Soil Assessments and Beyond: 5th Global Workshop on Digital Soil Mapping, Sydney, Australia. Schmidt, J. and Hewitt, A., Fuzzy land element classification from DTMs based on geometry and terrain position. Geoderma, 121(3-4): Thompson, J.A. et al., Regional Approach to Soil Property Mapping using Legacy Data and Spatial Disaggregation Techniques, 19th World Congress of Soil Science, Soil Solutions for a Changing World, Brisbane, Australia. Wei, S. et al., Digital Harmonisation of Adjacent Soil Survey areas - 4 Iowa Counties, 19th World Congress of Soil Science, Soils Solutions for a Changing World, Brisbane, Australia. Wielemaker, W.G., de Bruin, S., Epema, G.F. and Veldkamp, A., Significance and application of the multi-hierarchical landsystem in soil mapping. Catena, 43(1): Yang, L. et al., Updating Conventional Soil Maps through Digital Soil Mapping. Soil Science Society of America Journal, 75(3): Zhu, A.X., A similarity model for representing soil spatial information. Geoderma, 77(2-4): Zhu, A.X., Band, L., Vertessy, R. and Dutton, B., Derivation of soil properties using a soil land inference model (SoLIM). Soil Science Society of America Journal, 61(2): Zhu, A.X., Band, L.E., Dutton, B. and Nimlos, T.J., Automated soil inference under fuzzy logic. Ecological Modelling, 90(2):

46. Questions