1. 1 JRC – Ispra – 23 July 2004 Luis Rodríguez Lado E-mail : luis.rodriguez-lado@jrc.it Alpine Soil Information System Analysis of the accuracy of ESBD in the Alps region

2. 2 JRC – Ispra – 23 July 2004 Introduction There is a increasing demand of soil maps and of their properties in the frame of the EU. This information is needed to develop policies linked to sustainable land management practices, and to avoid the damage risk to ecosystems. At present, the 1:1M digital soil map and of some of their properties are available at European Scale.

3. 3 JRC – Ispra – 23 July 2004 Objective In this exercise, we evaluate the accuracy of the ESDB maps by comparison with some reference maps derived from detailed survey (ECALP).

4. 4 JRC – Ispra – 23 July 2004 Accurate digital soil maps were computed for 5 pilot areas in the Alps region (ECALP Project). Methodology data from ECALP areas

5. 5 JRC – Ispra – 23 July 2004 Maps in the ECALP areas are available as to raster based soil maps. The pilot areas were divided in 1Km 2 cells. In this analysis, the soil properties used for each cell are those of the main Soil Map Unit in the cell (% area). Methodology data from ECALP areas

6. 6 JRC – Ispra – 23 July 2004 The 1:1M ESDB was rasterized into a 1Km 2 cell raster grid. The soil properties for each grid cell were also those of the main Soil Map Unit in the cell (% area). We compare the results of both maps. Methodology data from ESDB areas

7. 7 JRC – Ispra – 23 July 2004 Texture. Depth of presence of an obstacle to roots. Depth of presence of an impermeable layer. Methodology properties analyzed

8. 8 JRC – Ispra – 23 July 2004 Methodology The accuracy of the 1:1 M map is expressed by naïve measures of accuracy using confusion matrices.

9. 9 JRC – Ispra – 23 July 2004 Objective PRUEBAECALP Cat1Cat2Cat3Cat4Cat5Cat6Cat7Cat8Cat9Cat10 Total pi+User accuracyProducer accuracyKappa UserKappa Producer ESD B Cat1 440137029413118 48630,90500,80080,86930,7372 0,90500,00760,00000,06050,00270,0243 Cat2 345412311263510 21200,25520,40550,20230,3355 0,01600,25520,10900,59580,02410,0000 Cat3 02847005421430 16690,41940,36670,35850,3094 0,00000,17020,41940,32470,08570,0000 Cat4 1054454182440913015 62440,70610,59380,53410,4109 0,16880,07270,02910,70610,02080,0024 Cat5 718782851626309 25930,24140,62980,20200,5750 0,00270,00690,30160,32820,24140,1192 Cat6 001466312507 26180,95760,85010,95030,8277 0,0000 0,00530,02520,01180,9576 Cat7 0 Cat8 0 Cat9 0 Cat1 0 0 Total54961334190974259942949 20107 p+j Overall Accuracy =0,6557Global Kappa =0,557939 S.D. =0,0034

10. 10 JRC – Ispra – 23 July 2004 Methodology The Users accuracy expresses the probability that one class (in ESDB) is well mapped in relation to the reference dataset (ECALP). The Producers accuracy indicates the proportion of cells that were correctly classified.

11. 11 JRC – Ispra – 23 July 2004 Objective PRUEBAECALP Cat1Cat2Cat3Cat4Cat5Cat6Cat7Cat8Cat9Cat10 Total pi+User accuracyProducer accuracyKappa UserKappa Producer ESD B Cat1 440137029413118 48630,90500,80080,86930,7372 0,90500,00760,00000,06050,00270,0243 Cat2 345412311263510 21200,25520,40550,20230,3355 0,01600,25520,10900,59580,02410,0000 Cat3 02847005421430 16690,41940,36670,35850,3094 0,00000,17020,41940,32470,08570,0000 Cat4 1054454182440913015 62440,70610,59380,53410,4109 0,16880,07270,02910,70610,02080,0024 Cat5 718782851626309 25930,24140,62980,20200,5750 0,00270,00690,30160,32820,24140,1192 Cat6 001466312507 26180,95760,85010,95030,8277 0,0000 0,00530,02520,01180,9576 Cat7 0 Cat8 0 Cat9 0 Cat1 0 0 Total54961334190974259942949 20107 p+j Overall Accuracy =0,6557Global Kappa =0,557939 S.D. =0,0034

12. 12 JRC – Ispra – 23 July 2004 Methodology The Overall accuracy is the sum of the correctly classified cells (diagonal values) divided by the total number of cells analyzed. It indicates the proportion in which those maps agree. The KAPPA coefficient of agreement is a measure of the chance in the agreement. It indicates whether the agreements found in the overall accuracy are due to the map accuracy of due to chance.

13. 13 JRC – Ispra – 23 July 2004 Objective PRUEBAECALP Cat1Cat2Cat3Cat4Cat5Cat6Cat7Cat8Cat9Cat10 Total pi+User accuracyProducer accuracyKappa UserKappa Producer ESD B Cat1 440137029413118 48630,90500,80080,86930,7372 0,90500,00760,00000,06050,00270,0243 Cat2 345412311263510 21200,25520,40550,20230,3355 0,01600,25520,10900,59580,02410,0000 Cat3 02847005421430 16690,41940,36670,35850,3094 0,00000,17020,41940,32470,08570,0000 Cat4 1054454182440913015 62440,70610,59380,53410,4109 0,16880,07270,02910,70610,02080,0024 Cat5 718782851626309 25930,24140,62980,20200,5750 0,00270,00690,30160,32820,24140,1192 Cat6 001466312507 26180,95760,85010,95030,8277 0,0000 0,00530,02520,01180,9576 Cat7 0 Cat8 0 Cat9 0 Cat1 0 0 Total54961334190974259942949 20107 p+j Overall Accuracy =0,6557Global Kappa =0,557939 S.D. =0,0034

14. 14 JRC – Ispra – 23 July 2004 Methodology For example: An Overall Accuracy of 0.655 indicate that both maps agree in 65% of the cases. A Kappa statistic of 0,557 indicates that 55,7% of this agreement is due to the mapper competency, and 9,3% of the agreements were due to chance.

15. 15 JRC – Ispra – 23 July 2004 Methodology Low values of Kappa indicate : a) Bad Map. Errors due the mapper or to the mapping technique. We can do another map with the same accuracy simply by random assignation using the same classes. b) An highly homogeneous area (1 class in whole area). For these areas, high values of agreement can be achieved also randomly.

16. 16 JRC – Ispra – 23 July 2004 Results

17. 17 JRC – Ispra – 23 July 2004 Texture Results frequency distribution (n = 1818 cells)

18. 18 JRC – Ispra – 23 July 2004 Texture confussion and probabilities matrices; Accuracy index

19. 19 JRC – Ispra – 23 July 2004 Texture class Lombardia-Switzerland (ECALP) (ESDB)

20. 20 JRC – Ispra – 23 July 2004 Texture Lombardia confussion and probabilities matrices; Accuracy index

21. 21 JRC – Ispra – 23 July 2004 Texture class Friuli-Slovenia (ECALP) (ESDB)

22. 22 JRC – Ispra – 23 July 2004 Texture Friuli confussion and probabilities matrices; Accuracy index

23. 23 JRC – Ispra – 23 July 2004 Conclusions Texture

24. 24 JRC – Ispra – 23 July 2004 Conclusions Texture

25. 25 JRC – Ispra – 23 July 2004 Conclusions Texture

26. 26 JRC – Ispra – 23 July 2004 Depth of an obstacle for roots Results frequency distribution (n = 1818 cells)

27. 27 JRC – Ispra – 23 July 2004 Depth of an obstacle for roots confussion and probabilities matrices; Accuracy indexes

28. 28 JRC – Ispra – 23 July 2004 Depth to obstacle to roots Lombardia-Switzerland (ECALP) (ESDB)

29. 29 JRC – Ispra – 23 July 2004 Conclusions Obstacle to roots

30. 30 JRC – Ispra – 23 July 2004 Conclusions Obstacles to roots

31. 31 JRC – Ispra – 23 July 2004 Conclusions Obstacles to roots

32. 32 JRC – Ispra – 23 July 2004 Depth of an impermeable layer Results frequency distribution (n = 1818 cells)

33. 33 JRC – Ispra – 23 July 2004 Depth of an impermeable layer Results confussion and probabilities matrices; Accuracy index (n = 1818 cells)

34. 34 JRC – Ispra – 23 July 2004 Conclusions Depth of an impermeable layer

35. 35 JRC – Ispra – 23 July 2004 Conclusions We found that the present 1:1M ESDB maps means a great generalization of soils and their properties, being inappropriate to derive effective policies in the EU at medium and large scales due to the uncertainty of its information. The overall accuracy of these maps is generally lower than 50%. It varies between 0,33 (obstacle to roots) to 0,8 (depth of impermeable layer) but low values of Kappa were found, indicating high influence of chance in the success of classification. This low values of Kappa are greatly due to the low discrimination in classes in ESDB (general map).

36. 36 JRC – Ispra – 23 July 2004 Conclusions Friuli-Slovenia was the region that showed a better agreement with the ECALP database, particularly for the depth of an obstacle to roots, where it also exhibits a high value of Kappa.

37. 37 JRC – Ispra – 23 July 2004 Conclusions 1.Need of more accurate soil maps than ESDB 2.Provide soil sample description as metadata 3.Consensus in the description of properties 4.Implementation of accuracy tests for maps