1. The e-sensing architecture for big Earth observation data analytics
Lubia Vinhas and Gilberto Câmara
National Institute for Space Research, INPE, Brazil

2. With many thanks to the esensing team…
Adeline
Lúbia
Ieda
Isabel
Karine
Gilberto
Luiz
Alber
Cartaxo
Eduardo
Pedro
Victor
Rodrigo
Michelle
Rolf
With many thanks to the esensing team…

3. INPE’s Data Cube Science advances by Free imagery cooperation
R for Data Science
SciDB for big scientific data
SITS: satellite image time series in R

4. From User Requirements to Data Architectures
Needs of research community
System architecture choices
State-of-the-art statistical tools for Data Science
Data management for scientific data
Analytical scaling: algorithms developed at the desktop run on big databases
Collaborative work: share results with the scientific community.
Replication: research teams can build their own infrastructure.

5. Datacubes: two strategy
Space-first: classify
images separately.
Compare results in time
Time-first: classify
time series separately.
Join results to get maps

6. Time first, space later Forest Pasture Agric Forest Forest Agriculture
Remote sensing time series with good resolution provide data for describing landscape dynamics

7. Machine learning with big data
Combine SVM, Random forest, Deep learning, Neural nets with satellite image time series

8. Matching land use patterns in time series
Finding subsequences in a time series
High computational complexity
Patterns are idealized, data is noisy

9. Machine learning for dense time series
Use all the data available in the time series samples!!
Maximize dimension spaces and sample sizes

10. Temporal patterns of different classes

11. Mato Grosso – Brazil’s agricultural frontier
Land change dynamics ( )

12. High accuracy for mapping land use change
Finding subsequences in a time series
Mato Grosso – Brazil’s agricultural frontier
Overall accuracy of 94%

13. What do these data have in common?

14. result = analysis_function (points in space-time )
Array databases: all data in a single array t y y X
result = analysis_function (points in space-time )

15. SciDB Architecture: “shared nothing”
Large data is broken into chunks
Distributed server process data in paralel

16. SciDB: efficient processing of R scripts

17. WTSS: web time series service
TerraAmazon
TWDTW via Web
Validation tool
via web
Client
APIs
wtss.cxx
wtss.R
wtss.py
wtss.js
HTTP/GET
HTTP/JSON
WTSS: web time series service
WTSS - Server
SciDB

18. Flexibility with performance

19. Global Land Observatory
Methods for land change for forestry and agriculture uses
40 years of LANDSAT years of MODIS + SENTINELs + CBERS
Building knowledge and data about global land change