1. GeoMesa, GeoBench & SFCurve: Measuring & Improving BigGeo performance
Jim Hughes

2. Analyzing & Visualizing Big Spatio-temporal data: GeoMesa
Integrates with MapReduce & Spark
Has Server-side processing for
Heatmaps / densities
Histograms
Can visualize live data
There’s also support for Amazon’s DynamoDB and Google Cloud BigTable

3. Storage: How does GeoMesa store geo-time data?
Accumulo is a key-value store and provides a linear index on the keys.
The data GeoMesa stores is at least 2 dimensional.
>> Question:
How do we go from 2, 3 (or more) dimensions to one
dimensional in a sane way?
>> Answer:
Space filling curves.

4. Space Filling Curves: The Math
Around 100 years ago, mathematicians asked the question,
“Is there a continuous function from the unit interval to the unit square
which covers it?” ...Several families of functions are created:
Row-Major
Z-Order
Hilbert

5. 2D Curves applied to Geo-coordinates

6. Query planning
To query for points in the grey rectangle, the query planner enumerates a collection of index ranges which cover the area.
Note: Most queries won’t line up perfectly with the gridding strategy.
Further filtering can be run on the Accumulo tablet servers.

7. 3D Curves

8. 3D Curves

9. GeoMesa Spatio-Temporal Index
KEY
VALUE
ROW
COLUMN
Simple
Feature
FAMILY
QUALIFIER
Shard
1-Byte
Epoch Week
2-Bytes
Z3(x,y,t)
8-Bytes
‘F’

10. GeoMesa Spatio-Temporal Index
KEY
VALUE
ROW
COLUMN
Simple
Feature
FAMILY
QUALIFIER
Shard
1-Byte
Epoch Week
2-Bytes
Z3(x,y,t)
8-Bytes
‘F’
Example: Here and now: 5:00pm, Wednesday, May 4th, 2016
Epoch Week: 2417
X value:
Y value:
T value:
Z3 (as a long):

11. SFCurve: Composable curves
Space filling curves can be composed.
GeoMesa’s index can be thought of as a row major composition of time (epoch week) with a Z3 of space and time (within the week).

12. GeoBench
GeoBench is project which aims to benchmark geospatial databases
and analytic platforms. Integrated with the:
GeoTools DataStore API
OGC WMS requests
Sample Use cases:
Comparing products with your data and your queries
Running tests in a CI environment
Developing new capabilities in an existing database
Understanding database features and indexing choices

13. GeoBench
GeoBench is project which aims to benchmark geospatial databases
and analytic platforms. Integrated with the:
GeoTools DataStore API
OGC WMS requests
Sample Use cases:
Comparing products with your data and your queries
Running tests in a CI environment
Developing new capabilities in an existing database
Understanding database features and indexing choices

14. Comparing loose vs precise bounding boxes
PostGIS and other databases offer ‘loose bounding box’ handling. For some cases, such as WMS, ‘loose’ handling is better.
Using GeoBench, we can measure the difference in loose vs precise timing in GeoMesa.

15. GeoMesa’s - new Spatial only index
The original GeoMesa index focused on geo-time use cases.
Worked ok with ‘spatial-only’ queries and for data without time.
New Z2 index doesn’t incorporate time: GeoBench helps us validate the new index.

16. GeoBench: Understanding Complex query behavior
GeoMesa’s Z3 Index uses a week resolution:
>> Q: How does this affect queries as the time range changes?
>> A: Timing for queries with large result sets depends on the number of records returned. Smaller queries are faster for time periods shorter than a week.

17. Questions? More info? Jim Hughes - james.hughes@ccri.com
GeoMesa:
SFCurve:
GeoBench:
Mailing lists via LocationTech:

18. GeoMesa: Visualizing big Spatio-temporal data
GeoMesa has a brand new TEXT ADVENTURE version:

19. Analyzing & Visualizing Big Spatio-temporal data: GeoMesa
Taxi Data NYC
Animated Points
There’s also support for Amazon’s DynamoDB and Google Cloud BigTable