1. Customizing ArcGIS Online Data Basin Project Overview
Web Mapping for Conservation Organizations Workshop
Redlands, California
March 8th – 9th, 2010

2. Agenda Project Goals Project Overview Technical Overview Challenges
Outcome
Q&A
Web Mapping for Conservation Workshop

3. Data Basin Project Goals for ESRI
Support the needs of the conservation community by providing infrastructure to store and disseminate spatial data and maps
Gather requirements and provide feedback to ESRI product development while implementing and testing ArcGIS 10 and ArcGIS Online 2.0.
Support successful implementation of ArcGIS in a cloud computing environment
Conservation needs – in addition, tailoring a system that is geared toward both GIS and non-GIS experts.
Web Mapping for Conservation Workshop

4. ESRI Product Development Teams ESRI Professional Services Team
Team Collaboration
ESRI Product Development Teams
ESRI Professional Services Team
CBI Development Team
Communication is key. Teams need to have confidence in each other’s abilities and be able to share mutual trust.
ESRI Prof services = liaison between CBI development and ESRI product development teams.
CBI requirements and feedback relayed by ESRI Prof Services to the ESRI product development teams
Product updates from ESRI software development teams relayed by ESRI professional services to CBI
Web Mapping for Conservation Workshop

5. Data Basin Project Lifecycle
Planning & Strategy
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Typical project life cycle used in Prof. Services development projects. Details on each stage and unique characteristics of each for this particular project.
Web Mapping for Conservation Workshop

6. Planning & Strategy Core ESRI product considerations
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Core ESRI product considerations
Alignment with ESRI product development release schedule
Alignment with CBI Data Basin release schedule
Web Mapping for Conservation Workshop

7. Requirements Analysis
Planning & Strategy
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Joint review of pre- defined Data Basin requirements
Communicating Data Basin requirements to ESRI product teams
Gap analysis
Web Mapping for Conservation Workshop

8. Design
Planning & Strategy
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Collaboratively designed specifications for both Data Basin website and GIS Web services
UI design independently conducted by CBI
Web Mapping for Conservation Workshop

9. Development
Planning & Strategy
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Amazon Cloud development environment (same as ArcGIS Online)
ESRI – GIS Web services
CBI – development of front end application and non-GIS components & services
Web Mapping for Conservation Workshop

10. Deployment Deployment requirements review
Planning & Strategy
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Deployment requirements review
Performance & scalability testing
Amazon Cloud environment managed by ESRI Managed Services
Deployment – performance & scalability results will assist ESRI with understanding how well ArcGIS performs on the cloud
Web Mapping for Conservation Workshop

11. Operations & Maintenance
Planning & Strategy
Requirements Analysis
Design
Development
Deployment
Operations & Maintenance
Technology Transfer Workshop
Public beta launch - May 1, 2010
Full Data Basin release target – July 2010
Data Basin data and application hosting provided by ESRI Managed Services
Tech transfer workshop – review ESRI-developed GIS services and perform knowledge transfer to CBI development team so they can maintain the services moving forward.
Operation & Maintenance – tech transfer allows for CBI to take over management of all GIS functionality (with ESRI’s support as needed)
Web Mapping for Conservation Workshop

12. Project Schedule
ESRI level of effort for overall project = ~1200 hours
Development timeline reflects both CBI and ESRI development activities
A little over 1.5 years to fully release Data Basin
Web Mapping for Conservation Workshop

13. Overview of Business Requirements
Easily find and access high-quality conservation-related data (including essential metadata)
Create and share compelling online maps to support conservation work
Contribute and publish data and share it publicly or with specified groups of users
Allow users to contribute information about themselves and the work they do to foster connections between people
Support development and promotion of conservation-related applications
Ensure secure access to restricted information
Provide full control over branding and design to CBI
Web Mapping for Conservation Workshop

14. Overview of Functional Requirements
User and group management
Workspace management
Searching and browsing
Web mapping including base maps
Limited GIS analysis
Web site content management
Web Mapping for Conservation Workshop

15. Overview of Operational Requirements
Build with scalability in mind – start small and evolve over time
Initial release with approximately 800 – 1000 datasets
Ability to grow number of datasets over time
Handle simultaneous access by large number of users
Provide reasonable level of availability
Limit operational cost and hosting cost
Web Mapping for Conservation Workshop

16. Design Considerations and Options
ArcGIS Online Data Sharing fulfilled 80% of requirements
User and group management
Workspace management
Searching, browsing, and Web mapping capabilities
Gaps:
Exposing Web mapping capabilities for user contributed content requires ArcGIS Server infrastructure
Inability to customize the look and feel
Lack of management tools for organizations
Other missing features (sorting of search results, comments and ratings, spatial search, ability to feature specific content)
Web Mapping for Conservation Workshop

17. ArcGIS Geowarehouse REST API
RESTful Web services enabling the sharing features of ArcGIS Online
Simple, easy to use model
Users
Groups
Content items
Accessible to any registered ArcGIS Online user
REST Services Directory and API Documentation are disabled
Web Mapping for Conservation Workshop

18. Implementation Approach
Leverage ArcGIS Geowarehouse REST API using a custom developed Web application
Provide control over branding and design
Identify common requirements and use cases and include them in the ArcGIS Online product development cycle
Utilize and extend the capabilities of the ArcGIS software platform to enable Data Basin specific features and to support an accelerated implementation to meet project schedule
Employ Amazon Elastic Compute Cloud (Amazon EC2)
Web Mapping for Conservation Workshop

19. Walkthrough: Automatically Publishing Datasets as Map Services
Problem statement: In order to use the Web mapping feature of ArcGIS Online users need to publish ArcGIS Server map services
This requirement is impractical considering the target audience of Data Basin
Solution: Automatically prepare datasets for publishing as map services and publish them on demand
Requires extending the capabilities of ArcGIS Server
Web Mapping for Conservation Workshop

20. Walkthrough: Approaches to Extending ArcGIS Server
Create Web services that leverage GIS business logic using ArcObjects
Easy to develop and deploy
Asynchronous and synchronous request execution
Can negatively impact performance due to high network overhead
Create custom server object extensions
More complex to develop and deploy
9.3.x: Manual XML processing for SOAP; REST requires custom Web service
10: Direct REST and SOAP support in SDK
Synchronous request execution only
Good for performance sensitive applications
Create custom geoprocessing functions
Easier to develop and deploy
SOAP and REST support out of the box by way of geoprocessing service
Both synchronous and asynchronous execution
Best for long running processes (>5 seconds)
Approach chosen for Data Basin
Web Mapping for Conservation Workshop

21. Walkthrough: Import Workflow1 2 3
User uploads dataset
System creates temp. map service
User enters metadata
Abstract and summary
Field descriptions
Custom styling
Shapefile Archive (.zip)
Layer Package (.lpk)
ESRI GRID
Create map document
Add dataset with default styling
Publish map service 5 4
Restore map service
Archive map service
Delete map service
Archive map document and store archive on Amazon S3
Create item in ArcGIS Online
Download and extract archive from S3
Publish map service
Map service ready for use
Map service does not consume resource when it is not being used
Web Mapping for Conservation Workshop

22. Demonstration
Web Mapping for Conservation Workshop

23. Solution Architecture
Design Considerations
Coarse-grained interfaces
Asynchronous processing
Stateless components
Partitioning of data and functionality
N-tier architecture
The processing requirements of the layers in the application differ. Processing in one layer could absorb sufficient resources to slow the processing in other layers.
Layered architectural style
Application is complex, and the high-level design demands separation so that teams can focus on different areas of functionality
Support multiple clients

24. Third-party Technology
Technology Stack
ESRI Technology
Third-party Technology
ArcGIS 10
ArcGIS Desktop
ArcGIS Server for the Java Platform
ArcGIS API for JavaScript 1.6
ArcGIS Geowarehouse REST API 1.3
ArcGIS Online Services
Java SE Development Kit (JDK) 6
Python 2.6
Dojo 1.4.1
Sun GlassFish Enterprise Server 2.1
MySQL Community Server
Fedora Linux release 8
Windows Server 2008
Web Mapping for Conservation Workshop

25. Project Challenges Developing with ArcGIS 10 beta versions
Management of joint development activities
ArcGIS Online integration and alignment with ESRI core development objectives
Amazon Cloud deployment of ArcGIS
Integration with ESRI global account system
Developing strategies for scaling ArcGIS Server
Web Mapping for Conservation Workshop

26. Project Outcomes Requirements sharing with ArcGIS Online
Comments and ratings
Sorting search results (by comments, ratings, etc.)
Increase user & community focus
Testing support for ArcGIS 10
Holistic testing of ArcGIS in an end to end scenario
Defect reporting and suggestions for workarounds
Testing support for ArcGIS on the Cloud
ArcGIS performance & scalability testing on the cloud
Amazon deployment experience
Requirements sharing – these are just some examples of the requirements that were shared with ArcGIS Online that were built into the product
Testing on the Cloud – note that Marwa will go into detail.
Web Mapping for Conservation Workshop

27. Questions
Web Mapping for Conservation Workshop