2. International Directory Network (IDN) Scalability, Security and Interoperability WGISS, 2006 Tom Northcutt Systems Administrator: GCMD September 13, 2006

3. I. Scalability, Interoperability

4. # GCMD/IDN Web Page Hits Since January 2003 Introduction of the new web page Cache opened to Internet Search robots

5. Freetext Freetext (Lucene) Database Index Layer Spatial Spatial Database Index Layer Controlled Vocabulary Controlled Vocabulary Database Layer joins delegates Controller Controller User performs query 1) 2) 3) Middleware Search/Retrieval Component (Integration of spatial, freetext, and controlled queries) 4) Set of unique IDs ResultProcessor Multi-Layer Search Component 5) Title set information, brief summary, dataset links, etc. 6)Returns to user SearchResults Cache 7) Refines search

6. Scalability: Core GCMD/IDN Architecture Complexity:“it is hard to make things look easy.” – These are complex queries, with very fast search results. – Another example: data resolution refinement ● difficult to implement ● Makes it easier for the user

7. Scalability ● Conventional clustering approach – Load balancing – High availability – (source: redhat.com)

8. Scalability: GCMD/IDN Implementation Stateful, Web Proxy Based Clusters http://gcmd.nasa.gov/Keywords.do?...&lbnode=2 http://gcmd.nasa.gov/DocumentBuilder/... Accelerated Caching

9. Scalability: Extensibility of Stateful Web Proxy Clusters http://gcmd.nasa.gov/OAI-script?... http://gcmd.nasa.gov/ontology.wsdl http://gcmd.nasa.gov/soap/http http://gcmd.nasa.gov/xml-rpc http://gcmd.nasa.gov/ajax/some.jsp Harvester SOAP XML-RPC AJAX

10. Scalability: Stateful Web Proxy Clusters How we implemented this architecture: – Modified version of Squid proxy server – Custom perl scripts to implement state and redirection – Dynamic query caching done on the server end so each refinement uses cached results

11. Scalability: Advantages of Web Proxy Clusters for CEOS Partners ● Accelerated Caching ● Load balance nodes ● Stateful architecture ● Open source ● Multiple uses: – Web services – Browse imagery – Metadata search – Data access and retrieval

12. Scalability: Google Map

13. ● Utility: – Google map is a form of spreading the load – Utilize third party resources for map generation – Google’s resources are distributed globally

14. II. Security

15. Security:Transparent Bridge Filters BRIDGEBRIDGE Firewall Network monitoring Port remapping Intrusion detection BRIDGEBRIDGE Internal Firewalling Network monitoring BRIDGE Project Segregation Network protection

16. Security:Transparent Bridge Advantages Applicability for CEOS Partners ● Applicable to heterogeneous environments – Unobtrusive ● No changes needed on servers or network controllers ● May assist with GRID network security requirements – Ultra secure: invisible at the IP level ● Implements emerging security policies – Two factor authentication – Efficient encryption, authentication – Port knocking capabilities ● Open source – Non-proprietary – Universal

17. Conclusion ● IDN continues to grow in popularity – Users – Earth science partnerships ● The system continues to develop – Scalability – Security – Usability – Interoperability

18. End