Presentation is loading. Please wait.

Presentation is loading. Please wait.

Open Source Intelligence: Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC IOP 06 Sheraton Premier, Tysons Corner, Virginia January.

Similar presentations


Presentation on theme: "Open Source Intelligence: Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC IOP 06 Sheraton Premier, Tysons Corner, Virginia January."— Presentation transcript:

1 Open Source Intelligence: Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC IOP 06 Sheraton Premier, Tysons Corner, Virginia January Access All Intelligence, in All Languages, All the Time

2 About Deep Web Technologies (DWT) Deployed first federated search portal in the Federal Government, 1999 Major clients include: –DOE Office of Scientific & Technical Information –Defense Technical Information Center –Science.gov Alliance –DOE Office of Science –National Agricultural Library DWT is a New Mexico based company focused on providing state-of-the-art software solutions which search, retrieve, aggregate, and analyze content.

3 Open Source Intelligence The Problem: Collecting and analyzing enormous quantities of information in any language, in myriad formats, located anywhere, accessible through a large variety of means, with a majority not accessible through the Internet

4 Shared Challenge: OSINT and Knowledge Discovery/Diffusion OSINT Challenges Knowledge Discovery/ Diffusion Challenges DWT for the past six years has been the lead technical organization addressing these challenges in collaboration with DOE Office of Scientific & Technical Information

5 The DWT Proposition To apply DWTs technology, expertise and ongoing innovations* to address the challenges of OSINT *Developed in partnership with DOE/OSTI

6 Challenges in Working with Thousands of Data Sources Locate Reliable Sources Categorize Sources by Content Configure Sources for Searching Maintain Sources

7 Challenges in Searching Thousands of Sources Automatically Select Sources to Search Perform Many Searches in Parallel Translate, Analyze and Organize Results Relevance Rank Cluster/ Visualize Extract Key Information

8 DWTs State-of-the-art Federated Search Engine Scalable, grid-computing based federated search engine Sophisticated Search Conductor Supports custom connectors Multi-tier relevance ranking Framework accepts integration of advanced linguistic, analyses, and visualization modules ResearchAssistant TM

9 Grid Computing: Distributing the Workload

10 Search Conductor Select sources to search Perform search Deliver results to user Can I get more results from good sources? Enough good results? YES NO

11 Multi-tier Relevance Ranking QuickRank TM – Ranks results based on occurrence of search terms in title and snippet MetaRank TM – Ranks results utilizing custom algorithms applied to metadata DeepRank TM – Downloads and indexes full-text documents

12 Science.gov Alliance Consortium of 12 Federal Government Agencies Dept of Agriculture Dept of Commerce Dept of Defense Dept of Education Dept of Energy Dept of Health/Human Services Dept of Interior Environmental Protection Agency NASA National Science Foundation US Government Printing Office National Archives & Records Administration Sponsoring Science.gov Portal (Access to most of Federal Government R&D

13 Science.gov Advanced Search Page

14 Science.gov Results Page

15 A Science.gov Document

16 Next Steps Identify Sponsors and development partners that can collaborate on the development of a pilot that integrates best- of-breed technologies of value to OSINT. This pilot will result in a portal that aggregates content of different types, generating actionable intelligence.

17 Contact Us Abe Lederman 122 Longview Drive Los Alamos, NM


Download ppt "Open Source Intelligence: Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC IOP 06 Sheraton Premier, Tysons Corner, Virginia January."

Similar presentations


Ads by Google