We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byGabriel Hicks
Modified over 2 years ago
Almaden Research Center © 2006 IBM Corporation IOP 06 Open Source Intelligence Lesson Learned
Almaden Research Center © 2006 IBM Corporation I 2 Issues in using open source for intelligence Growth and complexity of heterogeneous content Not all open source data is equal – Quantities vs. Qualitative Requirements of Ecoinformatics Architectures
Almaden Research Center © 2006 IBM Corporation I 3 Source: IBM 2005 GTO Years = 1Trillion Terabytes of data which is equivalent to all the information consumed visually by all humans in a year Digital content is growing at dramatic rate
Almaden Research Center © 2006 IBM Corporation I 4 Source: IBM 2005 GTO The scale of open source data and its heterogeneous form increases complexity of extracting intelligence Storage online Medical data stored Personal multimedia Surveillance bytes Photos multimedia Scalable Heterogeneity Intelligence Structured data Free from text
Almaden Research Center © 2006 IBM Corporation I 5 Industry Publication Company Internal Content Company Publication Industry Journals Conference Proceedings NGO Publications Website affiliated with an organization User Groups / Forums News Letters Content Aggregators News & Press Releases Legal Filings Government Publications Blogs / Weblogs Non affiliated Websites Qualitative Quantitative Open Source Intelligence from the periphery requires an understanding of its topology, including strengths and weaknesses sources in the periphery These are authoritative sources, where data is trusted and is defended These are credentialed opinions, the source is known and can be weighted Open opinion, it is impossible to verify the authority of the source
Almaden Research Center © 2006 IBM Corporation I 6 Ecoinformatics Architectures need to be multi- layered Cross-Page Annotators Classification Clustering Communities Ranking Applications Network Associations Network Associations Search Topic Tracking Topic Tracking Buzz Analysis Buzz Analysis Per-Page Annotators Auto Entity Spotters Auto Entity Spotters Auto Geography Spotter Auto Geography Spotter Porn & Dup Detection Porn & Dup Detection Customer Taxonomy Spotter Customer Taxonomy Spotter 100s 1000s (pages/second) World Wide Web Blogs Newspapers Licensed Feeds Data Bases Intranet DataTaxonomies Commercial Date Bases Index Store Un-Structured Data DATA ACQUISITION Structured Data Parsing/ Tokenizing Annotation Searching Natural Clustering Natural Clustering Affinity Analysis Affinity Analysis Snippet Analysis Snippet Analysis Trending Performance Management Drug Research Business Insights Workbench Customer Applications 10s Relevancy Volume WebFountain Business Insights Workbench WS OminFind II Index Store DATA ACQUISITION Date Spotters Language Spotters Source Spotters
Almaden Research Center © 2006 IBM Corporation I 7 0.0% 0.5% 1.0% 1.5% 2.0% 2.5% 3.0% 3.5% 4.0% 4.5% Congressman Rob Simmons Douglas Rushkoff Eliot Jardines Major General Patrick Cammaert Mr Arno Reuser Robert Steele Open Source Trend on Web Some event happened in August % of OSI web documents One dominant voice Finding intelligence can require different view of the same information
Almaden Research Center © 2006 IBM Corporation I 8 Context Network of Conference Attendees to auto-spotted Companies and Universities In this network view we dont care about association with Open Source Intelligence but with companies and universities
Almaden Research Center © 2006 IBM Corporation I 9 Computers dont create intelligence, people do – computers enable smart people Not all open source content is equal – know the sources Not every thing you see is right – its all about the CONTEXT Ecoinformation architecture supports - Large scale analytics of open source content - Integration of content other than open source - Power text analytic tools to support analysis of on topic stores Conclusions on Open Source Intelligence
Engineering Technology Management Tracking the Constant of Change Management History Society Legal Aspects LogisticsSupply Chain Systems Engineering Economics.
All Information, All Languages, All the Time Model for Making Amazon the Profitable World Brain Robert David Steele (Vivas) Seattle, 18 January 2007 FINAL.
IMS5401 Web-based Systems Development Topic 2: Elements of the Web (i)Web Services (j)Implications of web technologies for system developers.
8-1 McGraw-Hill/Irwin Copyright © 2007 by The McGraw-Hill Companies, Inc. All rights reserved.
GMD German National Research Center for Information Technology Darmstadt University of Technology Perspectives and Priorities for Digital Libraries Research.
Taxonomy Development in an Enterprise Context Tom Reamy Chief Knowledge Architect KAPS Group Knowledge Architecture Professional Services
Chapter 8 Innovative EC Systems: From E-Government and E-Learning to C2C.
Taxonomy Development An Infrastructure Model Tom Reamy Chief Knowledge Architect KAPS Group Knowledge Architecture Professional Services
REST AND JSON. Web 2.0 What is Web 2.0? Commonly associated with web applications that facilitate interactive information sharing, interoperability, user-centered.
Manchester Computing Cross Council ICT Conference For e-Science & GRID May 2004 End to End Services to support an e-Science Community Professor M.
Intelligence Through Learning from Data Monash University Semester 1, March 2006.
© 2012 IBM Corporation January 19, 2014 The Big Deal About Big Data Dean Compher Data Management Technical Professional for UT, NV
Grey Literature - A Digital Age Mosaic Peter R. Young Chief Asian Division Library of Congress 14 December 2009.
Taxonomy and Knowledge Organization Taxonomy in Context Tom Reamy Chief Knowledge Architect KAPS Group Knowledge Architecture Professional Services
News services provided by the Storting Library Ebbe Aarvåg and Jeannette Berseth.
Learning Objectives Describe the major business intelligence (BI) implementation issues List some critical success factors of BI implementation Describe.
1 Competitive Intelligence and the Web Presented at AMCIS2003 Tampa, Florida by Dr. Robert J. Boncella Washburn University.
© 2007 MIT Sloan School of Management June 2, 2014 Enterprise 2.0 M&A Proposal Timothy B. Jones Sloan Fellow 2007.
August 27, 2002Data Mining and Text-based Information - Mark Wasson 1 Data Mining and Text-based Information Mark Wasson Senior Architect, Research Scientist.
Data Dissemination in the Brazilian Institute of Geography and Statistics Working Session on Emerging Trends and Best Practices in Data Dissemination New.
McGraw-Hill/Irwin Copyright © 2013 by The McGraw-Hill Companies, Inc. All rights reserved.
1 © 2014 by McGraw-Hill Education. This is proprietary material solely for authorized instructor use. Not authorized for sale or distribution in any manner.
Machine readable licences An Introduction to ONIX-PL JIBS-Eduserv Seminar, Wednesday 16 June 2010 Mark Bide – Executive Director, EDItEUR.
Mine your data: contrasting data mining approaches to numeric and textual data sources IASSIST May 2006 conference Ann Arbor, USA Louise Corti UK Data.
Collaborative knowledge management A construction case study
Best Practices to Deploy a Successful Portal Carol Penne – International Monetary Fund Zach Wahl – Project Performance Corporation March 18, 2005 Portal.
© 2016 SlidePlayer.com Inc. All rights reserved.