1. What is Information? What will we retrieve with information retrieval?
There are several ways to define “information”
Subjective: People develop models of their environment. Information created by people makes those models more accurate.
Thing/artifact: Information is what’s captured in a book, web page, or other resource.
More information is digital & is increasing

2. Information - wikipedia
Information as a concept has a diversity of meanings, from everyday usage to technical settings. Generally speaking, the concept of information is closely related to notions of constraint, communication, control, data, form, instruction, knowledge, meaning, mental stimulus, pattern, perception, and representation.
Many people speak about the Information Age as the advent of the Knowledge Age or knowledge society, the information society, the Information revolution, and information technologies, and even though informatics, information science and computer science are often in the spotlight, the word "information" is often used without careful consideration of the various meanings it has acquired.

3. How much information is there in the world
Informetrics - the measurement of information
Stored
What can we store
What do we intend to store.
What is stored.
How do we use it
Decision making
Knowledge discovery

4. Aspects of the Information & Data Age
Much information/data will/can be made and stored digitally
Information/data can be automatically processed, mined, and accessed
Why? Moore’s Law

5. Information Age & Data Age
We have entered the information & data age
What is the information age?
When do we leave it and where do we go next?
David Weinberger’s Too Big to Know
What information was

6. Digitization of Everything: the Zettabytes are coming
Soon most everything will be recorded and indexed
Much will remain local
Most bytes will never be seen by humans.
Search, data summarization, trend detection, information and knowledge extraction and discovery are key technologies
So will be infrastructure to manage this.

7. Digital Information Created, Captured, Replicated Worldwide
Exabytes
10-fold Growth in 5 Years!
DVD
RFID
Digital TV
MP3 players
Digital cameras
Camera phones, VoIP
Medical imaging, Laptops,
Data center applications, Games
Satellite images, GPS, ATMs, Scanners
Sensors, Digital radio, DLP theaters, Telematics
Peer-to-peer, , Instant messaging, Videoconferencing,
CAD/CAM, Toys, Industrial machines, Security systems, Appliances
Source: IDC, 2008

8. How much information is there?
Yotta
Zetta
Exa
Peta
Tera
Giga
Mega
Kilo
Soon most everything will be recorded and indexed
Most bytes will never be seen by humans.
Data summarization, trend detection anomaly detection are key technologies
See Mike Lesk: How much information is there:
See Lyman & Varian:
How much information
Everything!
Recorded
All Books MultiMedia
All books
(words)
.Movie
A Photo
A Book
24 Yecto, 21 zepto, 18 atto, 15 femto, 12 pico, 9 nano, 6 micro, 3 milli

11. Big Data: Volume 30KB 5 MB 5 GB 1 TB 1 PB 1.8 ZB 1 YB 5 EB One page
of text
One song
One movie
6 million
books
55 storeys
of DVD
Data
up to
2003
Data
in 2011
NSA
data center
30KB
5 MB
5 GB
1 TB
1 PB
1.8 ZB
1 YB
5 EB
Byte Kilobyte Megabyte Gigabyte Terabyte Petabyte Exabyte Zettabyte Yottabyte
KB MB GB TB PB EB ZB YB
1000 bytes KB MB GB TB PB ZB YB

13. Information Facts
Print, film, magnetic, and optical storage media produced about 5 exabytes of new information in Ninety-two percent of the new information was stored on magnetic media, mostly in hard disks.
How big is five exabytes? If digitized with full formatting, the seventeen million books in the Library of Congress contain about 136 terabytes of information; five exabytes of information is equivalent in size to the information contained in 37,000 new libraries the size of the Library of Congress book collections.
Hard disks store most new information. Ninety-two percent of new information is stored on magnetic media, primarily hard disks. Film represents 7% of the total, paper 0.01%, and optical media 0.002%.
The United States produces about 40% of the world's new stored information, including 33% of the world's new printed information, 30% of the world's new film titles, 40% of the world's information stored on optical media, and about 50% of the information stored on magnetic media.
How much new information per person? According to the Population Reference Bureau, the world population is 6.3 billion, thus almost 800 MB of recorded information is produced per person each year. It would take about 30 feet of books to store the equivalent of 800 MB of information on paper.

14. Moore's Law Defined by Dr. Gordon Moore during the sixties.
Predicts an exponential increase in component density over time, with a doubling time of 18 months.
Applicable to microprocessors, DRAMs , DSPs and other microelectronics.
Monotonic increase in density observed since the 1960s.

15. First Disk 1956 IBM 305 RAMAC 4 MB 50x24” disks 1200 rpm 100 ms access
35k$/y rent
Included computer & accounting software (tubes not transistors)

16. 10 years later
30 MB
1.6 meters

17. Now - Terabytes on your desk
Terabyte external drive for
<$ cents a gigabyte.
In 10 years, 0.3 cent/gigabyte, $3 for a terabyte?

18. Moore’s Law - Density

19. Storage capacity beating Moore’s law
Improvements: Capacity 60%/y Bandwidth 40%/y Access time 16%/y
1000 $/TB today
100 $/TB in 2007
Moores law
58.70%
/year
TB growth
112.30%
/year since 1993
Price decline
50.70%
/year since 1993
Most (80%) data is personal (not enterprise) This will likely remain true.

20. Digital Immortality
Bell, Gray, CACM, ‘01
Requirements for storing various media for a single person’s lifetime at modest fidelity

21. What is Digital Immortality?
Preservation and interaction of digitized experiences for individuals and/or groups
Preservation and access
Active interaction with archives through queries and/or an avatar (agents)
Avatar interactions for group experiences
Issues:
Archiving
Indexing
Veracity
Access

22. All the world’s libraries on your iPod! NY Times Magazine
And you thought finding that
song was hard.
SmartPhone
Storage is practically free
Much is mobile
Access is crucial
Moore’s law keeps on trucking

23. Why Put Everything in Cyberspace?
Low rent
min $/byte
Shrinks time
now or later
Shrinks space
here or there
Automate processing
knowbots
Point-to-Point OR
Broadcast
Immediate OR Time Delayed
Locate
Process
Analyze
Summarize

24. Filling a terabyte in a year
Item
Items/TB
Items/day
300 KB JPEG
3 M
9,800
1 MB Doc
1 M
2,900
1 hour 256 kb/s MP3 audio
9 K 26
1 hour 1.5 Mbp/s MPEG video
290
0.8
Bottom line: we will be able to keep LOTS of video, and vast amounts of smaller data types (audio, photos, documents).
Note: probably not worth the time to delete an object

25. Progress of Science Thousand years ago: science was empirical
describing natural phenomena
Last few hundred years: theoretical branch
using models, generalizations
Last few decades: a computational branch
simulating complex phenomena
Today: (big data/information) data and information exploration (eScience)
unify theory, experiment, and simulation - information driven
Data captured by sensors, instruments or generated by simulator
Processed/searched by software
Information/Knowledge stored in computer
Scientist analyzes database / files using data management and statistics
Network Science
Cyberinfrastructure 33

26. People and Information
People process information based on their experience and context.
Human information processing is affected by emotions and needs.
Your data may be my information
Search engine relevance is the same

27. What is knowledge? Data - Facts, observations, or perceptions.
Information - Subset of data, only including those data that possess context, relevance, and purpose.
Knowledge - A more simplistic view considers knowledge as being at the highest level in a hierarchy with data (at the lowest level) and information (at the middle level).
Data refers to bare facts void of context.
A telephone number.
Information is data in context.
A phone book.
Knowledge is information that facilitates action.
Recognizing that a phone number belongs to a good client, who needs to be called once per week to get his orders.

28. From Facts to Wisdom (Haeckel & Nolan, 1993) one example of the hierarchy

29. What is knowledge?
Knowledge - A more complex view considers knowledge as intrinsically different from information. Instead of considering knowledge as richer or more detailed set of facts, we define knowledge in an area as justified beliefs about relationships among concepts relevant to that particular area.

30. Great Predictions
"Computers in the future may weigh no more than 1.5 tons.” Popular Mechanics, forecasting the relentless march of science, 1949
"I think there is a world market for maybe five computers.” Thomas Watson, chairman of IBM, 1943
"Heavier-than-air flying machines are impossible.” Lord Kelvin, president, Royal Society, 1895.
"Man will never reach the moon regardless of all future scientific advances."Dr. Lee De Forest, inventor of the vacuum tube and father of television.
"Everything that can be invented has been invented.” Charles H. Duell, Commissioner, U.S. Office of Patents, 1899.
“Nobody would ever need more than 640 kilobytes of memory on their personal computer,” 1981, Bill Gates.
Other predictions of Bill Gates?

31. Great Predictions RIGHT! Artificial Intelligence: WRONG!
speech recognition
Some reasoning; computer beats man in chess
Privacy and security problems
Computers can be a pain in the butt
WRONG!
Missed Moore’s law and ubiquity of computers

32. Predicting the future Can we really predict the future?
“The future ain’t what it used to be” Yogi Berra
Can we really predict the future?
Who predicted the implications of the web and search engines?
Social networking?
Can we understand power laws and their implications?
We have no examples of exponential growth in our evolution except plagues.
Can we understand the pervasiveness of computers?

33. Information Science and Data Generation Trends
What does large amounts of information provide?
New opportunities for search!
New discoveries
Business opportunities?
Research opportunities?
Problems?
Wisdom search engine?

34. Thanks to: Jim Gray, Microsoft L. Floridi, Hertfordshire
Robert Allen, Drexel
Wikipedia