1. Telepresence: An Umbrella Research Topic
Jim Gray
Microsoft Research

2. NSF: Nerve Center of Science If it’s not broke, don’t fix it. But….
US Science is the engine of progress BUT…..
Best and brightest are spending increasing time fundraising
Seems excessive to me.
Venture capital community is richer and more generous than NSF

3. Outline (ambitious!) Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution: processing moves to transducers

4. Microsoft Research -- 1991 Founded in 1991
Goal: pursue strategic technologies for Microsoft
Original research groups:
Natural Language Processing
Operating Systems
Programming Languages
Overall size < 20 at the end of 1992

5. Microsoft Research -- 1998 280 Researchers in 25 areas
Operating systems to Statistical Physics
Research lab locations:
Redmond, Cambridge, San Francisco
Internationally recognized research teams
Hundreds of publications, presentations
Leadership roles in professional societies, journals, conferences

6. MS Research Areas
Operating systems, languages, compilers, virtual machines, networking, wireless computing, fault-tolerance, large scale servers, security
Natural language, speech, vision, graphics, decision theory, information retrieval, UI, collaboration, statistics, signal processing
Cryptography, statistical physics and discrete mathematics

7. Growing Fast Grew 4x from ‘94 to ‘97
Decided in ‘97 to grow by a 3x in 3 years
200 in FY97 => 600 in FY00, primarily in Redmond
Major impact on MS products
Virtually all MS products shipped today use technology from MS Research
Critical role in MS growth
Pioneering research in software that allows computers to see, hear, speak and understand

8. Microsoft Research Philosophy
University organizational model
Flat structure, critical mass groups
Open research environment
Aggressive publication of research results in literature and on world wide web
Frequent visitors, daily seminars
Over 70 visiting professors and interns in 1997
Over 110 visiting researchers in 1998

9. Some Key Senior Researchers
Systems
Rick Rashid, Butler Lampson, Gordon Bell
Anoop Gupta, Roger Needham, Chuck Thacker
Databases & Data Mining
David Lomet, Jim Gray, Usama Fayyad
Graphics
Jim Kajiya, Jim Blinn, Alvy Ray Smith, Michael Cohen
Speech & Language
Karen Jensen, George Heidorn, X.D. Huang, Alex Acero, Hsiao-Wuen Hon, Scott Meredith

10. Some Key Senior Researchers
UI Design, Intelligent Systems, IR
George Robertson, Linda Stone, Susan Dumais, David Heckerman, Eric Horvitz, Jack Breese
Computer Vision & Signal Processing
Steve Shafer, Rick Szeliski, P. Anandan, Rico Malvar
Cryptography & Theory
Yacov Yacobi, Jennifer Chayes, Christian Borg, Michael Freedman
Languages & Compilers
Daniel Weise, Chris Fraser, Amitabh Srivastava, Luca Cardelli, David Hanson, Charles Simonyi, Todd Proebsting

11. Microsoft Research 1997 BusinessWeek Poll of Academia:
Voted #7 lab (overall) in Computer Science
Voted #3 industrial research lab (after Bell Labs and IBM Research)
Voted #2 most desirable lab to work (after Stanford)

12. Outline (ambitious!) Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution: processing moves to transducers

13. Gordon Bell on Tele Presentations

14. Motivation: Telepresentations
Presenter and/or audience telepresent
NOT: meeting or collaboration settings
Forget the nasty social issues!
Mostly one-way

15. Telepresentation Elements
Slides
Audio
Video
Script, text comments, hyperlinks, etc.

16. Telepresentations: The Essentials
Slide and audio a must
Add some video (low quality) to make us feel good
Storage and transmission costs low

17. Telepresentations: The Killer App
Increased attendance & lower travel costs
Practical and low-cost NOW
e.g. ACM97 - 2,000 visitors in real space, 20,000 visitors on Internet

18. Today’s Experiment Would you like to pause, rewind, browse?
Do you wish you could have seen this
At home?
At another time?
How much does a present speaker add? How much would you pay for real presence?

19. Outline (ambitious!) Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution: processing moves to transducers

20. Changing role of computation
Past: Computers for:
computing (Cray)
business data processing (IBM)
“document” creation (PC)
Future: Computers for:
understanding & learning
communicating
consuming & entertaining
Requires new User Interface to machines

21. Flows

22. Making “Flows” a Reality
Computer Graphics
Creating realistic looking environments, people
Computer Vision
Analyzing posture, gaze, gestures
Speech input/output
Natural Language
Analysis, IR
Implicit requests for information

23. Building life-like human characters

24. Recognizing gestures
Live video
Area of motion
H flow
V flow

25. Generating life-like speech from textual data
Data-driven stochastic speech
Natural sounding
Rapid, automatic customizability
Examples
Synthetic voice w/ transplanted speech contours

26. Artificial singing AT&T Voder, 1962, by Homer Dudley
Daisy (Inspiration for HAL’s voice in 2001)
Microsoft Research Whistler, 1997
Scarborough Fair

27. Analyzing language Language recognition shipped in Word 97
General purpose text-critiquing, summarization, Japanese word-breaking

28. Inside The Office Grammar Checker

29. Understanding language: MindNet
A huge language knowledge base
Automatically created from dictionaries
Words (nodes) linked by relationships
Millions of links
Recently added (Encarta) encyclopedia knowledge

30. MindNet -- “Going to the birds”
chicken
Is_a
poultry
Purpose
supply
Typ_obj
clean
Is_a
Quesp
smooth
keep
Is_a
hen
duck
Typ_0bj_of
Is_a
Typ_obj
Purpose
meat
preen
Typ_subj
Cause
Is_a
egg
Means
quack
Not_is_a
plant
chatter
Typ_subj
animal
Is_a
Is_a
Is_a
Is_a
creature
bird
Is_a
make
Typ_obj
sound
Part
feather
Is_a
gaggle
Is_a
goose
wing
Is_a
limb
peck
Is_a
Is_a
Means
Typ_subj_of
claw
Is_a
Is_a
beak
Part
Part_of
hawk
Is_a
Typ_obj
strike
Typ_subj_of
fly
leg
turtle
Is_a
catch
Typ_subj
Is_a
arm
bill
face
opening
Locn_of
mouth
Is_a

31. Changing balance between user & software systems
Yesterday:
Applications were single programs running in isolation
Users used to (more or less) understand systems that they used
Today:
Componentized applications operate in concert
Sophisticated users understand only small percentage of systems they use

32. Tomorrow’s Systems and Applications
Users will not be able to predict
where computations will be performed,
when they will be performed or
by what software components
Gap between system capabilities and user understanding will grow to the point that the only way user will be able to use system is through assisting agents

33. Examples of user agents & implicit actions
Lumiere (Office 97)
Monitoring user and program events to provide user help and assistance
Implicit queries
Inferring information needs from browsing
Lookout/SpamKiller
Monitoring mail activity to auto-categorize it

34. User Modeling Models of a user’s informational goals
User’s query (when available…)
User’s background
Acute and long-term search activity
Acute actions with objects and documents
Program data structures
Explicit and implicit information access and display

35. Outline (ambitious!) Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution: processing moves to transducers

36. Some Tera-Byte Databases
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
The Web: 1 TB of HTML
TerraServer 1 TB of images
Several other 1 TB (file) servers
Hotmail: 7 TB of
Sloan Digital Sky Survey: 40 TB raw, 2 TB cooked
EOS/DIS (picture of planet each week)
15 PB by 2007
Federal Clearing house: images of checks
15 PB by 2006 (7 year history)
Nuclear Stockpile Stewardship Program
10 Exabytes (???!!)

37. Library of Congress (text)
Info Capture
A letter
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
A novel
You can record everything you see or hear or read.
What would you do with it?
How would you organize & analyze it?
A Movie
Library of Congress (text)
LoC (image)
All Disks
Video 8 PB per lifetime (10GBph)
Audio 30 TB (10KBps)
Read or write: 8 GB (words)
See:
All Tapes

38. Library of Congress (text)
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
A letter
A novel
A Movie
Library of Congress (text)
LoC (image)
LoC (sound + cinima)
All Disks
All Photos
All Tapes
All Information!

39. Michael Lesk’s Points www.lesk.com/mlesk/ksg97/ksg.html
Soon everything can be recorded and kept
Most data will never be seen by humans
Precious Resource: Human attention Auto-Summarization Auto-Search will be a key enabling technology.

40. Outline (ambitious!) Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution: processing moves to transducers

41. Put Everything in Future (Disk) Controllers (it’s not “if”, it’s “when
Put Everything in Future (Disk) Controllers (it’s not “if”, it’s “when?”) Acknowledgements: Dave Patterson explained this to me a year ago Kim Keeton Erik Riedel Catharine Van Ingen
BARC started in 1995 with Jim Gray and Gordon Bell. We are part of Microsoft Research with a focus on Scaleable Servers (Gray, Barrera, Barclay, Slutz, VanIngen) and Telepresence (Bell, Gemmell).
In 1996 we grew to a staff of 6 and moved to our current location in downtown San Francisco (at the east end of Silicon Gulch).
We have close ties to the SQL, MTS, NT, PowerPoint, and NetMeeting groups. We also collaborate with UC Berkeley, Cornell, and Wisconsin on Scaleable computing, with UC Berkeley and U. Virginia on Telepresence.
Each summer we host two interns.
Our web site is
BARC is located at 301 Howard St, #830, San Francisco CA.94105
Humor: our next door neighbor is the Justice Department (Environmental Division). So the sign in the lobby reads:
Microsoft 830 <= Justice Department 870 =>
Helped me sharpen
these arguments

42. Remember Your Roots

43. Technology Drivers: Disks
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
Technology Drivers: Disks
Disks on track
100x in 10 years 2 TB 3.5” drive
Shrink to 1” is 200GB
Disk replaces tape?
Disk is super computer!

44. Data Gravity Processing Moves to Transducers
Move Processing to data sources
Move to where the power (and sheet metal) is
Processor in
Modem
Display
Microphones (speech recognition) & cameras (vision)
Storage: Data storage and analysis

45. It’s Already True of Printers Peripheral = CyberBrick
You buy a printer
You get a
several network interfaces
A Postscript engine
cpu,
memory,
software,
a spooler (soon)
and… a print engine.

46. All Device Controllers will be Cray 1’s
TODAY
Disk controller is 10 mips risc engine with 2MB DRAM
NIC is similar power
SOON
Will become 100 mips systems with 100 MB DRAM.
They are nodes in a federation (can run Oracle on NT in disk controller).
Advantages
Uniform programming model
Great tools
Security
economics (CyberBricks)
Move computation to data (minimize traffic)
Central
Processor & Memory
Tera Byte
Backplane

47. Basic Argument for x-Disks
Future disk controller is a super-computer.
1 bips processor
128 MB dram
100 GB disk plus one arm
Connects to SAN via high-level protocols
RPC, HTTP, DCOM, Kerberos, Directory Services,….
Commands are RPCs
Management, security,….
Services file/web/db/… requests
Managed by general-purpose OS with good dev environment
Apps in disk saves data movement
need programming environment in controller

48. The Slippery Slope If you add function to server
Nothing =
Sector Server
If you add function to server
Then you add more function to server
Function gravitates to data.
Fixed App Server
Something =
Everything =
App Server

49. Why Not a Sector Server? (let’s get physical!)
Good idea, that’s what we have today.
But
cache added for performance
Sector remap added for fault tolerance
error reporting and diagnostics added
SCSI commends (reserve,.. are growing)
Sharing problematic (space mgmt, security,…)
Slipping down the slope to a 2-D block server

50. Why Not a 1-D Block Server? Put A LITTLE on the Disk Server
Tried and true design
HSC - VAX cluster
EMC
IBM Sysplex (3980?)
But look inside
Has a cache
Has space management
Has error reporting & management
Has RAID 0, 1, 2, 3, 4, 5, 10, 50,…
Has locking
Has remote replication
Has an OS
Security is problematic
Low-level interface moves too many bytes

51. Why Not a 2-D Block Server? Put A LITTLE on the Disk Server
Tried and true design
Cedar -> NFS
file server, cache, space,..
Open file is many fewer msgs
Grows to have
Directories + Naming
Authentication + access control
RAID 0, 1, 2, 3, 4, 5, 10, 50,…
Locking
Backup/restore/admin
Cooperative caching with client
File Servers are a BIG hit: NetWare™
SNAP! is my favorite today

52. Why Not a File Server? Put a Little on the Disk Server
Tried and true design
Auspex, NetApp, ...
Netware
Yes, but look at NetWare
File interface gives you app invocation interface
Became an app server
Mail, DB, Web,….
Netware had a primitive OS
Hard to program, so optimized wrong thing

53. Why Not Everything? Allow Everything on Disk Server (thin client’s)
Tried and true design
Mainframes, Minis, ...
Web servers,…
Encapsulates data
Minimizes data moves
Scaleable
It is where everyone ends up.
All the arguments against are short-term.

54. The Slippery Slope If you add function to server
Nothing =
Sector Server
If you add function to server
Then you add more function to server
Function gravitates to data.
Fixed App Server
Something =
Everything =
App Server

55. Disk = Node has magnetic storage (100 GB?) has processor & DRAM
has SAN attachment
has execution environment
Applications
Services
DBMS
RPC, ...
File System
SAN driver
Disk driver
OS Kernel

56. Technology Drivers: System on a Chip
Integrate Processing with memory on one chip
chip is 75% memory now
1MB cache >> 1960 supercomputers
256 Mb memory chip is 32 MB!
IRAM, CRAM, PIM,… projects abound
Integrate Networking with processing on one chip
system bus is a kind of network
ATM, FiberChannel, Ethernet,.. Logic on chip.
Direct IO (no intermediate bus)
Functionally specialized cards shrink to a chip.

57. Technology Drivers: What if Networking Was as Cheap As Disk IO?
Unix/NT 8% 40MBps
TCP/IP
Unix/NT 100% 40MBps
Why the Difference?
Host Bus Adapter does
SCSI packetizing,
checksum,…
flow control
DMA
Host does
TCP/IP packetizing,
small buffers

58. Technology Drivers: The Promise of SAN/VIA:10x in 2 years. http://www
Today:
wires are 10 MBps (100 Mbps Ethernet)
~20 MBps tcp/ip saturates 2 cpus
round-trip latency is ~300 us
In the lab
Wires are 10x faster Myrinet, Gbps Ethernet, ServerNet,…
Fast user-level communication
tcp/ip ~ 100 MBps 10% of each processor
round-trip latency is 15 us

59. SAN: Standard Interconnect
RIP
FDDI
SAN: Standard Interconnect
RIP
ATM
Gbps Ethernet: 110 MBps
LAN faster than memory bus?
1 GBps links in lab.
100$ port cost soon
Port is computer
RIP
SCI
PCI: 70 MBps
RIP
SCSI
UW Scsi: 40 MBps
FW scsi: 20 MBps
RIP FC
scsi: 5 MBps
RIP ?

60. Technology Drivers: 100 GBps Ethernet replaces SCSI
Why I love SCSI
Its fast (40MBps)
The protocol uses little processor power
Why I hate SCSI
Wires must be short
Cables are pricey
pins bend

61. Functionally Specialized Cards
P mips processor
Storage
Network
Display
Today:
P=50 mips
M= 2 MB
ASIC
M MB DRAM
In a few years
P= 200 mips
M= 64 MB
ASIC
ASIC

62. Technology Drivers Plug & Play Software
RPC is standardizing: (DCOM, IIOP, HTTP)
Gives huge TOOL LEVERAGE
Solves the hard problems for you:
naming,
security,
directory service,
operations,...
Commoditized programming environments
FreeBSD, Linix, Solaris,…+ tools
NetWare + tools
WinCE, WinNT,…+ tools
JavaOS + tools
Apps gravitate to data.
General purpose OS on controller runs apps.

63. Basic Argument for x-Disks
Future disk controller is a super-computer.
1 bips processor
128 MB dram
100 GB disk plus one arm
Connects to SAN via high-level protocols
RPC, HTTP, DCOM, Kerberos, Directory Services,….
Commands are RPCs
management, security,….
Services file/web/db/… requests
Managed by general-purpose OS with good dev environment
Move apps to disk to save data movement
need programming environment in controller

64. Summary Microsoft Research (census)
Tele-Presentations (Gordon Bell, Jim Gemmell)
Microsoft Research initiative on Telepresence
What if you could record everything you see & hear?
The architecture revolution: processing moves to transducers