3 Actionable Ideas Co-lo if possible Adopt a “university model” Recruit from the topRecruit for passion and a desire to have impactInstall a Research Program Management organization to orchestrate tech-transferInstitute an annual TechFest
4 Innovation Build versus Buy versus Invest Build: Have in-house researchBell Labs, IBM, GM, Pfizer, Merc, Microsoft…Buy: Acquire startups or whole companiesIBM, Cisco, Intel, Microsoft, Pfizer, Merc…Invest: All boats riseGovernment research fundingAll 3 approaches validComplement one another
5 Companies Are Different IBMS G&A23%Product31%Gross38%other2%R&D6%IntelR&D15%S G&A16%Product19%Gross50%MicrosoftGross40%Product18%S G&A27%R&D15%OracleR&D12%Gross36%Product26%S G&AAccentureGross32%Product47%S G&A21%R&D0%HPS G&A16%Product44%Gross27%other7%R&D6%CiscoS G&A25%Product33%Gross26%R&D16%DELLR&D1%Gross18%Product73%S G&A8%EDSS G&A9%Product69%Gross8%other14%R&D0%Selected IT company FY02 R&D budgets:Notice that R&D is correlated with marginIBM and HP have large service revenues So, their “real” R&D investment rate is higherDell, Accenture, EDS have modest R&D – innovate in other ways
6 Microsoft Is Different It is a software company:Almost entirely an IP companyMargins on successful products are enormousThe cost of failure is enormous – missed marketIt is BIG and so must look for BIG betsHigh-velocity business: Product mix shifts every decade. If you miss a shift, you are dead.
7 Most R&D Is D How to Do Basic Research in Industry Most R&D Is D How to Do Basic Research in Industry? Critical questions (from Rick Rashid)How can I create and maintain a world class research organization in an industrial setting?How do I keep the lines of communication open between product teams and researchers?How do I get new technology into products quickly?
8 Approach Adapt the Academic Model Organizational goal: Advance state of the artUniversity organizational modelFlat structure, critical mass groupsOpen research environmentAggressive publication in peer-reviewed literatureFrequent visitors, daily seminarsStrong ties to University ResearchNearly 15% of basic research budget directly invested in UniversitiesLab grants, research grants, fellowships, etc.Hundreds of interns and visitors
10 Microsoft Research Expanding the State of the Art Thousands of peer-reviewed publications10%…30% of papers at our focus conferences graphics, programming, systems, data management…Community leadershipProfessional societiesJournalsConferencesMentoring InternsHosting academic summers and sabbaticalsSpecial workshops
11 How To Build A Group Identify a promising area Hire the leader (internal or external)Support her/himBuild team around senior researcherLook for people whoWant to have impactHave passion for their ideasSame template works for whole labsCambridge, Beijing, Silicon Valley
12 Keeping Open The Lines Of Communication To Product Teams Co-location helps: 75% “on campus”“How can I help?” attitude demonstrates willingness to “get dirty” to help product succeedProduct group spin-offs build strong ties Over time a number of product groups evolved from research (e.g., Windows Media)Researchers involved in all corporate product reviews
13 MSR Relationship To MS Products Virtually every research group actively engaged with product groupsE.G., Windows, Office, streaming media, SQL, Exchange, IIS, commerce server, visual studio, office, consumer products, MSN, etc.Tech transfer:IdeasCodePeopleContactsRecruiting
14 Focused Technology Transfer Quickly getting technology into products Program management team with sole focus on tech transferResearchers on product “advisory” boards“Mind-swaps” – joint product/research off-sitesJoint product/research teams, e.g.,ClearType (Windows XP)Datamining (SQL 2000)Natural Language & Speech (Office)TabletPCSmart Personal Objects (SPOT)Encourage and recognize contributions
15 MSR Techfest Internal open house for Microsoft Research Annual event since 2001~ 7000 attendees170 demos, 26 lectures“Research in progress”Breadboard demosThis is research idea/prototypeGreat networking event:Breaks down barriersSerendipitous connections.
16 Examples Of Technology Transfer Critical support technologiesMemory Optimization Technology enabled sim-ship of Win95/Office95Automated bug detection in Windows 2000Key technologies that drive productsE.G., MS audio 4.0, ClearType, intelligent search, collaborative filtering, Intellimirror, etc.Incubated major productsWindows streaming mediaWindows CE, TabletPC, eBookEcommerce, DataminingNatural language and speech technologies, etc.
17 MSR Mission StatementExpand the state of the art in each of the areas in which we do researchRapidly transfer innovative technologies into Microsoft productsEnsure that Microsoft products have a future
18 Personal Examples of R&D Scaleable ServersTerraServerSkyServerDatabasesData Cube, Snapshot IsolationSQL Stress testingReliable MulticastPersonal Media Management
21 TerraServer Tomorrow Mirrored System versus SAN 3 mirrored DB servers + spare versus 4 DB serversCommodity versus EnterpriseWhite box Dual Xeon versus 8-way brandedDAS 250GB SATA versus FC-SAN 73GB SCSINo Tape versus LTO Tape Robot$0.1M versus $1.8MGeoplex: 2 sitesYou can afford 2!KVM / IP
22 World Wide Telescope http://www.voforum.org/ Premise: Most Astro data is onlineSo, the Internet is the world’s best telescope:Has data on every part of the skyIn every measured spectral bandAs deep as the best instrumentsIt is up when you are up; the “seeing” is always great (no working at night, no clouds no moons no…)It’s a smart telescope:links objects and data to literature on them
23 Next-Generation Data Analysis Looking forNeedles in haystacks – the Higgs particleHaystacks: Dark matter, Dark energyNeedles are easier than haystacksGlobal statistics have poor scalingCorrelation functions are N2, likelihood techniques N3As data and computers grow at same rate, we can only keep up with N logNA way out?data is fuzzy, answers are approximateRequires combination of statistics and computer science
24 Data Federations Of Web Services Massive datasets live near their owners:Near the instrument’s software pipelineNear the applicationsNear data knowledge and curationSuper Computer centers become Super Data CentersEach Archive publishes a web serviceSchema: documents the dataMethods on objects (queries)Scientists get “personalized” extractsUniform access to multiple ArchivesA common global schemaChallenge:What is the object model for your science?Federation
25 Web Services – The Key? Web SERVER: Web SERVICE: YourprogramWebServicehttpWeb SERVER:Given a url + parametersReturns a web page (often dynamic)Web SERVICE:Given a XML document (soap msg)Returns an XML documentTools make this look like an RPC.F(x,y,z) returns (u, v, w)Distributed objects for the web.+ naming, discovery, security,..Internet-scale distributed computingWeb pageYourprogramWebServicesoapDataIn your address spaceobject in xml
26 Federating Astronomy Archives IRAS 25mGreat Test for data mining algorithmsIt is real and well documented dataHigh-dimensional data (with confidence intervals)Spatial dataTemporal dataMany different instruments from many different places and many different timesFederation is a goalThere is a lot of it (petabytes)Can share cross companyUniversity researchers2MASS 2mDSS OpticalIRAS 100mWENSS 92cmNVSS 20cmROSAT ~keVGB 6cm
27 SkyServer – One such archive SkyServer.SDSS.org Sloan Digital Sky Survey Pixels + Data Mining400 attributes per “object”Spectrograms for 1%Demo: pixel space record space set space teaching
28 SkyQuery: Federating Archives http://skyquery.net/ Distributed Query tool using a set of web servicesFederates ten astronomy archives from Pasadena, Chicago, Baltimore, Cambridge (England)Implemented in C# and .NETAllows queries like:SELECT o.objId, o.r, o.type, t.objIdFROM SDSS:PhotoPrimary o,TWOMASS:PhotoPrimary tWHERE XMATCH(o,t)<3.5AND AREA(181.3,-0.76,6.5)AND o.type=3 and (o.I - t.m_j)>2
29 SkyQuery Structure Portal Each SkyNode publishes Plans Query (2 phase) Integrates answersIs itself a web serviceEach SkyNode publishesSchema Web ServiceDatabase Web ServiceImage CutoutINTSDSSSkyQueryPortalFIRST2MASS
30 Databases Theory to practice Data CubeWrote paperSQL Server product and ISO Standard adopted ideaSnapshot IsolationPaper in 1996Product in 2004oldReaderversionnew
31 Databases Stress Test Generate millions of random SQL queries SqlServerGenerate millions of random SQL queriesSend them to 4 different productsCompare the answers:If all agree, good!If not, a bug somewhereFound many bugs in DB productsMuch appreciated by MS DB groupTool cloned by other DB vendorsDB2=OracleInformix
32 SQL Automated Test Example Four SQL systems on 2,000 statementsCaseW X Y ZAll four agree 84%W,X, and Y agree 95%Problem with intermediate table.Error
33 PGM Pretty Good Multicast Reliable multicast protocolScales using hierarchy, suppression, and FEC “on-demand” (FEC on-demand is our contribution)Joint work with Cisco and othersIETF standardImplemented prototype (Multicast PowerPoint)Shipped in Windows XP
34 MyLifeBits “A lifetime store of everything” The experiment: digitizing Gordon Bell’s lifeThe software:Based on SQL serverTools to capture web pages, IM chats, TV, radio & telephoneReports, links, full text search, pivot by time or any other attribute
35 MyLifeBits Software Internet MyLifeBits store Radio capture tool Telephone capture toolPocketPC transfer toolPocketRadio playerTV capture toolMyLifeBits storeInternetRadio EPG toolTV EPG download toolMAPI interfaceLegacy clientBrowser tooldatabasefilesLegacy applicationsMyLifeBits ShellVoice annotation toolText annotation tool
36 Research Failures Not everything is a success We had technology transfer failuresWe had projects with little impactSuccess and Failure depend on environmentEven if you have a GREAT! ideaThere are many exogenous factors in technology transferAnd, sometimes the idea or focus is wrongAllow people to fail once or twice.
37 Summary Actionable Ideas Co-lo if possibleAdopt a “university model”Recruit from the topRecruit for passion and a desire to have impactInstall a Research Program Management organization to orchestrate tech-transferInstitute an annual TechFest