Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Middleware Diarrhea and Other Ailments Michael Stonebraker Adjunct Professor Massachusetts Institute of Technology

Similar presentations


Presentation on theme: "1 Middleware Diarrhea and Other Ailments Michael Stonebraker Adjunct Professor Massachusetts Institute of Technology"— Presentation transcript:

1 1 Middleware Diarrhea and Other Ailments Michael Stonebraker Adjunct Professor Massachusetts Institute of Technology

2 2 M.I.T. OutlineOutline  Too much middleware  XML ailments  Web services ills  Our professional sickness  Too much middleware  XML ailments  Web services ills  Our professional sickness

3 3 M.I.T. Client-Server Got Replaced by N-Tier Computing  The Web  Gizmos  Scalability and management problems with client server  The Web  Gizmos  Scalability and management problems with client server

4 4 M.I.T. Humility Lesson  We all sold client-server hard  during the 80’s  and even into the 90’s  Less than 10 years later  it is the worst idea on the planet  We all sold client-server hard  during the 80’s  and even into the 90’s  Less than 10 years later  it is the worst idea on the planet We should feel really dumb!

5 5 M.I.T. N-Tier Computing Produced Lots of Middleware  App servers  EAI/messaging  ETL  Federators  Workflow  CMS  Portals  DBMS  App servers  EAI/messaging  ETL  Federators  Workflow  CMS  Portals  DBMS

6 6 M.I.T. Middleware Diarrhea  Average enterprise has  one (or more) app servers  one (or more) EAI packages  one (or more) ETL packages  one (or more) portal products  one (or more) application packages  and maybe someday a federated DBMS  Average enterprise has  one (or more) app servers  one (or more) EAI packages  one (or more) ETL packages  one (or more) portal products  one (or more) application packages  and maybe someday a federated DBMS

7 7 M.I.T. All of these systems All of these systems  Contain transformation engines  And often do function activation (app service)  And often have adapters to legacy systems  Contain transformation engines  And often do function activation (app service)  And often have adapters to legacy systems Huge overlap in functionality!!

8 8 M.I.T. Less Moving Parts  Less systems  More uniformity  Less duplication  Less systems  More uniformity  Less duplication

9 9 M.I.T. Less Systems  Less system administrators  Less training  Less manuals  Less bugs  Less cross system issues  Less system administrators  Less training  Less manuals  Less bugs  Less cross system issues

10 10 M.I.T. More Uniformity  Every island has  memory management  security model  threading model  Less is better  Every island has  memory management  security model  threading model  Less is better

11 11 M.I.T. Less Duplication  Most of the islands support transformations  reasonable chance you will do each one 6 or more times  maintenance headache  Most of the islands support transformations  reasonable chance you will do each one 6 or more times  maintenance headache

12 12 M.I.T. So How To Consolidate……  Converge app server into OR DBMS  dumbest OR query is execute function  Converge app server into OR DBMS  dumbest OR query is execute function Remember that everything looks like a nail to the guy with the hammer!

13 13 M.I.T. Pictorially client DBMS component

14 14 M.I.T. This Requires….  DBMS to send queries to other DBMSs  I.e. be a data federator  Load balance also requires a federator  DBMS to send queries to other DBMSs  I.e. be a data federator  Load balance also requires a federator

15 15 M.I.T. Best of Breed Federators  Support schema heterogeneity  by executing OR functions  Support materialized views  to cache static data  Support schema heterogeneity  by executing OR functions  Support materialized views  to cache static data

16 16 M.I.T. Less Moving Parts….  Federators dominate ETL  ETL only supports “push”  federators do both “push” and “pull”  Federators dominate ETL  ETL only supports “push”  federators do both “push” and “pull”

17 17 M.I.T. WorkflowWorkflow  A collection of rules  who’s allowed to buy what  and who must approve it  Best considered as a boxes and arrows diagram  And compiled into components to run on an app server  A collection of rules  who’s allowed to buy what  and who must approve it  Best considered as a boxes and arrows diagram  And compiled into components to run on an app server

18 18 M.I.T. Workflow Framework -- PO’s manager PO Big? no Laven yes IT? no yes

19 19 M.I.T. Data Intensive Workflow Should Move Inside an OR DBMS  GUI for “boxes and arrows”  Compiler for the diagram  processing steps become components  business rules become triggers  all data flow inside the DBMS  Worked great in Media/360  GUI for “boxes and arrows”  Compiler for the diagram  processing steps become components  business rules become triggers  all data flow inside the DBMS  Worked great in Media/360

20 20 M.I.T. Why?  Big Big Big performance advantage  no polling of the DBMS  no data movement  easy to change!  Big Big Big performance advantage  no polling of the DBMS  no data movement  easy to change! Watch for Informix product in this area!

21 21 M.I.T. NirvanaNirvana  One integrated system that does  federation  EAI  app service  With a single transformation system  Based on DBMS technology (or something else….)  One integrated system that does  federation  EAI  app service  With a single transformation system  Based on DBMS technology (or something else….)

22 22 M.I.T. XMLXML  Good for content storage and movement  Good as “on the wire” format for data movement  as long as you don’t need to send a lot of stuff fast  Bad for data storage!  Good for content storage and movement  Good as “on the wire” format for data movement  as long as you don’t need to send a lot of stuff fast  Bad for data storage!

23 23 M.I.T. History Lesson  1960’s  IMS and IDMS get traction  customers start complaining about rewriting everything when schema changes  1960’s  IMS and IDMS get traction  customers start complaining about rewriting everything when schema changes

24 24 M.I.T. History Lesson  1970  Codd writes pioneering paper  starts a decade long argument between IMS/CODASYL advocates and Codd supporters  1970  Codd writes pioneering paper  starts a decade long argument between IMS/CODASYL advocates and Codd supporters

25 25 M.I.T. Net-Net of Argument  Putting semantics into data order is bad  restricts storage options  Hidden meaning bad  no self-defining fields  Putting semantics into data order is bad  restricts storage options  Hidden meaning bad  no self-defining fields

26 26 M.I.T. Net-Net of Argument  Data independence is good  schemas change often  don’t want to rewrite anything when this happens  Data independence is good  schemas change often  don’t want to rewrite anything when this happens

27 27 M.I.T. Net-Net of Argument  Complexity is bad  high level query languages are good  KISS arguments  Call these three premises “Codd’s laws”  Complexity is bad  high level query languages are good  KISS arguments  Call these three premises “Codd’s laws”

28 28 M.I.T. History Lesson History Lesson  1983(?)  Codd wins Turing award  acknowledgement for being right  1983(?)  Codd wins Turing award  acknowledgement for being right

29 29 M.I.T. XML in This Historical Light XML in This Historical Light  Most of the bad features of IMS/Codasyl  allows semantics in data order  data independence will be a challenge  try updates on inverted hierarchies  look at IMS LDBs  more complex than Codasyl  Most of the bad features of IMS/Codasyl  allows semantics in data order  data independence will be a challenge  try updates on inverted hierarchies  look at IMS LDBs  more complex than Codasyl

30 30 M.I.T. Our Field Our Field  We look a little silly saying  an idea renounced in the 1970’s  is back  Leading our colleagues to ask “What’s different?”  if somebody disproved Codd’s laws; they didn’t tell me…..  We look a little silly saying  an idea renounced in the 1970’s  is back  Leading our colleagues to ask “What’s different?”  if somebody disproved Codd’s laws; they didn’t tell me…..

31 31 M.I.T. How to Win the Turing Award Circa 2020  2000’s  XML data storage gets traction  2010  dust off Codd’s paper  Wait 10 years to be proven right  2000’s  XML data storage gets traction  2010  dust off Codd’s paper  Wait 10 years to be proven right

32 32 M.I.T. In Any Case  In line tags turn 1Tbyte of EMP data into 10 Tbytes of EMP data  Won’t store anything big in native XML  will use something else….  like what?  In line tags turn 1Tbyte of EMP data into 10 Tbytes of EMP data  Won’t store anything big in native XML  will use something else….  like what?

33 33 M.I.T. OR DBMS OR DBMS  XML is merely this year’s data type  Next year it will be WML or …  and there will be a next year….  XML is merely this year’s data type  Next year it will be WML or …  and there will be a next year….

34 34 M.I.T. XMLSchema XMLSchema  Contains the kitchen sink  Complexity run amok  diarrhea from the SGML types  Includes lots of known hard stuff  e.g. union types  Contains the kitchen sink  Complexity run amok  diarrhea from the SGML types  Includes lots of known hard stuff  e.g. union types

35 35 M.I.T. Xquery Xquery  Mostly syntactic sugar on OR SQL  // is a user-defined function in Informix OR engine  Try to keep the semantics close to OR SQL  Mostly syntactic sugar on OR SQL  // is a user-defined function in Informix OR engine  Try to keep the semantics close to OR SQL

36 36 M.I.T. Another History Lesson  Typical enterprise wanted data integration for business analysis badly  needed data in a variety of systems  in a variety of formats  often with no unique ids  often with incompatible semantics  2 day delivery means lots of things  often dirty  Typical enterprise wanted data integration for business analysis badly  needed data in a variety of systems  in a variety of formats  often with no unique ids  often with incompatible semantics  2 day delivery means lots of things  often dirty

37 37 M.I.T. ETL Warehouse Projects of the 90’s  Well into 8 digits  Usually a factor of three behind schedule  Delivering a factor of 3 less stuff  Everybody dented their pick on semantic heterogeneity  which is hard, hard, hard  and not solved by the blizzard of 3 letter acronyms from Redmond  Well into 8 digits  Usually a factor of three behind schedule  Delivering a factor of 3 less stuff  Everybody dented their pick on semantic heterogeneity  which is hard, hard, hard  and not solved by the blizzard of 3 letter acronyms from Redmond

38 38 M.I.T. Web Services  Will be a long time coming outside of simple domains (where there is no data integration to deal with)  E.g. catalog management  Grainger perspiration….  Will be a long time coming outside of simple domains (where there is no data integration to deal with)  E.g. catalog management  Grainger perspiration….

39 39 M.I.T. The Depressing State of Affairs  ~50-75% of IT projects fail  if we built bridges, our profession would be fired  and the same mistakes are repeated over and over (excessive ambition, rolling specs, bad design, failure to load a large data set early)  ~50-75% of IT projects fail  if we built bridges, our profession would be fired  and the same mistakes are repeated over and over (excessive ambition, rolling specs, bad design, failure to load a large data set early)

40 40 M.I.T. What To Do?  We typically don’t teach this stuff (and do a serious disservice to our students)  probably because we don’t (can’t) spend any time in industry to figure it out  We typically don’t teach this stuff (and do a serious disservice to our students)  probably because we don’t (can’t) spend any time in industry to figure it out Action item: at the very least read a couple of Robert L. Glass’s books

41 41 M.I.T. The Depressing State of Affairs  Hardware “half-life” is 18 months  Software half-life is 18 years (or more)!  In 25 years we moved from  C to Java  SQL to Xquery  Hardware “half-life” is 18 months  Software half-life is 18 years (or more)!  In 25 years we moved from  C to Java  SQL to Xquery

42 42 M.I.T. What To Do?  Much higher level design environments  vis  workflow  special purpose languages (report writers,…)  And stop turning down papers on this stuff  Much higher level design environments  vis  workflow  special purpose languages (report writers,…)  And stop turning down papers on this stuff

43 43 M.I.T. Grand Challenge  Improve application productivity (probability of success * programmer productivity) by 2 this decade


Download ppt "1 Middleware Diarrhea and Other Ailments Michael Stonebraker Adjunct Professor Massachusetts Institute of Technology"

Similar presentations


Ads by Google