Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Database Concepts © Leo Mark. 2 Database Concepts © Leo Mark DATABASE CONCEPTS Leo Mark College of Computing Georgia Tech (January 1999)

Similar presentations


Presentation on theme: "1 Database Concepts © Leo Mark. 2 Database Concepts © Leo Mark DATABASE CONCEPTS Leo Mark College of Computing Georgia Tech (January 1999)"— Presentation transcript:

1 1 Database Concepts © Leo Mark

2 2 Database Concepts © Leo Mark DATABASE CONCEPTS Leo Mark College of Computing Georgia Tech (January 1999)

3 3 Database Concepts © Leo Mark Course Contents l Introduction l Database Terminology l Data Model Overview l Database Architecture l Database Management System Architecture l Database Capabilities l People That Work With Databases l The Database Market l Emerging Database Technologies l What You Will Be Able To Learn More About

4 4 Database Concepts © Leo Mark INTRODUCTION l What a Database Is and Is Not l Models of Reality l Why use Models? l A Map Is a Model of Reality l A Message to Map Makers l When to Use a DBMS? l Data Modeling l Process Modeling l Database Design l Abstraction

5 5 Database Concepts © Leo Mark What a Database Is and Is Not l your personal address book in a Word document l a collection of Word documents l a collection of Excel Spreadsheets l a very large flat file on which you run some statistical analysis functions l data collected, maintained, and used in airline reservation l data used to support the launch of a space shuttle The word database is commonly used to refer to any of the following:

6 6 Database Concepts © Leo Mark Models of Reality REALITY structures processes DATABASE SYSTEM DATABASE DML DDL l A database is a model of structures of reality l The use of a database reflect processes of reality l A database system is a software system which supports the definition and use of a database l DDL: Data Definition Language l DML: Data Manipulation Language

7 7 Database Concepts © Leo Mark Why Use Models? l Models can be useful when we want to examine or manage part of the real world l The costs of using a model are often considerably lower than the costs of using or experimenting with the real world itself l Examples: –airplane simulator –nuclear power plant simulator –flood warning system –model of US economy –model of a heat reservoir –map

8 8 Database Concepts © Leo Mark A Map Is a Model of Reality

9 9 Database Concepts © Leo Mark A Message to Map Makers l A model is a means of communication l Users of a model must have a certain amount of knowledge in common l A model on emphasized selected aspects l A model is described in some language l A model can be erroneous l A message to map makers: “Highways are not painted red, rivers don’t have county lines running down the middle, and you can’t see contour lines on a mountain” [Kent 78]

10 10 Database Concepts © Leo Mark Use a DBMS when this is important l persistent storage of data l centralized control of data l control of redundancy l control of consistency and integrity l multiple user support l sharing of data l data documentation l data independence l control of access and security l backup and recovery Do not use a DBMS when l the initial investment in hardware, software, and training is too high l the generality a DBMS provides is not needed l the overhead for security, concurrency control, and recovery is too high l data and applications are simple and stable l real-time requirements cannot be met by it l multiple user access is not needed

11 11 Database Concepts © Leo Mark Data Modeling REALITY structures processes DATABASE SYSTEM MODEL data modeling l The model represents a perception of structures of reality l The data modeling process is to fix a perception of structures of reality and represent this perception l In the data modeling process we select aspects and we abstract

12 12 Database Concepts © Leo Mark Process Modeling REALITY structures processes DATABASE SYSTEM MODEL process modeling l The use of the model reflects processes of reality l Processes may be represented by programs with embedded database queries and updates l Processes may be represented by ad-hoc database queries and updates at run-time DML PROG

13 13 Database Concepts © Leo Mark Database Design l is a model of structures of reality l supports queries and updates modeling processes of reality l runs efficiently The purpose of database design is to create a database which

14 14 Database Concepts © Leo Mark Abstraction l Classification l Aggregation l Generalization It is very important that the language used for data representation supports abstraction We will discuss three kinds of abstraction:

15 15 Database Concepts © Leo Mark Classification In a classification we form a concept in a way which allows us to decide whether or not a given phenomena is a member of the extension of the concept. CUSTOMER Tom Ed Nick... Liz Joe Louise

16 16 Database Concepts © Leo Mark Aggregation In an aggregation we form a concept from existing concepts. The phenomena that are members of the new concept’s extension are composed of phenomena from the extensions of the existing concepts AIRPLANE COCKPIT ENGINE WING

17 17 Database Concepts © Leo Mark Generalization In a generalization we form a new concept by emphasizing common aspects of existing concepts, leaving out special aspects CUSTOMER ECONOMY CLASS BUSINESS CLASS 1 ST CLASS

18 18 Database Concepts © Leo Mark Generalization (cont.) CUSTOMER BUSINESS CLASS 1 ST CLASS Subclasses may overlap Subclasses may have multiple superclasses MOTORIZED VEHICLES AIRBORNE VEHICLES TRUCKSHELICOPTERSGLIDERS

19 19 Database Concepts © Leo Mark Relationships Between Abstractions TTT O OO aggregationgeneralization classification AbstractionConcretization classificationexemplification aggregationdecomposition generalizationspecialization intension extension

20 20 Database Concepts © Leo Mark DATABASE TERMINOLOGY l Data Models l Keys and Identifiers l Integrity and Consistency l Triggers and Stored Procedures l Null Values l Normalization l Surrogates - Things and Names

21 21 Database Concepts © Leo Mark Data Model l data structures l integrity constraints l operations A data model consists of notations for expressing:

22 22 Database Concepts © Leo Mark Data Model - Data Structures l attribute types l entity types l relationship types FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231 DEPT-AIRPORT FLIGHT#AIRPORT-CODE 101atl 912cph 545lax All data models have notation for defining:

23 23 Database Concepts © Leo Mark Data Model - Constraints l Static constraints apply to database state l Dynamic constraints apply to change of database state E.g., “All FLIGHT-SCHEDULE entities must have precisely one DEPT-AIRPORT relationship FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231 Constraints express rules that cannot be expressed by the data structures alone: DEPT-AIRPORT FLIGHT#AIRPORT-CODE 101atl 912cph 545lax 242bos

24 24 Database Concepts © Leo Mark Data Model - Operations l insert FLIGHT-SCHEDULE (97, delta, tu, 258) ; insert DEPT-AIRPORT (97, atl) ; l select FLIGHT#, WEEKDAY from FLIGHT-SCHEDULE where AIRLINE= ‘delta’; Operations support change and retrieval of data: FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231 97deltatu258 DEPT-AIRPORT FLIGHT#AIRPORT-CODE 101atl 912cph 545lax 242bos 97atl

25 25 Database Concepts © Leo Mark Data Model - Operations from Programs declare C cursor for select FLIGHT#, WEEKDAY from FLIGHT-SCHEDULE where AIRLINE= ‘delta’; open C ; repeat fetch C into : FLIGHT#, :WEEKDAY; do your thing; until done; close C; FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231 97deltatu258

26 26 Database Concepts © Leo Mark Keys and Identifiers A key on FLIGHT# in FLIGHT-SCHEDULE will force all FLIGHT#’s to be unique in FLIGHT-SCHEDULE Consider the following keys on DEPT-AIRPORT: Keys (or identifiers) are uniqueness constraints FLIGHT#AIRPORT-CODEFLIGHT#AIRPORT-CODEFLIGHT#AIRPORT-CODEFLIGHT#AIRPORT-CODE DEPT-AIRPORT FLIGHT#AIRPORT-CODE 101atl 912cph 545lax 242bos FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231

27 27 Database Concepts © Leo Mark Integrity and Consistency l Integrity: does the model reflect reality well? l Consistency: is the model without internal conflicts? a FLIGHT# in FLIGHT-SCHEDULE cannot be null because it models the existence of an entity in the real world a FLIGHT# in DEPT-AIRPORT must exist in FLIGHT-SCHEDULE because it doesn’t make sense for a non-existing FLIGHT- SCHEDULE entity to have a DEPT-AIRPORT DEPT-AIRPORT FLIGHT#AIRPORT-CODE 101atl 912cph 545lax 242bos FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231

28 28 Database Concepts © Leo Mark Triggers and Stored Procedures Triggers can be defined to enforce constraints on a database, e.g., l DEFINE TRIGGER DELETE-FLIGHT-SCHEDULE ON DELETE FROM FLIGHT-SCHEDULE WHERE FLIGHT#=‘X’ ACTION DELETE FROM DEPT-AIRPORT WHERE FLIGHT#=‘X’; DEPT-AIRPORT FLIGHT#AIRPORT-CODE 101atl 912cph 545lax 242bos FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo156 545americanwe110 912scandinavianfr450 242usairmo231

29 29 Database Concepts © Leo Mark Null Values 123-45-6789 234-56-7890 345-67-8901 CUSTOMER Lisa Smith George Foreman unknown Lisa Jones inapplicable Mary Blake inapplicable drafted inapplicable CUSTOMER#NAMEMAIDEN NAMEDRAFT STATUS l Null-value unknown reflects that the attribute does apply, but the value is currently unknown. That’s ok! l Null-value inapplicable indicates that the attribute does not apply. That’s bad! l Null-value inapplicable results from the direct use of “catch all forms” in database design. l “Catch all forms” are ok in reality, but detrimental in database design.

30 30 Database Concepts © Leo Mark Normalization FLIGHT#AIRLINEPRICE FLIGHT-SCHEDULE 101delta156 545american110 912scandinavian450 FLIGHT#AIRLINEWEEKDAYPRICE FLIGHT-SCHEDULE 101deltamo 545americanmo110 912scandinavianfr450 156 101deltafr156 545americanwe110 545americanfr110 FLIGHT#AIRLINEWEEKDAYSPRICE FLIGHT-SCHEDULE 101deltamo,fr156 545americanmo,we,fr110 912scandinavianfr450 FLIGHT#WEEKDAY FLIGHT-WEEKDAY 101mo 545mo 912fr 101fr 545we 545fr unnormalized redundant just right!

31 31 Database Concepts © Leo Mark Surrogates - Things and Names name custom# addr customer name custom# addr customer reality customer custom# name addr customer name-based representation surrogate-based representation l name-based: a thing is what we know about it l surrogate-based: “Das ding an sich” [Kant] l surrogates are system-generated, unique, internal identifiers

32 32 Database Concepts © Leo Mark DATA MODEL OVERVIEW l ER-Model l Hierarchical Model l Network Model l Inverted Model - ADABAS l Relational Model l Object-Oriented Model(s)

33 33 Database Concepts © Leo Mark ER-Model l Data Structures l Integrity Constraints l Operations l The ER-Model is extremely successful as a database design model l Translation algorithms to many data models l Commercial database design tools, e.g., ERwin No generally accepted query language l No database system is based on the model

34 34 Database Concepts © Leo Mark ER-Model - Data Structures entity type relationship type attribute multivalued attribute derived attribute composite attribute subset relationship type

35 35 Database Concepts © Leo Mark ER-Model - Integrity Constraints A E E1E3E2 R E1 E2 1n R E1 E2 R E1 E2 (min,max) R E1 E2 R cardinality: 1:n for E1:E2 in R (min,max) participation of E2 in R total participation of E2 in R weak entity type E2; identifying relationship type R key attribute d x p disjoint exclusion partition

36 36 Database Concepts © Leo Mark ER Model - Example dept airport date international flight domestic flight p instance flight schedule instance of arriv airport 1 n n 1 street dept time airport code arriv time city zip airport addr airport name reserva- tion nn 1 n customer flight# customer name customer# seat# weekdays visa required

37 37 Database Concepts © Leo Mark ER-Model - Operations l Several navigational query languages have been proposed l A closed query language as powerful as relational languages has not been developed l None of the proposed query languages has been generally accepted

38 38 Database Concepts © Leo Mark Hierarchical Model l Data Structures l Integrity Constraints l Operations l Commercial systems include IBM’s IMS, MRI’s System-2000 (now sold by SAS), and CDC’s MARS IV

39 39 Database Concepts © Leo Mark Hierarchical Model - Data Structures l record types: flight-schedule, flight-instance, etc. l field types: flight#, date, customer#, etc. l parent-child relationship types (1:n only!!): (flight-sched,flight-inst), (flight-inst,customer) l one record type is the root, all other record types is a child of one parent record type only l substantial duplication of customer instances l asymmetrical model of n:m relationship types flight-sched customer customer#customer name flight-inst date dept-airp airport-code arriv-airp airport-code flight#

40 40 Database Concepts © Leo Mark Hierarchical Model - Data Structures - virtual records l duplication of customer instances avoided l still asymmetrical model of n:m relationship types customer customer namecustomer# flight-inst flight-sched date dept-airp airport-code arriv-airp airport-code flight# customer- pointer P

41 41 Database Concepts © Leo Mark Hierarchical Model - Operations flight-sched customer customer#customer name flight-inst date dept-airp airport-code arriv-airp airport-code flight# GET UNIQUE flight-sched (flight#=‘912’)[search flight-sched; get first such flight-sched] GET UNIQUE flight-sched[for each flight-sched flight-inst (date=‘102298’) for each flight-inst with date=102298 customer (name=‘Jensen’) for each customer with name=Jensen, get the first one] GET UNIQUE flight-sched[for each flight-sched flight-inst (date=‘102298’) for each flight-inst with date=102298, get the first GET NEXT flight-inst get the next flight-inst, whatever the date] GET UNIQUE flight-sched [for each flight-sched flight-inst (date=102298’) for each flight-inst get the first with date=102298 customer (name=‘Jensen’) for each customer with name=Jensen, get the first one GET NEXT WITHIN PARENT customer get the next customer, whatever his name, but only on that flight-inst]

42 42 Database Concepts © Leo Mark Network Model l Data Structures l Integrity Constraints l Operations l Based on the CODASYL-DBTG 1971 report l Commercial systems include, CA-IDMS and DMS-1100

43 43 Database Concepts © Leo Mark Network Model - Data Structures reservation flight#datecustomer# flight-schedule flight# customer customer#customer name FR CR R1R2R3R4R5R6 F1F2 C1C4 Type diagram Bachman Diagram Occurrence diagram The Spaghetti Model l owner record types: flight-schedule, customer l member record type: reservations l DBTG-set types: FR, CR l n-m relationships cannot be modeled directly l recursive relationships cannot be modeled directly

44 44 Database Concepts © Leo Mark Network Model - Integrity Constraints l set retention options: –fixed –mandatory –optional l set insertion options: –automatic –manual reservation flight#datecustomer# flight-schedule flight# customer customer#customer name FR CR flight-schedule flight# l keys l checks reservation flight#datecustomer#price check is price>100 FR and CR are fixed and automatic

45 45 Database Concepts © Leo Mark Network Model - Operations l The operations in the Network Model are generic, navigational, and procedural (1) find flight-schedule where flight#=F2 (2) find first reservation of FR (3) find next reservation of FR (4) find owner of CR R1R2R3R4R5R6 F1F2 C1C4 FR CR (F2) (R4) (R5) (C4) query:currency indicators:

46 46 Database Concepts © Leo Mark Network Model - Operations l navigation is cumbersome; tuple-at-a-time l many different currency indicators l multiple copies of currency indicators may be needed if the same path is traveled twice l external schemata are only sub-schemata

47 47 Database Concepts © Leo Mark Inverted Model - ADABAS l Data Structures l Integrity Constraints l Operations

48 48 Database Concepts © Leo Mark Relational Model l Data Structures l Integrity Constraints l Operations l Commercial systems include: ORACLE, DB2, SYBASE, INFORMIX, INGRES, SQL S erver l Dominates the database market on all platforms

49 49 Database Concepts © Leo Mark Relational Model - Data Structures l domains l attributes l relations flight-schedule flight#: integer airline: char(20) weekday: char(2) price: dec(6,2) relation name attribute names domain names

50 50 Database Concepts © Leo Mark Relational Model - Integrity Constraints l Keys l Primary Keys l Entity Integrity l Referential Integrity reservation flight#datecustomer# flight-schedule flight# p customer customer#customer name p

51 51 Database Concepts © Leo Mark Relational Model - Operations l Powerful set-oriented query languages l Relational Algebra: procedural; describes how to compute a query; operators like JOIN, SELECT, PROJECT l Relational Calculus: declarative; describes the desired result, e.g. SQL, QBE l insert, delete, and update capabilities

52 52 Database Concepts © Leo Mark Relational Model - Operations l tuple calculus example (SQL) select flight#, date from reservation R, customer C where R.customer#= C.customer# and customer-name=‘ LEO ’; l algebra example (ISBL) ((reservation join customer) where customer- name=‘ LEO ’) [flight#, date]; l domain calculus example (QBE) customer customer#customer- name _c LEO date reservation flight#customer# _c.P

53 53 Database Concepts © Leo Mark Object-Oriented Model(s) l based on the object-oriented paradigm, e.g., Simula, Smalltalk, C++, Java l area is in a state of flux l object-oriented model has object-oriented repository model; adds persistence and database capabilities; (see ODMG-93, ODL, OQL) l object-oriented commercial systems include GemStone, Ontos, Orion-2, Statice, Versant, O 2 l object-relational model has relational repository model; adds object-oriented features; (see SQL3) l object-relational commercial systems include Starburst, POSTGRES

54 54 Database Concepts © Leo Mark Object-Oriented Paradigm l object class l object attributes, primitive types, values l object interface, methods; body, implementations l messages; invoke methods; give method name and parameters; return a value l encapsulation l visible and hidden attributes and methods l object instance; object constructor & destructor l object identifier, immutable l complex objects; multimedia objects; extensible type system l subclasses; inheritance; multiple inheritance l operator overloading l references represent relationships l transient & persistent objects

55 55 Database Concepts © Leo Mark class flight-schedule { type tuple (flight#: integer, weekdays: set ( weekday: enumeration {mo,tu,we,th,fr,sa,su}) dept-airport: airport, arriv-airport: airport) method reschedule(new-dept: airport, new-arriv: airport)} class international-flight inherit flight-schedule { type tuple (visa-required:string) method change-visa-requirement(v: string): boolean} /* the reschedule method is inherited by international-flight; */ /* when reschedule is invoked in international-flight it may */ /* also invoke change-visa-requirement */ Object-Oriented Model - Structures O 2 -like syntax

56 56 Database Concepts © Leo Mark class flight-instance { type tuple (flight-date: tuple ( year: integer, month: integer, day: integer); instance-of: flight-schedule, passengers: set (customer) inv customer::reservations) method add-passenger(new-passenger:customer):boolean, /*adds to passengers; invokes customer.make-reservation */ remove-passenger(passenger: customer):boolean} /*removes from passengers; invokes customer.cancel-reservation*/ class customer { type tuple (customer#: integer, customer-name: tuple ( fname: string, lname: string) reservations: set (flight-instance) inv flight-instance::passengers) method make-reservation(new-reservation: flight-instance): boolean, cancel-reservation(reservation: flight-instance): boolean}

57 57 Database Concepts © Leo Mark Object-Oriented Model - Updates class customer { type tuple (customer#: integer, customer-name: tuple ( fname: string, lname: string) reservations: set (flight-instance) inv flight-instance::passengers) main () { transaction::begin(); all-customers: set( customer); /*makes persistent root to hold all customers */ customer c= new customer; /*creates new customer object */ c= tuple (customer#: “111223333”, customer-name: tuple( fname: “Leo”, lname: “Mark”)); all-customers += set( c); /*c becomes persistent by attaching to root */ transaction::commit();} O 2 -like syntax

58 58 Database Concepts © Leo Mark Object-Oriented Model - Queries “Find the customer#’s of all customers with first name Leo” select tuple (c#: c.customer#) from c in customer where c.customer-name.fname = “Leo”; “Find passenger lists, each with a flight# and a list of customer names, for flights out of Atlanta on October 22, 1998” select tuple(flight#: f.instance-of.flight#, passengers: select( tuple( c.customer#, c.customer-name.lname))) from f in flight-instance, c in f.passengers where f.flight-date=(1998, 10, 22) and f.instance-of.dept-airport.airport-code=“Atlanta”; O 2 -like syntax

59 59 Database Concepts © Leo Mark DATABASE ARCHITECTURE l ANSI/SPARC 3-Level DB Architecture l Metadata - What is it? Why is it important? l ISO Information Resource Dictionary System (ISO-IRDS)

60 60 Database Concepts © Leo Mark ANSI/SPARC 3-Level DB Architecture - separating concerns database system schema data database database system DDL DML l a database is divided into schema and data l the schema describes the intension (types) l the data describes the extension (data) l Why? Effective! Efficient!

61 61 Database Concepts © Leo Mark ANSI/SPARC 3-Level DB Architecture - separating concerns schema data schema conceptual schema internal schema data internal schema data external schema

62 62 Database Concepts © Leo Mark ANSI/SPARC 3-Level DB Architecture external schema 1 external schema 1 external schema 2 external schema 2 external schema 3 external schema 3 conceptual schema conceptual schema internal schema internal schema database external schema: use of data conceptual schema: meaning of data internal schema: storage of data

63 63 Database Concepts © Leo Mark Conceptual Schema l Describes all conceptually relevant, general, time-invariant structural aspects of the universe of discourse l Excludes aspects of data representation and physical organization, and access NAME ADDR SEX AGE CUSTOMER l An object-oriented conceptual schema would also describe all process aspects

64 64 Database Concepts © Leo Mark External Schema l Describes parts of the information in the conceptual schema in a form convenient to a particular user group’s view l Is derived from the conceptual schema NAME ADDR SEX AGE CUSTOMER NAME ADDR MALE-TEEN-CUSTOMER TEEN-CUSTOMER(X, Y) = CUSTOMER(X, Y, S, A) WHERE SEX=M AND 12 { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/8/2311281/slides/slide_64.jpg", "name": "64 Database Concepts © Leo Mark External Schema l Describes parts of the information in the conceptual schema in a form convenient to a particular user group’s view l Is derived from the conceptual schema NAME ADDR SEX AGE CUSTOMER NAME ADDR MALE-TEEN-CUSTOMER TEEN-CUSTOMER(X, Y) = CUSTOMER(X, Y, S, A) WHERE SEX=M AND 12

65 65 Database Concepts © Leo Mark Internal Schema l Describes how the information described in the conceptual schema is physically represented to provide the overall best performance NAME ADDR SEX AGE CUSTOMER NAME ADDR SEX AGE CUSTOMER B + -tree on AGE NAME PTR index on NAME

66 66 Database Concepts © Leo Mark Physical Data Independence external schema 1 external schema 1 external schema 2 external schema 2 external schema 3 external schema 3 conceptual schema conceptual schema internal schema internal schema database Physical data independence is a measure of how much the internal schema can change without affecting the application programs

67 67 Database Concepts © Leo Mark Logical Data Independence external schema 1 external schema 1 external schema 2 external schema 2 external schema 3 external schema 3 conceptual schema conceptual schema internal schema internal schema database Logical data independence is a measure of how much the conceptual schema can change without affecting the application programs

68 68 Database Concepts © Leo Mark Schema Compiler compiler metadata schemata The schema compiler compiles schemata and stores them in the metadatabase Catalog Data Dictionary Metadatabase

69 69 Database Concepts © Leo Mark Query Transformer metadata query transformer DML query transformer data Uses metadata to transform a query at the external schema level to a query at the storage level

70 70 Database Concepts © Leo Mark ANSI/SPARC DBMS Framework enterprise administrator database administrator application system administrator conceptual schema processor internal schema processor external schema processor storage internal transformer internal conceptual transformer conceptual external transformer metadata user data 1 33 1324 145 343638 21303112 schema compiler query transformer

71 71 Database Concepts © Leo Mark Metadata - What is it? l System metadata: –Where data came from –How data were changed –How data are stored –How data are mapped –Who owns data –Who can access data –Data usage history –Data usage statistics l Business metadata: –What data are available –Where data are located –What the data mean –How to access the data –Predefined reports –Predefined queries –How current the data are Metadata - Why is it important? l System metadata are critical in a DBMS l Business metadata are critical in a data warehouse

72 72 Database Concepts © Leo Mark ISO-IRDS - Why? l Are metadata different from data? l Are metadata and data stored separately? l Are metadata and data described by different models? l Is there a schema for metadata? A metaschema? l Are metadata and data changed through different interfaces? l Can a schema be changed on-line? l How does a schema change affect data?

73 73 Database Concepts © Leo Mark ISO-IRDS Architecture DL metaschema data dictionary schema data dictionary data raw formatted application data data dictionary data; schema for application data; data about application data data dictionary schema; contains copy of metaschema; schema for format definitions; schema for data about application data metaschema; describes all schemata that can be defined in the data model data

74 74 Database Concepts © Leo Mark ISO-IRDS - example metaschema data dictionary schema data dictionary data relations access-rights relations supplier rel-name att-name dom-name (u1, supplier, insert) (u2, supplier, delete) userrelationoperation s#snamelocation (s1, smith, london) (s2, jones, boston)

75 75 Database Concepts © Leo Mark DATABASE MANAGEMENT SYSTEM ARCHITECTURE l Teleprocessing Database l File-Sharing Database l Client-Server Database - Basic l Client-Server Database - w/Caching l Distributed Database l Federated Database l Multi-Database l Parallel Databases

76 76 Database Concepts © Leo Mark Teleprocessing Database OS TP AP 1 AP 2 AP 3 DBMS OS DB DB dumb terminal communication lines mainframe database

77 77 Database Concepts © Leo Mark Teleprocessing Database - characteristics l Dumb terminals l APs, DBMS, and DB reside on central computer l Communication lines are typically phone lines l Screen formatting transmitted via communication lines l User interface character oriented and primitive l Dumb terminals are gradually being replaced by micros

78 78 Database Concepts © Leo Mark File-Sharing Database OS NET AP 1 AP 2 AP 3 DBMS OS DB DB LAN database OS NET DBMS file server micro micros

79 79 Database Concepts © Leo Mark File-Sharing Database - characteristics l APs and DBMS on client micros l File-Server on server micro l Clients and file-server communicate via LAN l Substantial traffic on LAN because large files (and indices) must be sent to DBMS on clients for processing l Substantial lock contention for extended periods of time for the same reason l Good for extensive query processing on downloaded snapshot data l Bad for high-volume transaction processing

80 80 Database Concepts © Leo Mark Client-Server Database - Basic AP 1 AP 2 AP 3 DBMS OS DB DB micro(s) or mainframe database OS NET LAN micros

81 81 Database Concepts © Leo Mark Client-Server Database - Basic - characteristics l APs on client micros l Database-server on micro or mainframe l Multiple servers possible; no data replication l Clients and database-server communicate via LAN l Considerably less traffic on LAN than with file-server l Considerably less lock contention than with file-server

82 82 Database Concepts © Leo Mark Client-Server Database - w/Caching AP 1 AP 2 AP 3 DBMS OS DB DB micro(s) or mainframe database OS NET LAN micros DBMS DB

83 83 Database Concepts © Leo Mark Client-Server Database - w/Caching - characteristics l DBMS on server and clients l Database-server is primary update site l Downloaded queries are cached on clients l Change logs are downloaded on demand l Cached queries are updated incrementally l Less traffic on LAN than with basic client- server database because only initial query result is downloaded followed by change logs l Less lock contention than with basic client- server database for same reason

84 84 Database Concepts © Leo Mark Distributed Database AP 1 AP 2 AP 3 OS NET&DB micros(s) or mainframes DDBMS DB network conceptual internal external

85 85 Database Concepts © Leo Mark Distributed Database - characteristics l APs and DDBMS on multiple micros or mainframes l One distributed database l Communication via LAN or WAN l Horizontal and/or vertical data fragmentation l Replicated or non-replicated fragment allocation l Fragmentation and replication transparency l Data replication improves query processing l Data replication increases lock contention and slows down update transactions

86 86 Database Concepts © Leo Mark Distributed Database - Alternatives D C B A B A D C B A D C B AC D C partitioned non-replicated non-partitioned replicated partitioned replicated +- increasing parallelism, independence, flexibility, availability increasing cost, complexity, difficulty of control, security risk

87 87 Database Concepts © Leo Mark Federated Database AP 1 AP 2 AP 3 OS NET&DB micros(s) or mainframes DDBMS DB network internal 1 conceptual 1 conceptual 2 internal 2 conceptual 3 internal 3 export schema 1 export schema 3 export schema 2 federation schema

88 88 Database Concepts © Leo Mark Federated Database - characteristics l Each federate has a set of APs, a DDBMS, and a DB l Part of a federate’s database is exported, i.e., accessible to the federation l The union of the exported databases constitutes the federated database l Federates will respond to query and update requests from other federates l Federates have more autonomy than with a traditional distributed database

89 89 Database Concepts © Leo Mark Multi-Database AP 1 AP 2 AP 3 OS NET&DB micros(s) or mainframes MULTI-DBMS DB network, e.g WWW internal 1 conceptual 1 conceptual 2 internal 2 conceptual 3 internal 3

90 90 Database Concepts © Leo Mark Multi-Database - characteristics l A multi-database is a distributed database without a shared schema l A multi-DBMS provides a language for accessing multiple databases from its APs l A multi-DBMS accesses other databases via a network, like the www l Participants in a multi-database may respond to query and update requests from other participants l Participants in a multi-database have the highest possible level of autonomy

91 91 Database Concepts © Leo Mark Parallel Databases l A database in which a single query may be executed by multiple processors working together in parallel l There are three types of systems: –Shared memory –Shared disk –Shared nothing

92 92 Database Concepts © Leo Mark Parallel Databases - Shared Memory l processors share memory via bus l extremely efficient processor communication via memory writes l bus becomes the bottleneck l not scalable beyond 32 or 64 processors P processor M memory disk P M P P P

93 93 Database Concepts © Leo Mark Parallel Databases - Shared Disk l processors share disk via interconnection network l memory bus not a bottleneck l fault tolerance wrt. processor or memory failure l scales better than shared memory l interconnection network to disk subsystem is a bottleneck l used in ORACLE Rdb P P P P M M M M

94 94 Database Concepts © Leo Mark Parallel Databases - Shared Nothing l scales better than shared memory and shared disk l main drawbacks: –higher processor communication cost –higher cost of non-local disk access l used in the Teradata database machine P M P M P M P M

95 95 Database Concepts © Leo Mark RAID - redundant array of inexpensive disks l disk striping improves performance via parallelism (assume 4 disks worth of data is stored) l disk mirroring improves reliability via redundancy (assume 4 disks worth of data is stored) l mirroring: via copy of data (c); via bit parity (p) cccc p

96 96 Database Concepts © Leo Mark DATABASE CAPABILITIES l Data Storage l Queries l Optimization l Indexing l Concurrency Control l Recovery l Security

97 97 Database Concepts © Leo Mark Data Storage l Disk management l File management l Buffer management l Garbage collection l Compression

98 98 Database Concepts © Leo Mark Queries l Selection –Point –Range –Conjunction –Disjunction l Join –Natural join –Equi join –Theta join –Outer join l Projection l Set operations –Cartesian Product –Union –Intersection –Set Difference l Other –Duplicate elimination –Sorting –Built-in functions: count, sum, avg, min, max l Recursive (not in SQL) SQL queries are composed from the following:

99 99 Database Concepts © Leo Mark Query Optimization select flight#, date from reserv R, cust C where R.cust#=C.cust# and cust-name=‘LEO ’; date reserv flight#cust# customer cust#cust-name reservcust cust-name=Leo flight#, date reservcust cust-name=Leo flight#, date cust# 10,000 reserv blocks 3,000 cust blocks 30 “Leo” blocks cost: 10,000x3,000 cost: 3,000 cost: 10,000x30

100 100 Database Concepts © Leo Mark Query Optimization l Database statistics l Query statistics l Index information l Algebraic manipulation l Join strategies –Nested loops –Sort-merge –Index-based –Hash-based

101 101 Database Concepts © Leo Mark Indexing Why Bother? l Disk access time: 0.01-0.03 sec l Memory access time: 0.000001-0.000003 sec l Databases are I/O bound l Rate of improvement of (memory access time)/(disk access time) >>1 l Things won’t get better anytime soon! Indexing helps reduce I/O !

102 102 Database Concepts © Leo Mark Indexing (cont.) l Clustering vs. non-clustering l Primary and secondary indices l I/O cost for lookup: –Heap:N/2 –Sorted file:log 2 (N) –Single-level index:log 2 (n)+1 –Multi-level index; B + -tree:log fanout (n)+1 –Hashing:2-3 l View caching; incremental computation

103 103 Database Concepts © Leo Mark Concurrency Control date reserv flight#customer# flight-inst flight#date#avail- seats T1: read(flight-inst(flight#,date) seats:=#avail-seats if seats>0 then { seats:=seats-1 write(reserv(flight#,date,customer1)) write(flight-inst(flight#,date,seats))} T2: read(flight-inst(flight#,date) seats:=#avail-seats if seats>0 then { seats:=seats-1 write(reserv(flight#,date,customer2)) write(flight-inst(flight#,date,seats))} overbooking!

104 104 Database Concepts © Leo Mark Concurrency Control (cont.) ACID Transactions: l An ACID transaction is a sequence of database operations that has the following properties: l Atomicity –Either all operations are carries out, or none is –This property is the responsibility of the concurrency control and the recovery sub-systems l Consistency –A transaction maps a correct database state to another correct state –This requires that the transaction is correct, which is the responsibility of the application programmer

105 105 Database Concepts © Leo Mark Concurrency Control (cont.) l Isolation –Although multiple transactions execute concurrently, i.e. interleaved, not parallel, they appear to execute sequentially –This is the responsibility of the concurrency control sub- system l Durability –The effect of a completed transaction is permanent –This is the responsibility of the recovery manager

106 106 Database Concepts © Leo Mark Concurrency Control (cont.) l Serializability is a good definition of correctness l A variety of concurrency control protocols exist –Two-phase (2PL) locking l deadlock and livelock possible l deadlock prevention: wait-die, wound-wait l deadlock detection: rollback a transaction –Optimistic protocol: proceed optimistically; back up and repair if needed –Pessimistic protocol: do not proceed until knowing that no back up is needed

107 107 Database Concepts © Leo Mark Recovery date reserv flight#customer# flight-inst flight#date#avail- seats change-reservation(DL212,102298,DL212,102398,C) read(flight-inst(DL212,102298) #avail-seats:=#avail-seats+1 update(flight-inst(DL212,102298,#avail-seats) read(flight-inst(DL212,102398) #avail-seats:=#avail-seats-1 update(flight-inst(DL212,102398,#avail-seats) update(reserv(DL212,102298,C,DL212,102398,C) 10050 10150 10149 102298102398

108 108 Database Concepts © Leo Mark Recovery (cont.) Storage types: l Volatile: main memory l Nonvolatile: disk Errors: l Logical error: transaction fails; e.g. bad input, overflow l System error: transaction fails; e.g. deadlock l System crash: power failure; main memory lost, disk survives l Disk failure: head crash, sabotage, fire; disk lost What to do?

109 109 Database Concepts © Leo Mark Recovery (cont.) l Deferred update ( NO-UNDO/REDO ): –don’t change database until ready to commit –write-ahead to log to disk –change the database l Immediate update ( UNDO/NO-REDO ) : –write-ahead to log on disk –update database anytime –commit not allowed until database is completely updated l Immediate update ( UNDO/REDO ): –write-ahead to log on disk –update database anytime –commit allowed before database is completely updated l Shadow paging ( NO-UNDO/NO-REDO ): –write-ahead to log in disk –keep shadow page; update copy only; swap at commit

110 110 Database Concepts © Leo Mark Security DAC: Discretionary Access Control l is used to grant/revoke privileges to users, including access to files, records, fields (read, write, update mode) MAC: Mandatory Access Control l is used to enforce multilevel security by classifying data and users into security levels and allowing users access to data at their own or lower levels only

111 111 Database Concepts © Leo Mark PEOPLE THAT WORK WITH DATABASES l System Analysts l Database Designers l Application Developers l Database Administrators l End Users

112 112 Database Concepts © Leo Mark System Analysts l communicate with each prospective database user group in order to understand its –information needs –processing needs l develop a specification of each user group’s information and processing needs l develop a specification integrating the information and processing needs of the user groups l document the specification

113 113 Database Concepts © Leo Mark Database Designers l choose appropriate structures to represent the information specified by the system analysts l choose appropriate structures to store the information in a normalized manner in order to guarantee integrity and consistency of data l choose appropriate structures to guarantee an efficient system l document the database design

114 114 Database Concepts © Leo Mark Application Developers l implement the database design l implement the application programs to meet the program specifications l test and debug the database implementation and the application programs l document the database implementation and the application programs

115 115 Database Concepts © Leo Mark Database Administrators l Manage the database structure –participate in database and application development –assist in requirement analysis –participate in database design and creation –develop procedures for integrity and quality of data –facilitate changes to database structure –seek communitywide solutions –assess impact on all users –provide configuration control –be prepared for problems after changes are made –maintain documentation

116 116 Database Concepts © Leo Mark Database Administrators (cont.) l Manage data activity –establish database standards consistent with data administration standards –establish and maintain data dictionary –establish data proponencies –work with data proponents to develop data access and modification rights –develop, document, and train staff on backup and recovery procedures –publish and maintain data activity standards documentation

117 117 Database Concepts © Leo Mark Database Administrators (cont.) l Manage the database management system –generate database application performance reports –investigate user performance complaints –assess need for changes in database structure or application design –modify database structure –evaluate and implement new DBMS features –tune the database l Establish the database data dictionary –data names, formats, relationships –cross-references between data and application programs –(see metadata slide)

118 118 Database Concepts © Leo Mark End Users l Parametric end users constantly query and update the database. They use canned transactions to support standard queries and updates. l Casual end users occasional access the database, but may need different information each time. They use sophisticated query languages and browsers. l Sophisticated end users have complex requirement and need different information each time. They are thoroughly familiar with the capabilities of the DBMS.

119 119 Database Concepts © Leo Mark THE DATABASE MARKET l Prerelational vs. Relational l Database Vendors l Relational Database Products l Relational Databases for PCs l Object-Oriented Database Capabilities

120 120 Database Concepts © Leo Mark Prerelational vs. Relational l Prerelational market revenue shrinking about 9%/year. Currently 1.8 billion/year l Relational market revenue growing about 30%/year. Currently 11.5 billion/year l Object-Oriented market revenue about 150 million/year billion $

121 121 Database Concepts © Leo Mark Database Vendors Other ($2,272M) Oracle ($1,755M) IBM (IMS+DB2) ($1,460M) Sybase ($664M) Informix (+Illustra) ($492M) CA-IDMS (+Ingress) ($447M) NEC ($211M) Fujitsu ($186M) Hitachi ($117M) Software AG (ADABAS) ($136M) Microsoft (SQL Server) ($107M) Source: IDC, 1995 Other Oracle IBM Sybase InformixCA Total: $7,847M

122 122 Database Concepts © Leo Mark We compare the following products: l ORACLE 7 Version 7.3 l Sybase SQL Server 11 l Informix OnLine 7.2 l Microsoft SQL Server 6.5 l IBM DB2 2.1.1 l CA-OpenIngres 1.2 Relational Database Products

123 123 Database Concepts © Leo Mark Relational Database Products

124 124 Database Concepts © Leo Mark Relational Database Products

125 125 Database Concepts © Leo Mark Relational Database Products

126 126 Database Concepts © Leo Mark Relational Database Products

127 127 Database Concepts © Leo Mark Relational Database Products

128 128 Database Concepts © Leo Mark Relational Database Products

129 129 Database Concepts © Leo Mark Relational Database Products

130 130 Database Concepts © Leo Mark Relational Database Products

131 131 Database Concepts © Leo Mark Relational Databases for PCs Relational databases for PCs include: l Microsoft FoxPro for Windows l Microsoft FoxPro for DOS l Borland’s Paradox for Windows l Borland’s dBASE IV l Paradox for DOS l R:BASE l Microsoft Access

132 132 Database Concepts © Leo Mark Object-Oriented Database Capabilities

133 133 Database Concepts © Leo Mark Object-Oriented Database Capabilities

134 134 Database Concepts © Leo Mark EMERGING DB TECHNOLOGIES l WEB databases l Multimedia Databases l Mobile Databases l Data Warehousing and Mining l Geographic Information Systems l Genome Data Management l Temporal Databases l Spatial Databases

135 135 Database Concepts © Leo Mark WHAT YOU WILL BE ABLE TO LEARN MORE ABOUT The Database Certificate Program l Database Concepts l Data Modeling l Relational Database Design l Performance Issues & Tuning in Relational Databases l Data Warehousing and Mining


Download ppt "1 Database Concepts © Leo Mark. 2 Database Concepts © Leo Mark DATABASE CONCEPTS Leo Mark College of Computing Georgia Tech (January 1999)"

Similar presentations


Ads by Google