Teaching slides Chapter 8

Chapter 8: Database design and implementation Contents Software engineering methodology considerations Database technology Structured query language Database design (Entity relationship Diagram) Table joins and relationships Master and transaction tables Data normalization

Chapter 8: Database design and implementation Software engineering methodology considerations On Waterfall projects, database design and implementation is carried out for the entire software product. When software designs are created for the software product; database designing is also carried out as part of those software designs. But on Agile projects, it is not possible to create entire database design at once. It is because the customer provides only a bunch of user stories at a time (at the beginning of a Sprint) and database design has to be created only for those user stories. This scenario creates a challenge. After developing a software product over a long period of time, the existing database design may need to be changed for further development of the software product. This may require some additional work like porting existing data in the database before doing any changes in the database design. One more factor which need to be considered is that database design and implementation should be done before designing and implementing business logic. This is because business logic implementation uses databases if database programming is involved. To take care of this challenge, we can design and implement database design well in advance of designing and implementing business logic. So for example, we can design and implement database for Sprint 2 before Sprint 2 starts.

Chapter 8: Database design and implementation Software engineering methodology considerations

Chapter 8: Database design and implementation Database technology Databases are a special kind of software products which are designed to keep permanent data in such a manner that even huge amounts of stored data can be managed without any problems. Currently 2 types of databases are mostly used with software products: relational databases and noSQL databases. Relational databases store data across joined tables. The relationship (or join) among these tables ensure that even though data is stored separately across many tables; any piece of data can be easily searched and retrieved. Most of popularly used databases including Oracle, MySQL, SQL server etc. are all relational databases. noSQL databases are essentially flat file system. They store data in files. To identify pieces of data; various types of tags are used. The drawback of these databases is that when large amount of data is saved on these databases then performance becomes a real issue. Saving and retrieving data then becomes extremely difficult.

Chapter 8: Database design and implementation Structured query language Structure Query language (SQL) is used to perform all kinds of tasks related to relational databases. Relational databases no not allow any other mechanism to create database entities and data manipulation apart from SQL. When we need to create database entities like tables, indexes, schemas, primary keys, foreign keys etc. then we use SQL. This part of SQL is known as Data Definition language (DDL). DDL essentially creates structures to store data safely. When we need to manipulate data including data creation, data update or data deletion then we again use SQL. This part of SQL is known as Data Manipulation Language (DML).

Chapter 8: Database design and implementation Structured query language

Chapter 8: Database design and implementation Structured query language As we know DDL part of SQL is used to create structures like tables, schemas, indexes etc. But DDL is used to also create primary and secondary keys. While primary keys implement entity integrity rules; foreign keys implement referential integrity rules. Together they ensure data residing in different tables are joined with each other. The entity integrity rule states: There should be no NULL values in the column containing primary key Each value in the column with primary key must be unique. A NULL value in a database signifies that there is no value in the column where it appears. A column with primary key should never have a NULL value. This is because a primary key with a NULL value in a record will not be able to identify that record. Thus a primary key column should never have a NULL value. The primary key column must contain unique values. If a primary key column contains a value which is same for more than one record then again identifying a record uniquely will be difficult. So a primary key column must contain unique values.

Chapter 8: Database design and implementation Structured query language When 2 tables have a join then one table acts as the master table and the other as transaction table. The master table contains a key column which is referenced by the key column of the transaction table. Both the key columns of the 2 tables have primary keys. The key column of the transaction table contains one more key. This key references the key column of the master table. This reference key is known as foreign key. This foreign key is governed by referential integrity rules. The referential integrity rule is as follows: A valid foreign key value must always reference an existing primary key value or contain a NULL. Since a foreign key always works in tandem with the primary key of the table to which it is joined; the referential integrity always works with the entity integrity rule. For example, if we are trying to insert a new record which has a NULL value in the foreign key column then this operation will fail as in this case there will be no reference to the primary key in the other table. Similarly if an insert operation is tried where there is some value in the foreign key column which is not present in the primary key column then again there will be a referential integrity violation and so this operation will be aborted.

Chapter 8: Database design and implementation We need to create database designs as part of software design. Entity Relationship Diagrams (ER Diagrams) are used to create database designs. From user stories or epic or high level design, entities need to be found for which permanent data need to be either created or consumed. Data creation and consumption will determine the relationship among entities. What kind of permanent data need to be created for an entity will determine the attributes of an entity. For example, if have

Chapter 8: Database design and implementation The following is the high level considerations: Web page for creation of profile for administrator and sellers Web page for login for administrators and sellers Web page for creation of property types Web page for creation of budget ranges Web page for creation of locations Web page for creating property listing Web page for selecting criteria for searching the listing Web page to view listings Web Page to view details of a listing So here is the list of entities and their attributes: Administrator – profile attributes (could be name, email address, phone, username etc.). Suppose the customer confirms that these are the exact attributes. Seller - profile attributes (could be name, email address, phone, username etc.). Suppose the customer confirms that these are the exact attributes. Property – property attributes (property type, budget range, location). Suppose the customer confirms that these are the exact attributes. We can also observe that the 3 entities of administrator, seller and property have no relation with each other. So here are the ER diagrams for these entities.

Chapter 8: Database design and implementation

Chapter 8: Database design and implementation

Chapter 8: Database design and implementation

Chapter 8: Database design and implementation Once ER diagram are created then it is easy to create tables using SQL statements: Create table property (Id number, location String, budget_range String, property_type String); Create table administrator (Id number, name String, username String, address String, phone String, email String); Create table seller (Id number, name String, username String, address String, phone String, email String); After putting some sample data, the tables will look like as shown on next slide.

Chapter 8: Database design and implementation

Chapter 8: Database design and implementation Table joins and relationships In the case study related to online portal for property listing, we have seen that the tables have no join as the high level design had no such requirements. Suppose there was one more software requirement: “the seller’s contact details should be shown along with property details when a user clicks on the detail link for a property”. In this case, we will have to have a relationship between the seller and the listed property which the seller owns. There can be many types of relationships among tables. The most common type of relationship between 2 tables is one-to-many relationship. When one table with a primary key is joined with another table having foreign key then one-to-many relationship is the natural relationship. But one-to-one, many-to-many and self joins are also some other types of relationships among tables.

Chapter 8: Database design and implementation Table joins and relationships The changed software requirement for the online portal for property listing will necessitate changing the table definitions for the seller and property tables. Since we already have defined the 2 tables, we will need to modify them using the alter command. Alter table seller add constraint PK_seller primary key (Id); Alter table property add constraint PK_property primary key (Id), FK_property foreign key (Id) references seller(Id); Now the seller and property tables are joined with a foreign key on the Id column of the property table. This foreign key is referencing the Id column of the seller table. Now when records are created for these 2 tables then for each record existing in the seller table there can be one or more records in the property table. The ER diagram for the changed software requirement is depicted in next slide.

Chapter 8: Database design and implementation Table joins and relationships

Chapter 8: Database design and implementation Master and transaction tables Many software products are data intensive. Large amount of data is created for these software products. In most cases, it is the transaction data which gets created in large amounts. Master data on the other hand, do not get generated much. Once master data is created for a software product then data does not grow much in future. When we have a relationship between 2 tables using the primary and foreign keys; the table having the primary key becomes the master table and table having the foreign key becomes the transaction table. Since large amount of data will be generated in a transaction table in future, care should be taken to ensure that unnecessary data does not get created in such tables. We also know that a good database should not have duplicate and non atomic data. Thus a good database design is very important in such cases.

Chapter 8: Database design and implementation Data normalization Duplicate data in a database impacts performance and thus should be avoided. Non atomic data impacts searching a database. To ensure unique and atomic data is generated in a table; many techniques have evolved to analyze table structures. The best technique to do it is the 3 rules of data normalization.

Chapter 8: Database design and implementation Data normalization We start data normalization by implementing the first normal form. If a table is not in first normal form then data in the table will be non atomic and repeating. Suppose in our case study related to online portal for property listing a novice engineer has created the following table: Create table seller_property (Id number, location Varchar, budget_range Varchar, property_type Varchar. Name Varchar, username varchar, address Varchar, phone number, email Varchar); Sample data in this table is depicted in next slide. You can see that many pieces of data are repeating in the columns name, username, address, phone and email. To make this table conforming to the first normalization rule, we must remove columns which have repeating data.

Chapter 8: Database design and implementation Data normalization

Chapter 8: Database design and implementation Data normalization To normalize the data in the seller_property table, we can divide this table into 2 tables: Create table property (Id, location String, budget_range String, property_type String); Create table seller (Id number, name String, username String, address String, phone String, email String); Now both seller and property tables are in first normal form as data is in atomic form and it is not-repeating.

Chapter 8: Database design and implementation Data normalization Once a table is in first normal form then it can be refined further to make it in second normal form. A table is in second normal form if It is in first normal form and All columns depend on one column of the table to get identified. Currently our seller and property tables are in first normal form. But they are still not in second normal form as no columns are dependent on one identity column of the tables. Now to make our table conform to second normal form, we can create a primary key. Create table property (Id number not NULL primary key, location String, budget_range String, property_type String); Now our property table will be in second normal form. We can also create the seller table similarly as follows: Create table seller (Id number not NULL primary key, name String, username String, address String, phone String, email String); Now our seller table will also be in second normal form.

Chapter 8: Database design and implementation Data normalization A table is in third normal form if: The table is already in second normal form All other columns of the table are dependent only on the column which is used for referencing records in the master table. No column should be dependent on any other column apart from this identifying column. In cases where some of the other columns of a table are not dependent on the identifying column then the table is not in third normal form. Currently the property table is not joined with seller table. So no data in the property table is referencing any data from the seller table. So we need to join these 2 tables by creating a foreign key on the property table. We can use “alter table” statement to create a foreign key on the property table. Alter table property add constraint FK_property foreign key (Id) references seller(Id); Now all data created in the property table will be referencing data already present in the seller table. So data in property table will now be in third normal form.