1. COM336 Web Database Development Open Source Database Systems

2. Relational Databases Advantages over using flat files – RDB can provide faster access to data – RDB can be easily queried to extract sets of data that fits certain criteria – RDB have built-in mechanism for dealing with concurrent access (so you don’t have to worry about it) – RDB have built-in privilege system.

3. Relational Databases The RDB engine that we will use in the module is MySQL Before getting into the MySQL specifics, we need to cover: – RDB Concepts and Terminology – Web Database Design

4. RDB Concepts and Terminology

5. Relational DB Concepts Relational Databases (RDB) are the most common type of databases, they depend on on a sound theoretical basis in relational algebra. However, we don’t need to understand relational theory to use them. However, some basic concepts need to be understood

6. Tables Relational databases are made up of relations, more commonly called tables A table is exactly that: A table of data just like the one produced in a CustomerIDNameAddress 1Bill Jobs2011 Apple Way, SF, 10064 CA 2Steve Gates1 Microsoft Drive, SD, 98711 CA CUSTOMERS

7. Columns Columns in the table have unique names and contain different data (each column has an associated data type) – Columns are called FIELDS – On the previous example, the first column (customerID) is an integer and the other two Fields contain text strings

8. Rows & Values Each row in the table represents all the data relating to an individual (in this example, a customer) Rows are also called RECORDS or TUPLES Each row consists of a set of individual values that correspond to columns. Each value musst have the data type specified by its column.

9. Keys Usually, in a table, we need a way to uniquely identify each individual row. If the table is one like the example, using names might not be the best way of doing this. (think of a 3 rd entry: Peter Smith with address 15 Main Street). What are the chances of finding another customer named Peter Smith? Peter this peter might be the only one living at the given address, but then we are using 2 fields

10. Keys The way around this is to create an additional field (customerID) in this example that contains this unique identifier to the subject This identifier field on the table is called the key or the PRIMARY KEY. A Key can also be made of more than one field.

11. Keys Databases usually consist of multiple tables and use a keys as a reference from one table to another. For example, the following table represents orders placed by the customers: orderIDcustomerIDAmountDate 132700010 Aug 2011 234500022 Aug 2011 32142701 Oct 2011 ORDERS

12. CustomerIDNameAddress 1Bill Jobs2011 Apple Way, SF, 10064 CA 2Steve Gates1 Microsoft Drive, SD, 98711 CA 3Peter Smith15 Main Street orderIDcustomerIDAmountDate 132700010 Aug 2011 234500022 Aug 2011 32142701 Oct 2011 CUSTOMERS ORDERS

13. Schemas The complete set of table designs for a database is called the database schema. A schema shows tables along with their columns, data types of the column and indicate the primary key (PK) of each table (and any foreign key) Keys on the schema are recognised because they are underlined

14. Database Schema Schema Example Customers (customerID, Name, Address) Orders (orderID,customerID*, Amount, Date) Primary Key Foreign Key Note, the * besides customerID represents a foreign key

15. How to Design your Web Database

16. Wed DB Design Knowing when you need a new table and what the key should be can be something of an art. Most of the time, a few design principles can be followed

17. Principles 1.Think about the real world objects you are modelling. – Rule of Thumb: Each class of real world objects modelled will need its own table.

18. Principles 2.Avoid storing redundant data – On the working example, the customer information could be stored in the orders table, but now imagine a more complex customer information table with 5 fields and then the order table with another 2 fields – This “super table” will not only be huge but will also contain redundant data which is a waste of space (imagine Peter Smith doing weekly orders and you having to store all his details every time)

19. Principles – This kind of super tables can also lead to what is known as update anomalies (situations where we change the database and end up with inconsistent data) – There are 3 kinds of update anomalies that need to be avoided: Modification (i.e. a customer moves and we need to update the address fields…) Insertion (i.e. a new customer is inserted several times and the address needs to be entered correctly each time) Deletion (occurs when deleting rows from the database; imagine we delete the fulfilled orders from the database, when all the orders for a single customer have been fulfilled, we will no longer have his address on file)

20. Principles 3.Use Atomic Colum Values – In each attribute, in each row, store only ONE thing 4.Choose Sensible KEYS – Make sure that the keys you choose are unique 5.Thinks about the questions you want to ask the database – Make sure that the database contains all the data required, and the appropriate links exists between tables to answer the questions you have

21. Principles 6.Avoid Designs with Many Empty Attributes – Having many NULL values in the database is a bad idea, it wastes storage space and causes problems when working out totals and other functions on numerical columns. – When a user sees a null in a database, they don’t know if it’s because this attribute is irrelevant, whether there is a mistake in the database, or whether the data just has not been entered yet.

22. MySQL Databases SQL Language Tutorial

23. 23 Web Database Architecture web client (e.g browser) HTTP response Database HTTP request server system web server PHP Preprocessor

24. Web Database Architecture Typical web database transaction stages: –HTTP request issued from client –Web Server receives request for.php script and passes it for processing –PHP engine begins parsing the script. The script includes a command to connect to the database and execute a query. PHP opens a connection to the MySQL server –MySQL server receives the database query, processes it and sends the result back to the PHP Engine –PHP Engine finishes running the script – this usually involves the formatting of the results in HTML. Returns resulting HTML to server –Server passes the HTML back to the browser. 24

25. 25 Web Database Architecture Process is basically same regardless of scripting engine or database server Often Web Server, script engine and database server all run on the same machine As your application grows in size, you may want to separate your PHP application into tiers – a database layer (deals exclusively wit MySQL), a business logic layer (core of application), and a presentation layer (manages the HTML output).

26. Managing access to the Database Access to the database can be provided by using the root administrator level or by creating a user for this database For testing purposes, admin access is enough For security reasons, deploying using admin level access is NOT recommended. MySQL provides different security levels for users, called privileges 26

27. 27 MySQL Privilege System MySQL supports a sophisticated privilege system Principle of least privilege –A user (or process) should have the lowest level of privilege required to perform his assigned task. –For example, to run queries, a user does not need all privileges. Three basic types of privileges –Regular Users –Administrators –Special

28. 28 MySQL Privilege System - USERS PrivilegeApplies toDescription SELECTTables, columnsAllows users to select rows from tables INSERTTables, columnsAllows users to insert new rows into tables UPDATETables, columnsAllows users to modify values in existing table rows DELETETablesAllows users to delete existing table rows INDEXTablesAllows users to create and drop indexes on particular tables ALTERTablesAllows users to alter the structure of existing tables CREATEDatabases, tablesAllows users to create new databases or tables DROPDatabases, tablesAllows users to drop (delete) databases or tables.

29. 29 MySQL Privilege System - ADMINISTRATORS PrivilegeDescription CREATE TEMPORARY TABLESAllows an administrator to use the keyword TEMPORARY in a CREATE TABLE statement FILEAllows data to be read into tables from files and vice-versa LOCK TABLESAllows the explicit use of a LOCK TABLES statement PROCESSAllows an administrator to view server processes belonging to all user RELOADAllows an administrator to reload grant tables and flush privileges, hosts, logs and tables REPLICATION CLIENTAllows use of Show Status on replication masters and slaves REPLICATION SLAVEAllows replication slave servers to connect to the master server. SHOW DATABASESAllows a list of all databases to be seen with a SHOW DATABASE statement. Without this, users see only databases to which they have other privileges SHUTDOWNAllows an administrator to shut down the MySQL server SUPERAllows an administrator to kill threads belonging to any user

30. 30 MySQL Privilege System – SPECIAL PrivilegeDescription ALL Grants all privileges listed in previous tables USAGE Grants no privileges. This privilege creates a user and allows him to log on, but it doesn’t allow him to do anything. Usually you will add more privileges later.

31. MySQL Database Manipulation Strings

32. MySQL Databases Most commonly used database program for developing database-driven web sites with PHP MySQL is open source and runs on a majority of operating systems (UNIX, Linux, Mac Os and Windows) Although PHP can be used with any database through its set of ODBC functions, it comes loaded with MySQL specific functions which makes for a tight integration between PHP and MySQL

33. SQL Command/Queries SQL is a computer language, and like languages in general, SQL has its rules, grammar, and a set of special or reserved words. Different variants of the language have evolved over the years because different vendors offer additional features to manipulate data in the name of competition

34. SQL Command/Queries SQL has many commands, but they can be divided in two major categories: – The Commands to Manipulate the Tables in the database – The Commands to Manipulate the Database itself There are many excellent tutorials on the web that cover all the SQL commands and how to use them: http://www.w3schools.com/sql/default.asp

35. English Like Grammar An SQL statement makes a “REQUEST” or “QUERIES” the database in the form of a statement The first word is an English verb, an action word called a command such as (show, use, select, drop, etc.) The command is followed by a list of noun-like words, such as show database, use database, or create database. The statement might contain prepositions such as in or from; for example: – show tables in database – Select phones from customer_table

36. English Like Grammar The language also allows for conditional clauses to refine the queries such as: – select companyname from suppliers where suppllierid > 20 When listing multiple items in a query, the items are separated by commas – select companyname, phone, address from suppliers Semicolon (;) is the standard way to terminate each query statement. Some database systems do not require it, but MySQL does (except for the USE and QUIT command).

37. SQL Reserved Words ALTERCROSS JOINFULL JOINJOINONSELECT ANDDELETEGROUP BYLEFT JOINORSET ASDROPINSERTLIKEORDER BYUPDATE CREATEFROMINTOLIMITRIGHT JOINWHERE

38. Case Use Database and table names are case sensitive on UNIX but not in WINDOWS. A convention is to always use lowercase names for databases and their tables SQL commands are not case sensitive, but by convention SQL commands are CAPITALIZED for clarity while only the first letter of the field, table, and database names is capitalized – SELECT * FROM Persons WHERE FirstName=‘John’; If Performing pattern matching with the LIKE and NOT LIKE commands, the the pattern being searched for is case sensitive when using MySQL

39. Useful SQL Commands SHOW DATABASES: Used to see what databases are available on your database server USE: The USE command makes the specified database your default database SHOW databases; USE database_name;

40. Useful SQL Commands SHOW TABLES IN: Displays all the tables within a database SHOW/DESCRIBE: Either of these commands is used to see what type of data can be assigned to a table. The DESCRIBE command is specific to MySQL databases SHOW TABLES IN database; DESCRIBE table_name;

41. SQL Data Manipulation Language (DML) SQL is a non procedural language providing syntax for extracting data, including a sytax to update, insert, and delete records. The Query and Update commands together form the Data Manipulation (DML) part of SQL – SELECT – UPDATE – DELETE – INSERT INTO

42. SQL-DML SELECT SELECT: This command is mandatory when performing a query; it is used to retrieve data from a table based on some criteria It specifies a coma separated list of fields to be retrieved, and the FROM clause specifies the table(s) to be accessed. The results are stored in a result table known as the result-set. The * symbol can be used to represent all of the fields

43. SQL-DML SELECT Format: SELECT column_name(s) FROM table_name Example: SELECT LastName, FirstName, Address FROM Students; To select specified columns, the SELECT command is followed by a comma separated list of fields to be selected from the table

44. SQL-DML SELECT SELECT DISTINC: this keyword is used to return only distinct (unique) values from the table. If there are multiple values of a specified field, the distinct result-set will display only one. Format: SELECT DISTINCT column_name(s) FROM table_name Example: SELECT DISTINCT ShipName FROM Orders;

45. SQL-DML SELECT LIMIT(X): this keyword specifies the number of rows to be returned from the beginning of the result-set. X specifies the rows returned Format: SELECT column_name(s) FROM table_name LIMIT X; Example: SELECT ShipName FROM Orders LIMIT 10;

46. SQL-DML SELECT WHERE Clause: It is used to select a field when a certain criteria set of conditions are desired The WHERE Clause is optional To create the conditions (called selection criteria) SQL provides a set of operators to further qualify what criteria should be specified

47. SQL-DML Where Operators OperatorDescriptionExample =Equal toWHERE country = ‘ireland’ <>, !=Not equal toWHERE country != ‘USA’ >Greater thanWHERE salary > 28000 <Less thanWHERE age < 35 >=, <=Greater/Less Than or EqualWHERE cost >=1200 IS [NOT] NULLIs NULL (vo value) or Not NULLWHERE birth = NULL BETWEENBetween an inclusive rangeWHERE last_name BETWEEN ‘Doherty’ AND ‘McDAID’ LIKESearch for a value like a patternWHERE name LIKE ‘D%’ NOT LIKESearch for a value not like a patternWHERE country NOT LIKE ‘Sw%’ !, NOTLogical not for negationWHERE age ! 10; ||, ORLogical ORWHERE order_number > 10 || part_number = 80 &&, ANDLogical ANDWHERE age>12 && age < 21 XORExclusive ORWHERE status XOR

48. SQL-DML Using Quotes: Quotes are always an issue in programming languages. (single quotes?, double quotes?, when?) SQL uses single quotes around text values (MySQL also accepts double quotes) Numeric Values should not be enclosed in quotes.

49. SQL-DML Comparing Strings: When comparing strings using =, the string must be exactly as typed for the condition to be true – this include length and type of characters. NULL: Null means that there is not a value in the field, or it is unknown, but does not mean a value of zero.

50. DML-SQL LIKE – NOT LIKE: The pattern matching operator can be used as a condition in the WHERE clause, allowing the selection of rows that are ‘like’ or match a pattern A percent sign (%) can be used as a wildcard to match any possible character that might appear before and/or after the character(s) specified. A _ is used to match a single character. The LIKE/NOT LIKE condition can be used in any valid SQL statement, including SELECT, INSERT, UPDATE or DELETE.

51. DML-SQL Examples of the wildcard % uses: SELECT CompanyName, Country FROM Customers WHERE country LIKE ‘SW%’; – Returns all the customers and countries in which the country starts with “Sw” i.e. Sweden, Switzerland SELECT City, Country FROM suppliers WHERE City LIKE ‘%o’; – Returns all cities and countries where the % matches any city that ends with a letter o. SELECT CompanyName FROM customers WHERE CompanyName LIKE ‘%Super%’ – Returns all company names where the % matches any company name that contains the pattern “Super”

52. DML-SQL Examples of the wildcard _ uses: SELECT Extension, Firstname FROM Employees WHERE extension LIKE ‘4_ _’; – Returns all extensions and first names where the exetension has three characters and the first character is a 4.

53. DML-SQL ORDER BY: Used to sort the output of a query in either ascending (ASC, the default) or descending (DESC) order where the values being sorted are either strings or numbers Format: SELECT column_name(s) FROM table_name [WHERE condition] ORDER BY column [ASC, DESC] Example: SELECT Company,Ordernumber FROM Orders ORDER BY Company;

54. SQL-DML INSERT The INSERT statement is used ot insert new rows into a table. After the VALUES keyword, a comma- separated list of column names follows Format: INSERT INTO table_name VALUES (value1, value2, … ) INSERT INTO table_name (column1, column2,…) VALUES (value1, value2, … ) Example: INSERT INTO Shippers (CompanyName, Phone) VALUES (‘FEDEX’,’416- 555-1221’);

55. SQL-DML INSERT Usually, the tables have a primary key column that is usually set to auto-increment; when this is the case, the id of the table is created by the database engine automatically Letting the database increment the PRIMARY KEY ensures that the value is always unique.

56. SQL-DML UPDATE The UPDATE statement is used to modify data in a table. The UPDATE command is followed by the name of the table where the data will be changed, followed by the SET statement to inidcate what field will be changed, and then the new value that will be assigned to the field The WHERE clause further qualifies what data is to be modified, thereby limiting the scope of the UPDATE

57. SQL-DML UPDATE Format: UPDATE table_name SET column_name = new value WHERE column_name = some_value; Example: UPDATE orders SET ShipCountry=‘Spain’ WHERE CustomerId = ‘whitc’;

58. SQL-DML DELETE The DELETE statement is used to delete rows in a table and returns the number of rows that were deleted. DELETE uses the FROM clause to specify the name of the table that contains the data you want to delete The WHERE clause specifies the criteria to identify what data should be removed. BE CAREFUL: Without the WHERE clause ALL ROWS are DELETED

59. SQL-DML DELETE If the ORDER BY clause is specified, the rows are deleted in the order that is specified. The LIMIT clause places a limit on the number of rows that can be deleted. Format: DELETE FROM table_name WHERE column_name = some_value; Example: DELETE FROM orders WHERE ShipCountry = ‘Greenland’;

60. Database Creation using SQL Data Definition Language

61. The DDL part of SQL permits database objects to be created or destroyed Indexes (keys) can be defined Links between table can be specified Constraints between database tables can be imposed Often decisions to create and remove databases are handled by a database administrator and having permission to create and drop tables depends on what access rights are granted.

62. Data Definition Language The most important Data Definition Statements in SQL are: – CREATE TABLE – ALTER TABLE – DROP TABLE – CREATE INDEX – DROP INDEX DML and DDL can be executed on the MySQL command line or in the MySQL tab of PHPMyAdmin

63. MySQL Command Line Windows 1.Start WAMP Server 2.On the command prompt browse to find the mysql.exe file (usually located in c:\wamp\bin\mysql\mysqlX.X.X\bin\) – (X.X.X=5.5.8) 3.At the end of the command line add the following:..\mysql.exe --user=student --password=student If it is your own laptop, then user=root -- password= MAC 1.Start MAMP and Start the Servers 2.Go to Terminal 3.Type in the following command: /Applications/MAMP/Library/bin/mysql --host=localhost -uroot -proot

64. DDL DATABASES on SERVER Once on the MySQL command line the SHOW DATABASES command will show you the existing databases on the server: mysql> show databases;

65. DDL CHOOSING a DATABASE In order to select a database from the list to use it, you will need the USE command: mysql> use test;

66. DDL Show & Describe To see what tables are in the database, use the SHOW TABLES command: mysql> show tables; To display the contents of a table, you can use the SELECT command.

67. DDL Show & Describe To see what type of data can be assigned to a table, use the DESCRIBE command (specific to MySQL) or the SHOW FIELDS IN command (standard SQL) mysql> describe table_name;

68. DDL Creating a Database To create a new database, use the CREATE DATABASE command: CREATE DATABASE database_name;

69. DDL Creating a Database mysql>CREATE DATABASE gallerydb; mysql>USE DATABASE gallerydb;

70. Art Gallery Database

71. Creating Tables In order to create the tables, the database should be first properly designed: For each field on each table you should know what type of data is it going to store After that is done, then the required datatype should be selected Note: it is a waste of space (memory) to, for example, declare a field as integer when you know beforehand that it will not store a number bigger that 1000.

72. SQL Data Types Numbers 72 TypeRangeStorage (Bytes) Description INTEGER-2 31..2 31 -1 / 0.. 2 32 -14Whole Number INT UNSIGNED0..2 32 -14Non Negative Whole Number TINYINT-127..128 / 0..2551Very small integers SMALLINT-32768..32767 /0..65535 2Small Integers MEDIUMINT-8388608..8388607 / 0..16777215 3Medium Sized Integers BIGINT-2 63..2 63 -1 / 0..2 64 -18Big Integers

73. SQL Data Types Numbers 73 TypeRangeStorage (Bytes) Description FLOAT (precision)Depends on precisionVariesCan be used to specify single or double precision floating point numbers FLOAT [(M,D)]±1.175494351E-38.. ±3.402823466E+38 4Single precision floating point number. These numbers are equivalent to FLOAT(4) but with a specified display width(M) and number of decimal places(D). DOUBLE[(M,D)]±1.7E+308.. ±2.2E-3088Double precision floating point number. These numbers are equivalent to FLOAT(8) but with a specified display width (M) and number of decimal places(D). DECIMAL[(M,D)]VariesM+2Floating point stored as a CHAR NUMERIC / DEC / FIXEDAs AboveSynonym for DECIMAL

74. SQL Data Types STRINGS 74 TypeRangeDescription CHAR (M)0 – 255 Characters Fixed-length string of length M, where M is between 0 and 255. Using the BINARY keyword specifies that the data should be considered casesensitive. VARCHAR (M)1 - 255Same as above, except they are variable length.

75. SQL Data Types STRINGS 75 TypeRangeDescription BLOB65535 charactersA normal sized Binary Large Object (BLOB) (Case sensitive) TEXT65535 charactersNormal Sized TEXT object. (Case insensitive)

76. MySQL Data Types – DATE and TIME 76 TypeRangeDescription DATE1000-01-01.. 9999-12-31A date. Will be displayed as YYYY-MM-DD TIME-838:59:59.. 838:59:59A time. Will be displayed as HH:MM:SS. Note that the range is much wider than you probably will ever want to use. DATETIME1000-01-01 00:00:00.. 9999-12-31 23:59:59 A date and time. Will be displayed as YYYY- MM-DD HH:MM:SS TIMESTAMP (M)1970-01-01 00:00:00.. Sometime in 2037 A timestamp useful for transaction reporting. The display format depends on the value of M. The top range depends on the limit on Unix YEAR[(2|4)]70 – 69 (1970 – 2069) 1901 - 2155 A year. You can specify two or four digits format. Each has a different range.

77. SQL Data Types Timestamp Examples 77 TypeDisplay TIMESTAMPYYYYMMDDHHMMSS TIMESTAMP(14)YYYYMMDDHHMMSS TIMESTAMP(12)YYMMDDHHMMSS TIMESTAMP(10)YYMMDDHHMM TIMESTAMP(8)YYYYMMDD TIMESTAMP(6)YYMMDD

78. DML-DDL-Example We are going to create a database to store pets information and on it we are going to create a table called dog PETS dog(name,owner,breed,sex,birth,death) Note: when designing your database, be mindful of all the already learned database design techniques like normalization.

79. PET DB Dog Table SQL Instructions: mysql>create database pets; mysql>use pets; mysql>create table dog ( -> name varchar(20), -> owner varchar(20), -> breed varchar(20), -> sex char(1), -> birth date, -> death date); mysql>describe dog;

80. Altering a Table Adding a Primary Key On the previous example, we forgot to add a primary key field; we are going to correct that example using the ALTER TABLE command: When altering a table, we redefine its structure by adding or dropping columns, keys, indexes and tables. The Alter command can also be used to change column names, types and the table name.

81. Altering a Table Adding a Primary Key First, we need to add a new column to serve as the primary key; we will call it pet_id: mysql>alter table dog add pet_id int(11) first; Now we need to modify this new column to make it the primary key – primary key fields should not be null and should have an auto_increment property set: mysql>alter table dog modify column pet_id int(11) -->not null auto_increment primary key; The following command accomplishes the same task in a single line: mysql>alter table dog add pet_id int(11) not null auto_increment first, -->add primary key(pet_id);

82. Adding PK

83. Dropping Tables Dropping Databases To delete a table use the DROP command: mysql>drop table dog; To delete a database use the DROP command: mysql>drop database pets;

84. SQL Functions The following functions are used to alter or format the output of a SQL query. Functions are provided for strings, numbers, dates, server and information, and so on. They return a result-set. When using SELECT with a function, the function, as it was called, is displayed as the name of the column in the result-set

85. Numeric Functions

86. String Functions

87. Date & Time Functions FunctionExample NOW()select NOW()  2012-01-17 15:58:45 CURDATE()select CURDATE()  2012-01-17 CURTIME()select CURTIME()  15:58:45 DAYOFYEAR(date)select DAYOFYEAR(‘2006-12-15’)  349 DAYOFMONTH(date)select DAYOFMONT(‘2012-01-19’)  19 DAYOFWEEK(date)select DAYOFWEEK(‘2012-01-19’)  5 (Thursday); Sunday is 1 WEEKDAY(date)select WEEKDAY(‘2012-01-19’)  3 (week starts at 0 on Monday) MONTHNAME(date)select MONTHNAME(‘2012-01-19’)  January DAYNAME(date)select DAYNAME(‘2012-01-19’)  Thursday YEAR(date)select YEAR(‘2012-01-19’)  2012 QUARTER(date)select QUARTER(‘2012-01-19)  1