1. with specialization in SQLite 22 Mar 2017
Introduction to SQL
with specialization in SQLite
22 Mar 2017

2. Motivation Background about SQL language and SQL solutions
Provide pointers to good tutorials on SQL
Hands-on session with SQLite
SQLite and MySQL interaction with Python
Disclaimer: I am not expert in SQL. If you find some material somehow contradicting this presentation, most likely they are right and I am wrong
Bottom line: Try to get a feeling of SQL if you know even less than I do … and enjoy the cookies !

3. Relational database management system (RDBMS) (from Wikipedia)
Relational model: invented by E. F. Codd, of IBM's San Jose Research Laboratory (1970)
A common choice for the storage of information since the 1980s
They have replaced hierarchical and network databases, but have not been replaced by object databases
As a today, they are co-used with NoSQL databases
Minimum requirements:
Present the data to the user with tables of tuples (i.e. as a collection of tables with each table consisting of a set of rows and columns)
Provide relational operators to manipulate the data in tabular form

4. Structured Query Language (SQL) (from Wikipedia)
Query Language for most commercial relational DBMSes
Percentage of database sites using any given technology
Oracle Database – 70%
Microsoft SQL Server – 68%
MySQL (Oracle Corporation) – 50%
IBM DB2 – 39%
IBM Informix – 18%
SAP Sybase Adaptive Server Enterprise – 15%
SAP Sybase IQ – 14%Teradata – 11%

5. General structure of an SQL database (from wikipedia)

6. History of SQL (from wikipedia)
SQL was initially developed at IBM by Donald D. Chamberlin and Raymond F. Boyce in the early 1970s
In the relational model, tables contain tuples. In SQL tables are lists of rows: duplicates are allowed, and sequencing inherent
ANSI standard in 1986 and ISO standard in 1987
Revised several times since them
Different vendors implement parts of the standards, and they are not fully compatible
SQL components:
Data Definition Language (DML)
Data Manipulation language (DML)
Query language

7. Is SQLite just another SQL database implementation ?
Released in 2000, SQLite is file-based, unlike other databases, like SQL Server and MySQL which are server-based
No setup or administration is needed
Database files are cross-platform
Available in Unix (Linux, Android, MacOSX) and Windows
Check if you have it installed:
In Mac: Execute sqlite3 from command line (.exit to leave)
In Windows: You may have to install. Follow instructions from

8. SQL and NoSQL DBs The new buzzword in the database world is NoSQL
Advantages of Non-relational databases
Table-less: easier to manage and they provide a higher level of flexibility
Document oriented: No need to develop a detailed database model
Easier scalability through support for Map Reduce
Disadvantages:
you would not use it for tabular data or data where you’re interested in navigating and querying through joins
Lack of standardization
Conclusion: Use the right DB for your data
Nice Quora discussion here

9. Where to learn SQL
Although you have MOOCs in Coursera and edX, I found the following tutorials more useful:
Codeacademy SQL course
Tutorials Point for SQLite
Tutorials Point for MySQL
Tutorial that incorporates differences among SQL implementations
There are also many interesting pages where you can find for particular topics
SQLite webpage
Comparison of SQL Implementations

10. How to use SQL SQL GUIs: From the shell
MySQL Workbench
SQLite DB Browser
From the shell
With python (or other languages) SQL extensions

11. SQL Language elements Statements can be nested into another statements
A transaction consists of several statements: needs to be committed for other database users to see the changes
For queries, we can operate carry out the union, intersection, or complement of two statements
Flow control is possible, but we will not study that

12. MySQL Data Types Numeric Data Types (ANSI types) Date and Time Types
(UNSIGNED) INT, TINYINT, SMALLINT, MEDIUMINT, BIGINT, FLOAT (M,D), DOUBLE (M,D), DECIMAL(M,D)
Date and Time Types
DATE, DATETIME, TIMESTAMP, TIME, YEAR(M)
String Types
Indexable: CHAR(M), VARCHAR(M)
Non-indexable: BLOB (binary), TEXT (+ TINY, MEDIUM, LONG versions)

13. SQLite Data Types
SQLite uses a more general dynamic type system. In SQLite, the datatype of a value is associated with the value itself, not with its container.
SQLite supports the concept of type affinity on columns. Any column can still store any type of data but the preferred storage class for a column is called its affinity
SQLite does not have a separate Boolean storage class. Instead, Boolean values are stored as integers 0 (false) and 1 (true)
SQLite does not have a separate storage class for storing dates and/or times, but SQLite is capable of storing dates and times as TEXT, REAL or INTEGER values.

14. SQL Operators (from Wikipedia)

15. SQLite Hands-on Tutorial
sqlite>.help (have a look at the different commands)
sqlite>.quit (or .exit)

16. Create a Database from .csv file
>sqlite3 usflights.db sqlite>.database sqlite>CREATE TABLE flights( id INT NOT NULL PRIMARY KEY, carrier VARCHAR(2), origin VARCHAR(3) NOT NULL, destination VARCHAR(3) NOT NULL, flightnum VARCHAR(4) NOT NULL, dep_time DATETIME NOT NULL, arr_time DATETIME NOT NULL, delay INT NOT NULL ); sqlite>.separator ",” sqlite>.import flights_small.csv flights sqlite>.mode column sqlite>.header on sqlite>SELECT * FROM flights LIMIT 5; (PRAGMA table_info(flights) shows table structure (SQLite only)) (DROP TABLE table_name would remove the table from the database)

17. Querying a Database Table (I)
Check the difference between these two:
sqlite> SELECT carrier, destination FROM flights LIMIT 10;
sqlite> SELECT carrier, destination AS dest_CODE FROM flights LIMIT 10;
Finding unique values (or combinations of values)
sqlite> SELECT DISTINCT origin FROM flights;
sqlite> SELECT DISTINCT origin, destination FROM flights;
And counting the number of unique values (or combinations)
sqlite> SELECT COUNT(DISTINCT origin) FROM flights;
sqlite> SELECT COUNT(*) FROM (SELECT DISTINCT origin, destination FROM flights);
Where clause:
sqlite> SELECT carrier FROM flights WHERE origin="JFK";
Exercises:
How many different carriers fly from JFK? (4)
How many flights in the table left from or arrive to JFK? (13)

18. Querying a Database Table (II)
sqlite> SELECT carrier, COUNT(*) FROM flights GROUP BY carrier;
sqlite> SELECT carrier, AVG(delay) FROM flights GROUP BY carrier;
sqlite> SELECT carrier, AVG(delay)
FROM flights
GROUP BY carrier
HAVING AVG(delay)>10
ORDER BY AVG(delay) DESC;
Exercises:
How many flights have the different carriers operated from JFK? ([2,1,2,2])
What is the maximum delay of any flight in the table? (277)
Calculate the number of delayed and non-delayed flights by carrier:
carrier               Vuelos retrasados  
AA                    27               
AS                    11               
CO                    9                
DH                    2                
DL                    24               
EV                    5                
FL                    2                
HP                    3                
MQ                    10               
NW                    5                
OH                    2                
OO                    2                
RU                    8                
TW                    1                
UA                    11               
US                    12               
WN                    23  
carrier               Vuelos no retrasados  
AA                    48                  
AQ                    1                   
AS                    7                   
B6                    1                   
CO                    30                  
DH                    7                   
DL                    35                  
EV                    11                  
FL                    4                   
HA                    1                   
HP                    10                  
MQ                    22                  
NW                    27                  
OH                    7                   
OO                    13                  
RU                    12                  
TW                    6                   
TZ                    1                   
UA                    29                  
US                    29                  
WN                    41

19. Creating a table for airports
Exercise: Create a second table “airports” with the following columns:
code VARCHAR(3) NOT NULL PRIMARY INDEX
city VARCHAR(20) NOT NULL
We can now insert new data in the table:
sqlite> INSERT INTO airports VALUES ("LAX", "Los Angeles");
sqlite> INSERT INTO airports VALUES ("JFK", "New York");
sqlite> INSERT INTO airports VALUES ("ATL", "Atlanta");
Now, we add an additional column for the altitude of the airport
sqlite> ALTER TABLE airports ADD COLUMN altitude CHECK(altitude >0);
It is also possible to rename the column, or to change its type. SQLite does not allow to drop a column from a table, but other SQL databases do.
Try the following commands:
sqlite> UPDATE airports SET altitude=-10 WHERE code="JFK";
sqlite> UPDATE airports SET altitude=13 WHERE code="JFK";
sqlite> INSERT INTO airports (code,altitude) VALUES ("DFW", 603);
sqlite> INSERT INTO airports (code,altitude,city) VALUES ("DFW", 603, "Dallas");
sqlite> DELETE FROM airports WHERE code="LAX";
sqlite> INSERT INTO airports VALUES ("LAX", "Los Angeles", 126);

20. Joins Try the following commands:
sqlite> SELECT city, carrier, flightnum, dep_time
FROM airports T1 JOIN flights T2
ON T1.code=T2.origin;
sqlite> SELECT count(*)
FROM airports CROSS JOIN flights;
Exercise:
Add a new airport without flights:
sqlite> INSERT INTO airports
VALUES ("MAD", "Madrid", 610);
Check now the number of records of the following joins:
LEFT JOIN
LEFT JOIN WHERE B.Key is NULL
JOIN
CROSS JOIN
SQLite does not currently support other kinds of joins

21. Joins It is also possible to join the results of statements:
sqlite> SELECT T1.carrier, T1."Vuelos retrasados", T2."Vuelos no retrasados”
FROM (SELECT carrier, count(*) AS "Vuelos retrasados”
FROM flights
WHERE delay>0
GROUP BY carrier) T1
JOIN (SELECT carrier, count(*) AS "Vuelos no retrasados”
FROM flights WHERE delay<=0
GROUP BY carrier) T2
ON T1.carrier=T2.carrier
WHERE T1.carrier LIKE "%A%";;
Exercises:
Find the average and maximum altitude of the departure airport for all the flights per carrier. You should only consider the airports for which the altitude is known
Using a left join operation, count the number of flights for which the departure airport is not in the airports table

22. Views
A view is a SQLite statement that is stored in the database with an associated name
They are virtual, i.e, they do not take additional space
Views allow users to do the following:
Structure data in a way that users or classes of users find natural or intuitive
Restrict access to the data such that a user can only see limited data instead of complete table
Summarize data from various tables which can be used to generate reports
SQLite views are read-only
sqlite> CREATE VIEW smart_flights AS SELECT T1.carrier, T1.flightnum, T2.city AS "Ciudad de salida" FROM flights T1 JOIN airports T2 ON T1.origin=T2.code;
sqlite> .tables
sqlite> SELECT * FROM smart_flights;
sqlite> DELETE FROM flights WHERE flightnum=1179;

23. Python sqlite3 module
sqlite3.connect(database [,timeout ,other optional arguments])
connection.cursor([cursorClass])
cursor.execute(sql [, optional parameters])
connection.execute(sql [, optional parameters])
cursor.executemany(sql, seq_of_parameters)
cursor.executescript(sql_script)
connection.total_changes()
connection.commit()
connection.rollback()
connection.close()
cursor.fetchone()
cursor.fetchall()