Relational Databases: Basic Concepts BCHB Lecture 21 11/12/2014BCHB Edwards

11/12/2014BCHB Edwards2 Outline What is a (relational) database? When are relational databases used? Commonly used database management systems Using existing databases Creating and populating new databases Python and relational databases Exercises

11/12/2014BCHB Edwards3 (Relational) Databases Databases store information Bioinformatics has lots of file-based information: FASTA sequence databases Genbank format sequences Store sequence, annotation, references, annotation Good as archive or comprehensive reference Poor for a few items Relational databases also store information Good for a few items at a time Flexible on which items

11/12/2014BCHB Edwards4 Relational Databases Store information in a table Rows represent items Columns represent items' properties or attributes NameContinentRegionSurface AreaPopulationGNP BrazilSouth America IndonesiaAsiaSoutheast Asia IndiaAsiaSouthern and Central Asia ChinaAsiaEastern Asia PakistanAsiaSouthern and Central Asia United StatesNorth America

11/12/2014BCHB Edwards5 Relational Databases Tables can be millions of rows Can access a few rows fast Countries more than 100,000,000 in population? Countries on the “Asia” continent? Countries that start with “U”? Countries with GNP = NameContinentRegionSurface AreaPopulationGNP BrazilSouth America IndonesiaAsiaSoutheast Asia IndiaAsiaSouthern and Central Asia ChinaAsiaEastern Asia PakistanAsiaSouthern and Central Asia United StatesNorth America

11/12/2014BCHB Edwards6 When are Relational Databases Used? LARGE datasets Does data fit in memory? Store data first ask questions later Lookup or sort by many keys For single key, simple data structures often work Store results of expensive compute or data-cleanup Compute once and return results many times "Random" or unknown access patterns Specialized data-structures not appropriate Use string/sequence indexes for sequence data

11/12/2014BCHB Edwards7 Common DBMS Oracle Commercial, market leader, widely used in businesses MySQL Free, open-source, widely used in bioinformatics, suitable for large scale deployment Sqlite Free, open-source, minimal installation requirements, no users, suitable for small scale deployment

11/12/2014BCHB Edwards8 Lets look at some examples We'll use a third-party program to "look at" Sqlite databases: SqliteStudio (Linux), SqliteSpy (Windows), … Download examples: World.db3, taxa.db3 from Course data folder Use SqliteStudio to look at examples World.db3, taxa.db3

11/12/2014BCHB Edwards9 Using existing databases Use the "select" SQL command to find relevant rows select * from Country where Population > ; select * from Country where Continent = 'Asia'; select * from Country where Name like 'U%'; select * from Country where GNP = ; Each command ends in semicolon ";". "where" specifies the condition/constraint/rule. "*" asks for all attributes from the relevant rows. Lets experiment with world and taxa databases.

11/12/2014BCHB Edwards10 Using existing databases Select can combine (“join”) multiple tables Use the where condition to match rows from each table and “link” corresponding rows… select * from taxonomy, name where taxonomy.rank = 'species' and name.name_class = 'misspelling' and name.tax_id = taxonomy.tax_id

11/12/2014BCHB Edwards11 Using existing databases Select can sort and/or return top 10 select * from taxonomy limit 10; select * from taxonomy order by scientific_name; select * from taxonomy order by tax_id desc limit 10;

11/12/2014BCHB Edwards12 Using existing databases Select can count and do string matching. "like" uses special symbols: % matches zero or more symbols _ match exactly one symbol Some RDBMS support regular expressions MySQL, for example. select count(*) from taxonomy where scientific_name like 'D%';

11/12/2014BCHB Edwards13 Creating databases Use the "create" SQL command to create tables CREATE TABLE taxonomy ( tax_id INTEGER PRIMARY KEY, scientific_name TEXT, rank TEXT, parent_id INT ); CREATE TABLE name ( id INTEGER PRIMARY KEY, tax_id INT, name TEXT, name_class TEXT );

11/12/2014BCHB Edwards14 Populating databases Use the "insert" SQL command to add rows to tables Usually, the special id column is initialized automatically INSERT INTO name (tax_id,name,name_class) VALUES (9606,'H. sapiens','synonym'); SELECT * from name where tax_id = 9606;

11/12/2014BCHB Edwards15 Python and Relational Databases Issue select statements from python and iterate through the results Sometimes it is easiest to make Python do some of the work! import sqlite3 conn = sqlite3.connect('taxa.db3') c = conn.cursor() c.execute(""" select * from name where name like 'D%' limit 10; """) for row in c: print row

11/12/2014BCHB Edwards16 Python and Relational Databases Use parameter substitution for run-time values import sys import sqlite3 tid = int(sys.argv[1]) conn = sqlite3.connect('taxa.db3') params = [tid,'scientific name'] c = conn.cursor() c.execute(""" select * from name where tax_id = ? and name_class = ?; """,params) for row in c: print row

11/12/2014BCHB Edwards17 Next-time: Object-relational mappers Setup python to treat tables as classes, rows as objects # Set up data-model from model import * hs = Taxonomy.get(9606) for n in hs.names: print n.name, "|", n.nameClass condition = Name.q.name.startswith('Da') for n in Name.select(condition): print n.name, "|", n.nameClass

11/12/2014BCHB Edwards18 Lab exercises Read through an online course in SQL sqlcourse.com, sql-tutorial.net,... Write a python program to lookup the scientific name for a user-supplied organism name.