MIAMExpress development and local installation DESPRAD Meeting,November 2002 Mohammad shojatalab

Talk structure History Underlying concepts Design & Development Current status Local Installation Future

History (March 2001 ~ present) Need for a submission tool for ArrayExpress Obviously It should be a web base tool It was supposed to be quick and dirty prototype Start about May 2001

Underlying concepts Based on MIAME concepts and questionnaire Submission of Experiment, Arrays, Protocols Avoiding free text as much as possible Using controlled vocabulary

Login Pending/New Experiment Sample1Sample2Sample3 Sample n Sample protocol Hybridisations Hyb protocol Array 1 Array 2 Array 3 Array n Scanning protocol Data 1 Data 2 Data 3 Data n Image analysis protocol Combined Experiment Data Transformation protocol Submit Final free text comment Extracts 1…n E1E1 E2E2 EnEn E1E1 E2E2 EnEn E1E1 E2E2 EnEn E1E1 E2E2 EnEn Extraction protocol

Design considerations Complex submission structure Long submission time (may be weeks) So it needs a database It meant to be fast, Open source and free Usable as a Lab Notebook Free database management system Web based submissions MAGE-ML file as output

Technologies & Tools Using MySQL DBMS for its database Using Perl CGI technology Using DBI, DBD::MySQL to interact with db Using javascript CVS as source code repository to keep track of changes and also incorporate changes made by other developers in to all developers working copy.

MIAMExpress GUI, MAGExpress, … Biology Layer Data acces dat a Physical data layer (Database, files,…) Biology;MIAME Layer Data access functions

Development Started at May 2001 Simple data model; around 30 tables Avoiding hard code in program Debug tools to assist developer Log functionality which write down user’s activities in a file to assist developer to find out what is wrong if an controlled error happened. Readable and maintainable code

Impact on data access layer and APIs New Requirement Requirement Analysis Impact analysis Essential information for development Impact on data model Impact on UIImpact on MAGExpress Development Test Development New Release Beta version

Requiremen t Analysis  Impact Analysis  Developmen t  Test  Steady state MIAMExpress development states

Apache HTTP Test Production Apache HTTP Developers CVS Infrastructure

Submissions Types Array submission: Array Description File (Excel sheet, tab delimited files) Experiment submission experiment design,samples,extractions,labelled extracts,protocols (Web based forms) Hybridization data files (Excel sheet, tab delimited files) combined data file (Excel sheet, tab delimited files)

MAGE-ML Creation; MAGExpress module MAGEstk: A set of APIs which are created from OM and are able to read MAGE-ML file and create object structure and vice versa To create MAGE-ML file from MIAMExpress we have to know which piece of data fits where in the model that means we need map MIAMExpress Data model(schema) to MAGE-OM Still we have problem because MAGE classes are quite abstract in terms of working with physical data

MAGE-ML Creation (2) To have a object oriented design we need to have a new set of classes which are derived from MAGE classes but are MIAMExpress specific Objects which are instantiated from these new set of classes know how get their attributes from MIAMExpress database Also they are inheriting all of the properties of their parents

MAGE-ML Creation; Example biosrc instantiated from original BioSource class mx_biosrc instantiated from MXBioSource class MXBioSource class is inherited from original BioSource class biosrc.go&Load_your_data -> ERROR!!! mx_biosrc.go&Load_your_data -> you have a loaded object from MIAMExpress Encapsulation of dirty works inside the object

How go&Load_your_data works? By calling our one or a sequence of appropriate biology(MIAME) APIs. Note that the method name in all Classes is the same but behaviour is context sensitive which is hard coded inside; For example ; for BioSource we say; mx_biosrc. go&Load_your_data; And also for labelled extract; Mx_label. go&Load_your_data and all others the like.

How we create MAGE-ML file We Create the top level object; experiment experiment.go&create_all_your_associations; which creates whole object structure experiment.go&Load_your_data; which load data to whole structure write MAGE-MLby starting from experiment top level object

Incorporating data into MAGE-ML BioAssay Data, Derived BioAssay Data Problem with different data formats; Generic, GenePix, Genesping,… So we have to have plugins in MIAMExpress for each of these formats. Retrieving the following info from data files: Quantitation types, Features, Reporters, Composite Sequences

Problem with new releases Usually new releases means new schema as well. that means we have to change the data access layer. that means we have to change our mapping model which map MIAMExpress schema to MAGE model. that means we have to migrate the existing data. that introduce data migration module; dmm dmm is have to be provided with new release

Missing bits behind the scene We are using MySQL which at least at the moment doesn’t support constraints and foreign keys, sub queries. They have promised to to solve these in their next releases soon. Persistent connection with database. Having an automated schema  MAGE mapping.

Current status Release 1.0 is ready soon (Dec 2002) With this we get the experiment submissions, and create the MAGE-ML file get the Array submissions and create the MAGE- ML file

Local Installation Installation of most stable release of MySQL Installing Apache HTTP server Perl core modules and some other from CPAN Xerces-C and Xerces-Perl modules Create MIAMExpress database and load seed data Change database entry in a constant module Change data of the system option table -machine name; on which MySql is running -Database name,Address of database configuration file -Path of the data files,Path of the application -Main URL, CGI URL, Path of log files,…

Future MIAMExpress 1.x/2.0 ?? Graphical representation of experiment structure. Organism specific Integrated with Curation tool ?? be able to work with a full MAGE-ML file? cover all missing pieces behind the scene

URL European Union for funding (TEMBLOR). MIAMExpress developers: Niran,Sergio Helen Parkinson and others database curators in Microarray Group in EBI. Ugis Sarkans, Alvis Brazma All people involved in MAGEstk collaborative effort. Acknowledgement

Questions