Semantically Enhanced Model Experiment Evaluation Process (SeMEEP) within the Atmospheric Chemistry Community Chris Martin 1,2, Mo Haji 2, Peter Dew 2, Peter Jimack 2, Mike Pilling 1 1 School of Chemistry, University of Leeds 2 School of Computing, University of Leeds

2 Outline of the Presentation Introduction Atmospheric community SeMEEP ELN Provenance capture Conclusion and next stage

3 Section 1 Overview Application domain – atmospheric community –Reliance on computational models to evaluate data Motivation –Study how to transition from today's ad-hoc processes practises –Sustainable process of Gathering, community evaluation and sharing data & models between scientists Minimising changes to proven working practises of the scientist Within world-wide co-laboratories

4 Related projects CombeChem –Experimental organic chemistry –From source to long term data –perseveration (knowledge) –Semantically-enabled ELN –Data-driven workflow Collaboratory for Multi-Scale Chemical Science –Multi-layer chemical model myGrid –Bio-informatics and related areas (semantic pattern matching –Reusable semantic workflow using SMD (semantic metadata) –Data Quality Karama2 –Weather forecasting – computation modelling –Data-driven workflow Quantum Thermo Kinetic Mechanism Reacting Flow Chemistry Chemistry Simulation

5 Section 2 Atmospheric Chemistry Seeks to understand the chemical processes (reactions) taking place in the lower atmosphere (e.g. smoke) It has significant implication for both: –Air Quality –Climate Change

6 The Master Chemical Mechanism (MCM) Data repository of elementary chemical reactions & rate constants The mechanism is described by a computational model that is evaluated against experimental data –Chamber experiments –Field experiments

7 Section 3 SeMEEP Today –Typically within the atmospheric chemistry community the provenance is recorded in an ad-hoc, unstructured fashion, using a combination of traditional lab-book, word processing documents and spreadsheet. Move to more sustainable evaluation process supports the gathering, evaluation and sharing of data and models Using semantic metadata

8 SeMEEP Vision SeMEEP semantically-enabled MEEP –Supports the organisation of information but critically, records its provenance (say to recover secondary data) Mike Pilling : “SeMEEP approach will radically enhance the effectiveness of a research community to deliver new science“

9 Requirements with metadata for elementary chemical reactions

10 Requirements for metadata capture for elementary reactions Only published data Rate constants from several labs No access to the raw data No access to secondary data SeMEEP will provide this.

11 Current Evaluation Processes for the MCM

12 Envisioned Evaluation Processes

13 Section 4 Electronic Lab-Books (ELNs) ELNs address the limitations of the current methods of provenance capture. Southampton ELN for organic chemistry experiments. Benefits to the modeller Modelling process can be automatically captured Searchable Remote access is possible Provenance is structured Possible to use resolvable references to resources

14 Will User attach quality metadata? Motivate users: –By demonstrating the value of provenance in their day-to-day work Writing publication Managing their data Reinterpretting the data. –Management –Publishers

15 Recoding provenance for modelling 3-level process –Experimental modelling plan –Modelling iterations –Modelling layer where provenance is capture has the modelling process proceeds using data-driven workflows where data is a first class object

16 The Modelling Process - A Three Layer Mapping

17 MCM Mechanism being investigated

18 ELN Process

19 ELN Screenshots Prompts displayed when changing the changing the chemical mechanism; Editing a reaction Adding a new reaction

20 ELN Screenshots

21 ELN Modelling SMD Architecture

22 Evaluation Methodology In-depth interviews with members of the atmospheric chemistry model group here at Leeds, covering: –Demonstration of the prototype –User testing of the prototype –Discussion of scenarios involving the use of the prototype (e.g. ) Analysis –Interviews recorded and transcribed –Analysed using techniques from grounded theory

23 Evaluation Barriers to adoption: –Effort required at modelling time for provenance capture “[in] your lab book you can write down what ever you want [but with an ELN] it is going to take time to go through the different protocol steps”. –When asked if they would use an ELN requiring a similar amount of user input to the prototype the response was positive: “Yeah, I think it would be a good thing. I don’t think it is too much extra … work.” –Rather than viewing the prompts for user annotation as interruption to their normal work the user recognised the value of being prompted “is a good way to do it because otherwise you won’t [record the provenance].”

24 Evaluation Users intuitively grasped the benefits of recording provenance with an ELN and that the benefits would be realised after the time of modelling by a number of stakeholders: –“if someone else wants to look at … [your provenance], that’s great because the person can see exactly what you have done, where you have been and where to go next. And for yourself, if you are writing up a PhD... [you can] … see exactly what you’ve done whereas currently you have to rifle through lab-books to see exactly what you have done.”

25 Section 5 Conclusions and future work Outlined SeMEEP and ELN –User evaluated proposed modelling ELN Addressed case studies –IUPAC –MCM Developing a case study with the Geomagnetic community User and System issues –Application of actively theory to capture requirements and user evaluation –Querying and inference –Address QoS issues (e.g. security, scalabilty, dynamic roles-based access control)

26 Questions