CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 CAA Data Mining Tool Chris Perry

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 CAA DATA MINING SERVICE The CAA Data Mining Service is intended to assist users in identifying intervals suitable for further study. Any CAA CEF parameter can be accessed via the service However, it is ideally used with simple, low resolution datasets Complex expressions can be defined to locate intervals of interest By default nearest neighbour join is applied to allow quick data comparison The non-joined source data is also accessible if required A join function can be applied in the expressions for linear/cubic/spline interpolation where this is known to be appropriate A range of functions are provided for building expressions The design allows easy extension of additional functions Loops are not supported Feedback on what advanced functions are wanted would be useful

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 Data Mining Service. Can be operated either with direct CAA access or via Stream I/F Small number of functions but easy to extend Various limitations and needs to be web safe – needs a lot of testing CAA Database CAA Database Catalogue cefmerge Data Mining Service Quickplots, event lists Design

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 DATA MINING SERVICE CAA DATA MINING WORKFLOW STREAMING I/F STREAMING I/F OTHER DATA I/F OTHER DATA I/F CAA DM FUNCTIONS CAA DM FUNCTIONS CAA DM PLOTTING CAA DM PLOTTING CXFORM & SSW DLM CXFORM & SSW DLM 3 RD PARTY PRO/DLM 3 RD PARTY PRO/DLM IDL WEB DEV GUI WEB DEV GUI BESPOKE WEB GUI BESPOKE WEB GUI HTTP GET/POST HTTP GET/POST IDL PROCEDURE IDL PROCEDURE Layered design of the data mining service. This helps with maintainability and expandability of the service Also aids web security since user does not have direct access to IDL Design

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 Developer Web GUI Current interface is basically unchanged from 2012 review.

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 Developer Web GUI Not yet clear what CSA approach will be to DM. CAA system will continue to be used internally to help define requirements and for verification, validation and prototyping of value added products. Example: Used to help verify new WBD electron density product by reversing the algorithm and over plotting over WHISPER spectrogram. {sc=C4}{wspec} fpe:= 8.980*SQRT(electron_density__{sc}_CP_WBD_ELECTRON_DENSITY) fce:= B_mag__{sc}_CP_FGM_SPIN *28.0/ fl:= fpe * fce oplot(t_,fl,P_COL_:=255,P_THICK_:=2.0) oplot(t_,fpe, P_LINE_:=1,P_COL_:=255) oplot(t_,fce, P_LINE_:=1)

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 Developer Web GUI Development interface provides a lot of flexibility. But has a learning curve

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 Developer Web GUI Easily embed scripts within a web page to provide a simpler user interface (with specific functionality)

CAA Cross-Cal 18, Cosener’s, 23-25th Oct 2013 Demo: Need input on what users would like from a data mining service What users really want out of a DM service Flexibility vs ease of use Next step is to prototype some more user friendly bespoke interfaces following on from the simple query