Presentation is loading. Please wait.

Presentation is loading. Please wait.

Less orthodox uses of SiR at Aberdeen Less orthodox uses of SiR at University of Aberdeen Scotland John S. Lemon.

Similar presentations


Presentation on theme: "Less orthodox uses of SiR at Aberdeen Less orthodox uses of SiR at University of Aberdeen Scotland John S. Lemon."— Presentation transcript:

1 Less orthodox uses of SiR at Aberdeen Less orthodox uses of SiR at University of Aberdeen Scotland John S. Lemon

2 Less orthodox uses of SiR at Aberdeen Six Examples Create multiple files from single SiR schema and data from SPSS.sav Convert SPSS output to formatted data Analysing web access logs Decoding XML for RSC archives Merging raw data files to give MySQL update commands

3 Less orthodox uses of SiR at Aberdeen Multiple files from single SNAP questionnaire software Web surveys return responses via E- mail Designed for Micros..t software Expects a file per response UoA has one file per mailbox so contains many s Need to process responses

4 Less orthodox uses of SiR at Aberdeen Multiple files from single Responses delimited by ***START SURVEY DATA*** ***END SURVEY DATA*** Procedures –Filter responses into one mailbox –Create one file / message with name based on date + counter Use PATTERN to find START & END OPEN + date functions to create files

5 Less orthodox uses of SiR at Aberdeen SiR schema and data from SPSS Common theme – hint for Tony ? –Michael StaleyCyril –Jon Johnston? –MeSPSS2SiR Read the output from SPSS with vPQL –Descriptives –File Info Create schema and data files

6 Less orthodox uses of SiR at Aberdeen SPSS output to formatted data More complex interpretation of SPSS output Multiple Regression and Correlation analysis converted to –30 input files with 20 rows x 15 cols Run annually Saves manual extraction and re-typing Reduction of error rate

7 Less orthodox uses of SiR at Aberdeen Analysing web access logs Problem - needed to find number of accesses to a specific web page Had to be done retrospectively as no “access_log” file available Variable format Increase ‘counter’ – IF( PATTERN > 0 ) for relevant URL /Apr/2005:03:00:29 /~che194/research/rovib99/bk/img242.gif sv-crawlfw3.looksmart.com 03/Apr/2005:03:00:29 /newsletter/i22/index.shtml

8 Less orthodox uses of SiR at Aberdeen Decoding XML for RSC archives Royal Society of Chemists archives – 50m of shelving –160 Gbytes of PDF files 195,000+ XML files to describe contents of the matching 195,000+.pdf files In 56 folders with up to 70 subfolders Need to recurse, retain and also modify directory structure

9 Less orthodox uses of SiR at Aberdeen Decoding XML for RSC archives No reasonably priced web based bibliographic software can read XML Didn’t investigate expensive ones !! XML from RSC contained many ‘strange’ phrases such as –éé –ωω Which wouldn’t translate

10 Less orthodox uses of SiR at Aberdeen Decoding XML for RSC archives Some lines contained more than one ‘strange’ phrase –Tried DO REPEAT Too rigid Won’t work with –SUBPROCEDURE worked - reading a file Some files didn’t follow conventions –Non-paired syntax on own –Extreme values – 150 authors !!!

11 Less orthodox uses of SiR at Aberdeen Decoding XML for RSC archives Originally created upload / input file for biblio package directly – no data base ‘Management’ decided to change target biblio software part way through Different format of input Required new approach –Data Base –Multiple output files

12 Less orthodox uses of SiR at Aberdeen Decoding XML for RSC archives New target software ( DSPACE ) has different –IMPORT format –Directory structure –‘Reject list’ / problem phrases in XML input record changed & increased Re-think & re-code – easy with vPQL –Especially when using subroutines

13 Less orthodox uses of SiR at Aberdeen Raw data files to MySQL update Final unorthodox use Involves creating a MySQL command file with SiR Integrated HelpDesk managed by SupportWorks software –18,000+ clients ( students & staff ) –40+ analysts( ‘front’ & ‘back’ office ) Based on MySQL data base

14 Less orthodox uses of SiR at Aberdeen Raw data files to MySQL update Update user information via –GUI interface OK for small jobs –SQL commands to MySQL data base Need to ‘automate’ as –2,500+ students arrive and 2,500+ leave each year –Staff changes as well Daily operation

15 Less orthodox uses of SiR at Aberdeen Raw data files to MySQL update Support Works SiR & vPQL Pre-processor SQL commands Distance Learners Telephones Departments Daily update UserlistStudentspasswd

16 Less orthodox uses of SiR at Aberdeen Raw data files to MySQL update Data comes from different sources with no file containing all information for user Need to extract best information –Mr. J. Lemon –J. Lemon –John S. lemon Merged to give Mr. John S. Lemon

17 Less orthodox uses of SiR at Aberdeen Raw data files to MySQL update File formats cover –Fixed ASCII –Tab / Comma / vertical bar delimited Use vPQL to read files and store in different records / tables vPQL programme to write out complete records to give SQL update commands

18 Less orthodox uses of SiR at Aberdeen Raw data files to MySQL update com045:##com045:7910:2000:J.S.LEMON, J.S. LEMON, COMPUTING CENTRE, EDWARD WRIGHT BUILDING:/home/com/com045:/bin/csh |GLOBAL|Y|EMADDR1|j.s.lemon|EMDIR1|E|EMADDR2|teach45|EMDIR2|N |EMADDR3|com059|EMDIR3|N|TELENUM1|3350|TELEDIR1|E|TELEPRI1|a|TELEN UM2|n/a|TELEDIR2|E|TELEPRI2|a|TELENUM3|n/a|TELEDIR3|E|TELEPRI3|a| X COM045 MR. JOHN S. LEMON J.S.LEMON X COM DIR INFORMATION SYSTEMS & SERVICES COMPUTING CENTRE DIS DISS UPDATE userdb SET firstname = "John S.", surname = "Lemon", abdn_school = "DISS", abdn_department = "Computing Centre", abdn_deptmnemonic = "com", costcenter = "DISS", site = "Computing Centre", abdn_status = "Staff", abdn_title = "Mr.", = password = telext = "3350", abdn_faculty = "dis", attrib3 = "Active", priority = "Low", webflag = "1775", abdn_lastupdatex = " " where keysearch = "com045";

19 Less orthodox uses of SiR at Aberdeen General rules / Conclusions Have an overall idea – but be flexible Think ‘out of the box’ – exercise your lateral thinking Data bases aren’t always needed Develop in sections Use small samples to test Expect re-planning

20 Less orthodox uses of SiR at Aberdeen General rules / Conclusions Read the list of vPQL functions it will surprise you !! –PATTERN} –SBST} –SRST} very useful –FORMAT} –NUMBR} Be prepared to have multiple ‘reads’ of data

21 Less orthodox uses of SiR at Aberdeen General rules / Conclusions If you use a Data base exploit all the facilities in SiR / vPQL –Records –Tabfiles –Secondary indexes Use WRITE statements and / or DEBUG with or without FULLMONITOR Sometimes you have to accept 99% success as good !

22 Less orthodox uses of SiR at Aberdeen


Download ppt "Less orthodox uses of SiR at Aberdeen Less orthodox uses of SiR at University of Aberdeen Scotland John S. Lemon."

Similar presentations


Ads by Google