2. Quality Assessment Program as a Contribution to the SIBi/USP Management IATUL – 2006 Porto, Portugal Cybelle de Assumpção Fontes caf@usp.br University of São Paulo Integrated Library System São Paulo, SP - Brazil

3. Quality Assessment Program as a Contribution to the SIBi/USP Management Maria Imaculada Cardoso Sampaio (Coordinator) Adherbal Caminada Netto Adriana A. Barreiros Adriana Cybele Ferrari Cybelle Assumpção Fontes Márcia Elisa Garcia de Grandi Maria Cristina Olaio Villela Rosa Maria F. Zani IATUL – 2006 Porto, Portugal

4. IATUL - 2006 Who we are:

5. IATUL - 2006 University of São Paulo A public, autonomous institution, funded by the State of São Paulo, created in 1934. The largest institution of higher education and research in Brazil, and the third in size in Latin America. Teaching units in São Paulo, Bauru, Piracicaba, Pirassununga, Ribeirão Preto and São Carlos. Numbers 202 Undergraduate Courses 544 Graduate Courses 77.205 Students Enrolled 5.078 Faculty Members 15.008 Staff (Database 2004)

6. IATUL - 2006 University of São Paulo Integrated Library System (SIBi) 40 Libraries Collection Books 1.827.872 Theses and Dissertations 196.679 Journals (Current Titles) 20.985 Users 126.192 SIBiNet – SIBi/USP Services Network DEDALUS - Bibliographic Database (OPAC) Access to electronic journals Access to data bases Rare Books online Digital Library of Dissertations and Theses (Database 2004)

7. IATUL - 2006 Background Users and Libraries Library focus Perceptions and needs of users Information systems can better meet their users’ needs with services more suitable and useful to their demands Users’ opinion as a tool in the information system management

8. IATUL - 2006 Background Quality and Libraries SERVQUAL Wide application for search service quality Gap between user’s expectation and perception (what is really provided) Dimensions for identify quality –Tangibles: appearance of physical facilities, equipment, personnel, and communication materials –Reliability: ability to perform the promised service dependably and accurately –Responsiveness: willingness to help customers and provide prompt service –Assurance: knowledge and courtesy of employees and their ability to convey trust and confidence –Empathy: the caring, individualized attention provided to the customers

9. IATUL - 2006 Background Quality and Libraries LibQUAL+ SERVQUAL application to the library environment Association of Research Libraries (ARL) and Texas A&M University Libraries (EUA) Users’ perceptions of excellence, current and minimal level of service acceptable Longitudinal data and benchmark

10. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) PAQ Structured in 2002 Focused on the users’ perception Based on SERVQUAL/LibQUAL+ dimensions: tangibles reliability responsiveness assurance empathy Search First Phase: 2002/2004 Second Phase: 2005/2006

11. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) First Phase 2002/2004 Users’ opinion regarding products and services provided by the libraries of the System Five electronic questionnaires – –Questions based on SERVQUAL’s dimensions Personal interview – –Open questions: what the user likes least, likes most and what he/she would like to change in the library

12. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) First Phase 2002/2004 Results Local actions reverted on behalf of the users   Subsidized the SIBi/USP strategic planning from 2003   Proposal of a new Program’s phase

13. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Second Phase 2005/2006 Users’ needs – SIBiNet SIBi Service Network Resources Interview based on the Critical Incident Technique Electronic questionnaire

14. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Second Phase 2005/2006 Interview based on the Critical Incident Technique Procedure for collection of direct observations of human behavior (Flanagan, 1954) Incidents observed which present special significance and to find systematically defined criteria A valid critical incident that can define users’ needs must consider two characteristics: be specific and refer to a clearly demarcated scene be able to describe either the service provider in behavioral terms, or the product and services, with particular adjectives

15. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Second Phase 2005/2006 Interview based on the Critical Incident Technique Users in libraries in which PAQ participants work Analysis and selection of main critical incidents   Critical incidents and SERVQUAL dimensions   tangibles – broken links   reliability – lack of full theses   responsiveness – training on SIBiNet   assurance – site updating   empathy – suitable layout

16. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Second Phase 2005/2006 Electronic questionnaire (at SIBiNet) Based on the categorized information (Critical Incident) Structure   Access to the site – compatibility and network   Contents – database, periodicals   Services – another portals   DEDALUS   Users’ categories: USP community, non-USP community and SIBi/USP technical team   Open form: free expression

17. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Second Phase 2005/2006 Results 2304 questionnaires Open form – answers analyzed DEDALUS – weak points Site – weak points Questionnaire Level of dissatisfaction – constant in the three users’ groups Global analysis of dissatisfaction – Pareto’s Chart

18. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Second Phase 2005/2006 Pareto’s Chart (Disagreement + Total disagreement) + -

19. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Final Considerations First phase Enhancements regarding products and services were implemented Supplementation of the allocated budget for the acquisition of bibliographic material; renewal and updating of the libraries’ technological facilities; extension of service hours in some libraries; remodeling of physical spaces in the library; changes in the libraries access control systems; grounds for the Project of replacement of the Libraries’ database management software (DEDALUS); inclusion of new information sources in several knowledge areas; definition of a continuous capacity building of the teams, focused on quality.

20. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Final Considerations Second Phase Supported proposal for the overall change of the SIBi/USP site.

21. IATUL - 2006 Quality Assessment Program (PAQ) Integrated Library System (SIBi) Final Considerations The proposed continuous assessment program was proven feasible in the management and has been adopted as a managerial tool. The credibility and respect that the information system has before its community can only be certified by the actual system’s auditor: the user.

22. IATUL - 2006 Thank you! dtsibi@org.usp.br

23. IATUL - 2006 Good luck!