1. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Performance Audit E-Government Program Evaluation Cláudia Augusto Dias Minister-Rapporteur: Valmir Campelo Minister-Rapporteur: Valmir Campelo Audit Team: Daniel Dias Pereira (supervisor) Cláudia Augusto Dias (coordinator) Cláudia Augusto Dias (coordinator) André Kirchheim (Secex-RS) André Kirchheim (Secex-RS) Carmen Pereira Rêgo Meireles Carmen Pereira Rêgo Meireles Paulo Henrique Ramos Medeiros Paulo Henrique Ramos Medeiros

2. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI E-Gov Programs overall purpose to increase the supply and improve the quality of public services and information provided through electronic means to increase the supply and improve the quality of public services and information provided through electronic means

3. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Audit Scope Audit problem Audit problem In what way have the Program actions contributed toward the provision of electronic public services directly to citizens?

4. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Audit Scope Audit questions Audit questions 1 - Has the Program structure been able to stimulate the Federal Public Administration Agencies to implement the electronic government policy? 2 - Have monitoring and evaluation activities contributed to improve the provision of electronic public services? 3 - Is the provision of electronic public services citizen-centric?

5. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Methodological Strategy Documental research about the Brazilian E-Gov Program; national and international laws, rules, and technical guidelines related to E-Gov Documental research about the Brazilian E-Gov Program; national and international laws, rules, and technical guidelines related to E-Gov Context analysis using diagnostic tools Context analysis using diagnostic tools Interviews with E-Gov Program coordinators and web portal managers Interviews with E-Gov Program coordinators and web portal managers Heuristic evaluation and tests applied to selected electronic public services Heuristic evaluation and tests applied to selected electronic public services Focus groups with Technical Committees coordinators and web portal managers Focus groups with Technical Committees coordinators and web portal managers

6. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI On-line surveys - 76 web portal managers and 4.944 citizens (users of the selected services) On-line surveys - 76 web portal managers and 4.944 citizens (users of the selected services)

7. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI How easy is to locate the agencys website How easy is to locate the agencys website How easy is to locate the service How easy is to locate the service Accessibility Accessibility Availablity Availablity Usability Usability Reliability Reliability Capacity to answer user queries Capacity to answer user queries Privacy and security Privacy and security Service time Service time Overall satisfaction Overall satisfaction Ten dimensions

8. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Evaluated services Annual Exempt Tax Return 2005 Annual Exempt Tax Return 2005 Application for Sickness Insurance Application for Sickness Insurance Application for Death Grant Application for Death Grant Application for Paid Maternity Leave Application for Paid Maternity Leave Public Domain Portal Public Domain Portal National First Job Program National First Job Program

9. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Question 1 - Program coordination and articulation Some conclusions Weakening of discussion bodies and formulation of e-gov policy Poor dissemination of the guidelines and documents generated by the E-Gov Program Some recommendations review the institutional model in force define the timeline for implementing the National E-Gov Development Plan disseminate the guidelines and documents generated by the E-Gov Program

10. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Question 2 - Monitoring and evaluation Some conclusions Insufficient systematic monitoring and evaluation by Program coordinators and web portal managers Poor dissemination of good e-gov practices by Program coordinators Some recommendations define performance indicators for E-Gov services evaluation identify and disseminate tools, techniques and good practices to public managers

11. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Question 3 - Provision of citizen-centric electronic public services Low percentage of citizen-centric transactional services Recommendation Brazilian agencies should identify priority services from citizens point of views and evaluate their adaptation or implementation on the Web

12. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Question 3 - Provision of citizen-centric electronic public services Non-compliance with recommendations proposed in documents issued by E-Gov Program coordinators Recommendation monitor and evaluate samples of e-gov portals in order to guide web portal managers to comply with e-gov guidelines Images enabled Images disabled Save Clear Submit Help For data to be saved, click on the SAVE button

13. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Benefits of implementing TCU recommendations Better and stronger articulation between E-Gov Program coordinators and Executive Branch agencies Better dissemination of E-Gov guidelines to agencies Exchange of experiences and dissemination of good practices among public managers More managerial data for improving the provision of public services on the Web More information about the target audience of electronic public services (needs and opinions) Identification of priority services, from citizens point of views, to be implemented on the Web Higher citizen satisfaction

14. Brazilian Court of Audit Secretary for Information Technology Audit - SEFTI Contact information Telephone: 55 61 3316-5253 E-mail:claudiaad@tcu.gov.br Full report and Executive summary (in Portuguese): www.tcu.gov.br/avaliacaodeprogramasdegoverno Executive summary (in Spanish and English): Intosai E-Gov Database