brazilian court of audit secretary for information technology audit - sefti performance audit...

Brazilian Court of AuditBrazilian Court of AuditSecretary for Information Technology Audit - SEFTISecretary for Information Technology Audit - SEFTI

Performance AuditPerformance Audit

E-Government Program EvaluationE-Government Program Evaluation

Cláudia Augusto DiasCláudia Augusto Dias

Minister-Rapporteur: Valmir CampeloMinister-Rapporteur: Valmir Campelo Audit Team: Daniel Dias Pereira (supervisor)Audit Team: Daniel Dias Pereira (supervisor)

Cláudia Augusto Dias (coordinator)Cláudia Augusto Dias (coordinator) André Kirchheim (Secex-RS)André Kirchheim (Secex-RS) Carmen Pereira Rêgo MeirelesCarmen Pereira Rêgo Meireles Paulo Henrique Ramos Medeiros Paulo Henrique Ramos Medeiros


E-Gov Program’s overall E-Gov Program’s overall purposepurpose

““to increase the to increase the supply and supply and improve the improve the quality of public quality of public services and services and information information provided through provided through electronic electronic meansmeans””


Audit ScopeAudit Scope

Audit problemAudit problem

““In what way have the Program actions In what way have the Program actions contributed toward the provision of contributed toward the provision of electronic public services directly to electronic public services directly to citizens?”citizens?”


Audit ScopeAudit Scope

Audit questionsAudit questions

1 - Has the 1 - Has the Program structureProgram structure been able to been able to stimulate the Federal Public Administration stimulate the Federal Public Administration Agencies to implement the electronic government Agencies to implement the electronic government policy?policy?

2 - Have 2 - Have monitoring and evaluationmonitoring and evaluation activities activities contributed to improve the provision of electronic contributed to improve the provision of electronic public services?public services?

3 - Is the provision of electronic public services 3 - Is the provision of electronic public services citizen-centriccitizen-centric??


Methodological StrategyMethodological Strategy

Documental research about the Brazilian E-Gov Documental research about the Brazilian E-Gov Program; national and international laws, rules, and Program; national and international laws, rules, and technical guidelines related to E-Govtechnical guidelines related to E-Gov

Context analysis using diagnostic tools Context analysis using diagnostic tools

Interviews with E-Gov Program coordinators and Interviews with E-Gov Program coordinators and web portal managersweb portal managers

Heuristic evaluation and tests applied to selected Heuristic evaluation and tests applied to selected electronic public serviceselectronic public services

Focus groups with Technical Committees Focus groups with Technical Committees coordinators and web portal managerscoordinators and web portal managers


On-line surveys - 76 web portal managers and On-line surveys - 76 web portal managers and 4.944 citizens (users of the selected services)4.944 citizens (users of the selected services)


How easy is to locate the agency’s websiteHow easy is to locate the agency’s website How easy is to locate the serviceHow easy is to locate the service AccessibilityAccessibility AvailablityAvailablity UsabilityUsability ReliabilityReliability Capacity to answer user queriesCapacity to answer user queries Privacy and securityPrivacy and security Service timeService time Overall satisfactionOverall satisfaction

Ten dimensionsTen dimensions


Evaluated servicesEvaluated services Annual Exempt Tax Return 2005Annual Exempt Tax Return 2005 Application for Sickness InsuranceApplication for Sickness Insurance Application for Death Grant Application for Death Grant Application for Paid Maternity LeaveApplication for Paid Maternity Leave Public Domain Portal Public Domain Portal National First Job ProgramNational First Job Program


Question 1 - Program coordination and Question 1 - Program coordination and articulation articulation

Some conclusionsSome conclusions Weakening of

discussion bodies and formulation of e-gov policy

Poor dissemination of the guidelines and documents generated by the E-Gov Program

Some recommendationsSome recommendations review the institutional model in force define the timeline for implementing the National E-Gov

Development Plan disseminate the guidelines and documents generated by the E-Gov

Program

63 13

1 35 40

1 38 37

2 41 33

1 37 38

53 23

0 10 20 30 40 50 60 70 80

CEGE Resolution n. 7

Usability Guidelines

AccessibilityTechnical Guidelines

Accessibility Model

InteroperabilityStandards

Visual IdentityManual

Do you know this law, rule or recommendation?

No response

Yes

No


Question 2 - Monitoring and evaluationQuestion 2 - Monitoring and evaluationSome conclusionsSome conclusions Insufficient systematic

monitoring and evaluation by Program coordinators and web portal managers

Poor dissemination of good e-gov practices by Program coordinators

Some recommendationsSome recommendations define performance indicators for E-Gov services evaluation identify and disseminate tools, techniques and good practices to

public managers

Are there policies for evaluating your electronic

services ?

3

3637

Yes, a formally defined policy.

Yes, an informal policy.

No.


Question 3 - Provision of citizen-centric electronic public Question 3 - Provision of citizen-centric electronic public services services

Low percentage of citizen-centric transactional services

RecommendationRecommendation Brazilian agencies should identify priority services from citizens’

point of views and evaluate their adaptation or implementation on the Web

Number of electronic services available on the agency's portal

(N=76)

17

40

8

56

No response

Less than 10

From 10 to 20

From 21 to 30

More than 30

Predominant E-Gov Stage (N=76)

2

37

11

13

11

2

No response

Information Presentation

Information Search

Limited Interactivity

Electronic Transaction

Seamless E-Gov


Question 3 - Provision of citizen-centric electronic public Question 3 - Provision of citizen-centric electronic public services services Non-compliance with recommendations proposed in documents

issued by E-Gov Program coordinators

RecommendationRecommendation monitor and evaluate samples of e-gov portals in order to guide

web portal managers to comply with e-gov guidelines

Images enabled Images disabled

Save Clear

Submit Help

For data to be saved, click on the SAVE button


Benefits of implementing TCU Benefits of implementing TCU recommendations recommendations

Better and stronger articulation between E-Gov Program coordinators and Executive Branch agencies

Better dissemination of E-Gov guidelines to agencies

Exchange of experiences and dissemination of good practices among public managers

More managerial data for improving the provision of public services on the Web

More information about the target audience of electronic public services (needs and opinions)

Identification of priority services, from citizens’ point of views, to be implemented on the Web

Higher citizen satisfaction


Contact informationContact information

Telephone: 55 61 3316-5253Telephone: 55 61 3316-5253

E-mail:E-mail: [email protected]@tcu.gov.br

Full report and Executive summary (in Portuguese): Full report and Executive summary (in Portuguese):

www.tcu.gov.br/avaliacaodeprogramasdegovernowww.tcu.gov.br/avaliacaodeprogramasdegoverno

Executive summary (in Spanish and English):Executive summary (in Spanish and English):

Intosai E-Gov DatabaseIntosai E-Gov Database

brazilian court of audit secretary for information technology audit - sefti performance audit...

Documents

egov program slide

egov services evaluation

egov program coordinators

public managers

web slide

valmir campelo audit

quality of public services

job program slide