ombudsman operations: evaluation planning and basics · • # of people using ombudsman services...
TRANSCRIPT
![Page 1: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/1.jpg)
Ombudsman Operations: Evaluation Planning and Basics
ICANN, Marina Del Rey
February 2009
![Page 2: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/2.jpg)
About your presenters
• Frank Fowlie• ICANN’s 1st Ombudsman
– November 2004
• Varied career
• Wrote doctoral dissertation on Ombudsman evaluation
• John Zinsser• Principal ‐ Pacifica Human Communications
• MA ‐ 1st MACRA Grad
• Consultant & Researcher
![Page 3: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/3.jpg)
![Page 4: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/4.jpg)
Your Expectations:
• The Gain a greater understanding of Ombudsmanship and systems design.
• To discuss effective performance measurements How to effectively survey our client base (especially complainants ‐when their expectations differ from our legislative authority) How to interpret survey results
• In conjunction with a course that I am currently enrolled in, I would enjoy learning more about what an Ombuds is and does, as well as learning about the global market for services of this kind.
![Page 5: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/5.jpg)
Your expectations:
• The Ombudsman's work is unique in nature. The Ombudsman is posssessed with certain characteristics above all those under his/her jurisdiction. But at the same time, tje Ombudsman has to balance this in terms of effectiveness, productivity and efficiency etc. I expect from the assessment course to give some answers on how one can assess and measure the Ombudsman's work. Whether it is useful to have one in the absence of any assessment tools? Should the office remain complaint driven? As a public office, accountability of the office is necessary.
![Page 6: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/6.jpg)
Your expectations:
• A primer on the roles and responsibilities of Ombudsmen. Networking and possible employment / consulting opportunities. Ability to develop new means for evaluating the effectiveness of systems.
• that the time together will provide groundwork for the future
• To obtain more practical knowledge. Learn from experienced professionals in the filed of dispute system design.
• Learn best practices for assessing the effectivemess of Ombudsman's Office.
![Page 7: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/7.jpg)
What you think is critical:
• How to best assess a dispute resolution system
• to effectively survey complainants
• To upskill myself and impart the knowledge gained to others in the office
• How to complete my work for your company in a satisfactorymanner
• Assessing a system with information that is often subjective, personal and often confidential.
• metrics that others use successfully
• Theory transforms into practice
• Providing upward feedback
![Page 8: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/8.jpg)
To Start• What is the purpose of your Office of the Ombudsman?– Is this what you, your Organization, or chartering institution wants?
– How do you know that?
• Are you strategically tuned to deliver that?– How do you know?– Has anyone helped you check?
• What value does your Office of the Ombudsman create and deliver?
![Page 9: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/9.jpg)
What we will cover:
• Evaluation basic terminology;
• Review of the ICANN Ombudsman model;
• Review 6 reasons to evaluate;
• Value statements;
• Create an RMAF;
• Valuing the Ombudsman in $$
• Self assessment; external assessment; third party reviews; surveys; and out reporting
![Page 10: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/10.jpg)
Lexicon
Measurement1. The act of measuring or the process of being measured.2. A system of measuring: measurement in miles. 3. The dimension, quantity, or capacity determined by measuring:
the measurements of a room.(dictionary.com)
For our purposes:Activities, undertaken PRIMARILY by the Ombudsman,to determine what outputs and outcomes occur or not, and what satisfaction results from these activities.
![Page 11: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/11.jpg)
Lexicon
Evaluation1. Act of ascertaining or fixing the value or worth of2. An appraisal of the value of something
[synonym: valuation, rating]3. Converting an expression into a value using some reduction
strategy. (dictionary.com)
For our purposes:Efforts to determine the economic and/or humanistic value of Ombudsman activities and the outcomes of these activities. Can be determined by the Ombudsman with input from the organization and external experts. Greater validity when generated by others.
![Page 12: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/12.jpg)
Lexicon
Effectiveness1. a. Having an intended or expected effect.
b. Producing a strong impression or response; striking: an effective performance
2. Capable of producing a desired effect: an effective reprimand; an effectual complaint; an efficacious remedy. (dictionary.com)
For our purposes: The alignment among Ombudsman activities, these activities’ outcomes, and the intended effects desired by an informed and well‐meaning host organization. Generally determined by the organization in concert with the Ombudsman or other experts.
![Page 13: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/13.jpg)
Lexicon
Assessment1. The classification of someone or something with respect
to its worth [synonym: appraisal] 2. The market value set on assets 3. The act of judging or assessing a person or situation or
event [synonym: judgment] (dictionary.com)
For our purposes:The entire set of activities undertaken to understand an Office of the Ombudsman’s activities, outputs and the value of these, i.e. a set of sets. Likely done by others, with the Ombudsman’s support.
![Page 14: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/14.jpg)
“Value locator” Model
MEASUREMENT
EvaluationEffectiveness
![Page 15: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/15.jpg)
“Value locator” Model
MEASUREMENT
EvaluationEffectivenessValue?
![Page 16: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/16.jpg)
“Value locator” Model
MEASUREMENT
EvaluationEffectiveness
Value?
![Page 17: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/17.jpg)
“Value Locator” Model
MEASUREMENT
EvaluationEffectiveness
Value?
![Page 18: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/18.jpg)
This session is about:
• Using planning tools to assist Ombudsman offices to:– Create methodologies to evaluate operations and effectiveness
– Demonstrate value
![Page 19: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/19.jpg)
What we are going to do:
• evaluation planning, using the tools, reporting out
• Materials for participants:– slides, evaluation guide, logic model blanks
• Bovine Scatology testing aka …….
![Page 20: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/20.jpg)
EVALUATION PLANNING FOR OMBUDSMAN OFFICES
![Page 21: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/21.jpg)
ICANN AND ITS OMBUDSMAN
![Page 22: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/22.jpg)
About ICANN
• Internet Corporation for Assigned Names and Numbers
• Administers the Domain Name System DNS
• .com. Net. .org etc (Top Level domains)
• .mobi asia .travel etc (sponsored TLD)
• .us .ca .uk etc
• New gTld expanison
![Page 23: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/23.jpg)
More ICANN Info
• $40 million budget based on domain names
• Around 100 staff and a team of consultants
• Head office in Marina del Rey
• Office in Brussels, Washington, and Sydney
• Staff in Canada, Mexico, Italy, Egypt, Bulgaria, Niger, Netherlands, France, Australia
• Staff represent about 20 countries and 29 languages
![Page 24: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/24.jpg)
ICANN’s Office of the Ombudsman
• Product of a reform movement
• First Ombudsman appointed Nov 1, 2004
• Executive Ombudsman scheme– In the organization but complaints from the outside
– Investigates
– Reports
– Recommends
![Page 25: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/25.jpg)
Purpose of the Ombudsman
• …ICANN is working towards a stable, secure, and universal internet.
• In working towards that goal, the Office of the Ombudsman will assist ICANN by:– Ensuring that members of the community receive fair and equitable treatment;
– Helping to raise service and administrative standards;
![Page 26: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/26.jpg)
Purpose of the Ombudsman cont.
• …ICANN is working towards a stable, secure, and universal internet.
• In working towards that goal, the Office of the Ombudsman will assist ICANN by:– Improving communication links between ICANN and the community; and,
– Issuing reports, and maintaining a website that informs consumers, and the ICANN community about issues of interest.
![Page 27: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/27.jpg)
SOME EVALUATION BASICS
![Page 28: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/28.jpg)
Why evaluate; or, Tell a Story That
• Demonstrates value
![Page 29: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/29.jpg)
Tell a Story That
• Demonstrates Value
• Shows your program is working
![Page 30: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/30.jpg)
Tell a Story That
• Demonstrates Value
• Shows your program is working
• Honestly talks about what can be improved
![Page 31: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/31.jpg)
Tell a Story That
• Demonstrates Value
• Shows your program is working
• Honestly talks about what can be improved
• Identifies the successes you can celebrate
![Page 32: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/32.jpg)
Tell a Story That
• Demonstrates Value
• Shows your program is working
• Honestly talks about what can be improved
• Identifies the successes you can celebrate
• Illustrates professionally linked and service standards
![Page 33: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/33.jpg)
Tell a Story That
• Demonstrates Value
• Shows your program is working
• Honestly talks about what can be improved
• Identifies the successes you can celebrate
• Illustrates profession linked and service standards
• Exemplifies integrity
![Page 34: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/34.jpg)
Other reasons for evaluation
• Governing body– Retention
– Bonus
– Budgets – staffing ‐ support
![Page 35: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/35.jpg)
Three basic evaluation time frames
• Ongoing– You want to monitor through the evaluation cycle
• Formative– 2 – 3 years into operations or evaluation cycle
• Summative– 5 years into operations or evaluation cycle
![Page 36: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/36.jpg)
VALUE STATEMENTS
![Page 37: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/37.jpg)
The Values of this Office are:
• ‐Respect for Diversity;
• ‐Excellence in Ombudsmanship;
• ‐Professionalism;
• ‐Confidentiality;
• ‐Impartiality;
• ‐and Independence
• Look at back inside cover of handout
![Page 38: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/38.jpg)
Your Turn
• Small team work
• What are the values of your office?– Two values, and explanation
![Page 39: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/39.jpg)
OMBUDSMAN OFFICE PROFILE
![Page 40: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/40.jpg)
Ombudsman Office Profile
• Origin and rationale
![Page 41: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/41.jpg)
Ombudsman Office Profile
• Origin and rationale
• Mandate and Objective
![Page 42: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/42.jpg)
Ombudsman Office Profile
• Origin and rationale
• Mandate and Objective
• Governance structure
![Page 43: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/43.jpg)
Ombudsman Office Profile
• Origin and rationale
• Mandate and Objective
• Governance structure
• Clients
![Page 44: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/44.jpg)
Ombudsman Office Profile
• Origin and rationale
• Mandate and Objective
• Governance structure
• Clients
• Delivery approach
![Page 45: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/45.jpg)
Ombudsman Office Profile
• Origin and rationale
• Mandate and Objective
• Governance structure
• Clients
• Delivery approach
• Outcome and results
![Page 46: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/46.jpg)
Your turn….
• Working in teams of four to six– Discuss your office profile
– Make notes
– Report back to group
![Page 47: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/47.jpg)
COMPONENTS OF AN EVALUATION PLAN
![Page 48: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/48.jpg)
THE LOGIC MODEL
![Page 49: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/49.jpg)
![Page 50: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/50.jpg)
Your turn….
• Complete the logic model for your office
![Page 51: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/51.jpg)
![Page 52: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/52.jpg)
EVALUATION STRATEGY
![Page 53: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/53.jpg)
Start with Overview Questions
Relevance– Is there a continued need for the Office of the Ombudsman?
– Is the Office of the Ombudsman relevant for the continued success of the entity?
![Page 54: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/54.jpg)
Start with Overview Questions
Management Support– Are the resources given to the Office of the Ombudsman sufficient to its mandate?
![Page 55: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/55.jpg)
Program Delivery– Has the Office of the Ombudsman been able to discharge the mandate?
• Relationships?• Backlog?• Timely, effective results?• Change in the environment?
– Very much dependent on your office type and mandate
Start with Overview Questions
![Page 56: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/56.jpg)
Cost Effectiveness of Operations– Does your program conduct its work as a productive and efficient part of the organization?
– Are there alternatives to your Ombudsman’s Office?
Start with Overview Questions
![Page 57: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/57.jpg)
Then Add Evaluation Criteria..
• # of people using Ombudsman services (call centre, website, complaint form etc)
• # and type of complaints• # and type of resolutions• # and type of media mentions that focus on the Office of the Ombudsman
• % of complainants/community who agree there is a need for the Ombudsman
• % of complainants/community who know of an alternative to resolving a complaint if Ombudsman did not exist
![Page 58: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/58.jpg)
Your Turn…
• Management
• Relevance
• Program Delivery
• Cost Effectiveness
• In your groups develop two questions and criteria for each as it relates to your office
![Page 59: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/59.jpg)
Tools for evaluation
• Self evaluation
• Application of Standards (ISO)
• Statistical comparisons – longitudinal surveys
• Client Surveys
• Value statement
• Mission statement
![Page 60: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/60.jpg)
REPORTING STRATEGIES
![Page 61: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/61.jpg)
Telling your story
• Annual reports
• Blogs
• Case Studies
• Out reporting
• Story telling
• Media
![Page 62: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/62.jpg)
Your turn
• In your groups discuss how you can let your community, stakeholders, and governing body know about your strategic goal setting, and your evaluation activities.
– What might work, what might not
![Page 63: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/63.jpg)
Evaluation planning
• Is the first step in the process
• Implement the plan
• Talk about the results
![Page 64: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/64.jpg)
A VALUE MODEL
![Page 65: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/65.jpg)
A value Model
• Guest speaker presentation
• John Zinsser – presentation by long distance
![Page 66: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/66.jpg)
Your Value in Your Organization
Four perspectives on value– Intended/Desired (designers)– Expected (users & potential users, partners & collaterals, other stakeholders)
– Appreciated (recognized)– Unappreciated (missed)
![Page 67: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/67.jpg)
Sources of Value
• Economic– Time– Productivity– Workforce – Hard assets– Risk Management– Reputation– Culture
– Others?
• Humanistic– Morale– Trust– Creativity– Risk Taking– Reputation– “Of Choice”– Culture
– Others?
![Page 68: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/68.jpg)
A Two‐part Value Model
• "Cost-Effectiveness of Ombudsman Offices"Anthony Perneski & Mary Rowe
• “Formality Avoidance as Cost Effectiveness”John Zinsser
![Page 69: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/69.jpg)
A Value Example
•
Productivity $1,197,351 Management Time $1,030,786Personnel $1,524,798Legal $0Miscellaneous $200,000
Sub $3,952,935Formal Costs Avoided $3,892,890
Total $7,845,825
Acme’s Ombudsman Generated Savings for One Year
![Page 70: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/70.jpg)
Cost Effectiveness from Value
Acme’s Ombudsman returned $7,845,825or more than $9.50 for every $1 invested.
– This return is in addition to an appreciated but unknowable value of reduced litigation costs.
– This return does not include many additional and especially humanistic benefits.
– This return is EXTREMELY conservative.
![Page 71: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/71.jpg)
Productivity
• Multiply number of resolved cases by the productivity percentage increase (.02). Then, multiply this result by the average loaded salary of all employees.
• Alternatively multiply productivity increase by the number of parties to case by the daily loaded salary by duration of issue.
![Page 72: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/72.jpg)
Management Time
• Multiply the number of cases involving management in the issues by the daily management average loaded salary. Then, multiply the result by the number of days of the case.
• Multiply result by .1.
![Page 73: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/73.jpg)
Personnel
• Retained employees = number of satisfaction survey results stating “would have left.”
• Multiply number of retained “desired” employees by 1.7 times the average loaded salary.
• Multiply number of retained “highly desired” employees by 2.4 times the average loaded salary.
![Page 74: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/74.jpg)
Formal Costs Avoided
• Determine the average cost for each formal process.– Include process and outcome costs.
• Identify types of cases with multiple cases going to formal levels.
• Count the number of cases “of type” managed to close by the ombudsman. (not going to formal levels).
• Multiply the number of cases not going forward of type by the average process cost.
– Weight increase for serious cases.
![Page 75: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/75.jpg)
Additional Value Contributions
• Ombudsman Handles Issues No One Else Does– Less likely of the issue being “stranded.”
• Ombudsman Enhances Other Programs– More “just‐right” issues navigated to the best resource equates to more efficient programs & resolutions
• Ombudsman Adds Educational Value– Coaching and processes used— interest‐based negotiation, active listening, etc., teaches users better communication, conflict management techniques, and interpersonal skills.
![Page 76: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/76.jpg)
To Estimate Value You Must Know
Activities– What you do– Who you do it for
(level, loaded cost, cost to replace)
– How many involved– How long it takes
(duration and time in step/activity)
![Page 77: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/77.jpg)
To Estimate Value You Must Know
Outcomes– Individual impacts
(retention, envaluement,etc.)– Case impact
(formality averted, individual & other changes) – Collateral impacts
(others ‐ individual/program ‐ involved, impacted or not)
– Environmental or System impacts (institutional changes, reputation enhancement protection, etc.)
![Page 78: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/78.jpg)
EVALUATION TOOLS
![Page 79: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/79.jpg)
Self Assessment 50 Questions
• There is little research in the literature• Three articles and one dissertation
– Danet 1978, next significant work in – Ayeni 1990, – Male 1999 and– Hertogh 2000
• General approaches: files closed, per capita work, benchmarking, overview and qualitative
![Page 80: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/80.jpg)
Whither Criteria???
• Three key documents– Creating the Office of the Ombudsman
• (Rick Russell)
– Essential Characteristics of an Ombudsman • (USOA, Gottehrer et al)
– ABA Ombudsman Standards
![Page 81: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/81.jpg)
What to do with the Criteria
• Compare the criteria to your Office• Apply
– Logic– Fair and reasonable standards and comparisons
• Use your – knowledge base– Documentation, annual reports, charter– Survey findings
![Page 82: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/82.jpg)
The List
• Handout listing the criteria used
• Bear in mind that The List is not limited to one of organizational, executive, or classical Ombuds. It is a generic list, and some criteria may not apply to you, or some criteria may be repetitive.
• Other sources may also speak to your office.
![Page 83: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/83.jpg)
50 Questions ‐ Context
• When one endeavors to determine what any ombuds program, of any type, does or achieves, the absence of established measurement, evaluation, and assessment criteria immediately becomes apparent.
• Only recently has the subject received meaningful attention. Much more remains necessary.
![Page 84: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/84.jpg)
50 Questions ‐ Limitations
• Not about program performance
• Does not quantify Office activities/Outcomes (measurement).
• Does not examine measures to generate a value proposition (evaluation).
• Does not pursue whether ICANN achieves, by creating the Office, any intended outcome (effectiveness).
![Page 85: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/85.jpg)
Your turn
• Each participant has completed the online survey and answered the 50 questions
• Let’s spend an hour going through this exercise.– What did you learn?
– What stood out for you?
– What was the single best question?
– How do the 50 questions reflect the 6 reasons to evaluate?
![Page 86: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/86.jpg)
ISO Standards
• ISO 10002– design and implementation of an effective and efficient complains‐handling process for all types of commercial or non‐commercial activities, including those related to electronic commerce.
![Page 87: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/87.jpg)
ISO Standards
• ISO 10003– provides guidance for the handling of complaints when they are not resolved internally
![Page 88: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/88.jpg)
ISO Standards
The organization should plan and design an effective and efficient complaints‐handling process in order to increase customer loyalty and satisfaction, and also to improve the quality of the products provided. This process should comprise a set of interrelated activities that function harmoniously and use various personnel, information, material, financial and infrastructure resources to conform to the complaints‐handling policy and achieve the objectives. The organization should take into account the best practices of other organizations with regard to complaints handling.
![Page 89: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/89.jpg)
Statistical Comparisons
• A useful tool
• Comparisons based on:– Raw number of files
– Files per staff member
– Cost per file
– Cost per file per staff member
![Page 90: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/90.jpg)
Your turn
• Who would you compare yourself to?
• What would the points of comparison be?
• Who would be interested in knowing what you found out and why?
![Page 91: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/91.jpg)
System User Surveys
• Again, the survey should start with the reason that you want to evaluate:– Demonstrating value
– Competence
– Improvements
– Successes
– Service standards
– Integrity
![Page 92: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/92.jpg)
System User Surveys
• Think about your responses to your pre‐course survey and what you learned
![Page 93: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/93.jpg)
Your Turn
• Xref pages 19 – 21 in the guide
• working in teams
• Go to http://www.surveymonkey.com/Default.aspx
• Login as [email protected] ‐ ombuddy
• Click on create survey
• Create 5 questions
![Page 94: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/94.jpg)
Satisfaction based on Jurisdiction and Outcome
0 .0 0
1.0 0
2 .0 0
3 .0 0
4 .0 0
5 .0 0
6 .0 0
7 .0 0
8 .0 0
9 .0 0
High 8.00 7.50 6.79 5.20 4.50
Low 6.33 4.75 4.13 2.75 1.50
High Jurisdiction - High Outcome
High Jurisdiction - Moderate High
OutcomeOmnibus
Low Jurisdiction - Moderate Low
Outcome
Low Jurisdiction - Low Outcome
![Page 95: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/95.jpg)
![Page 96: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/96.jpg)
![Page 97: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/97.jpg)
The Third Party Review
• Fact‐Check/Verify
• Commentary
• A neutral, independent voice about Ombuds functions– For Leadership
– For Users
– For the Ombuds
![Page 98: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/98.jpg)
The Third Party Review
• Simple Process– Review the contributing material/report– Interview (if needed)– Examine performance claims via data
• Simple Report– Repetition– Augmentation of voice
![Page 99: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/99.jpg)
The Third Party Review
• Adds Another Iteration/A Hand• Enhances Veracity• Relieves Self‐promotional Challenge
![Page 100: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/100.jpg)
Third Party reviewers?
• Peers
• Learners
• Consultants
• Retired Ombudsman
• Others?
![Page 101: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/101.jpg)
Your turn
• What part of the evaluation process would you be able to use a third party reviewer with?
• Why is that a critical part of the process that needs the third party validity?
• Who would you consider as your reviewer?
![Page 102: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/102.jpg)
![Page 103: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/103.jpg)
So, now what???
• Who do you give copies/summaries to?– Government
– Executive Management
– Staff
– Stakeholders ‐ Critics
– Public postings (Annual Report – website)
– Other Ombudsmen
– Other
![Page 104: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/104.jpg)
How do you…..??
• Implement improvements you identify• Fix problems• Celebrate your successes• Communicate:
your contribution and program integrity
• Deal with service standards
![Page 105: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/105.jpg)
Your turn
• Reflect on your thoughts yesterday to the four key evaluation criteria– Relevance
– Management Support
– Program delivery
– And cost effectiveness
– Based on everything we have done, prepare a summary paragraph on each
![Page 106: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/106.jpg)
Did We:
• The Gain a greater understanding of Ombudsmanship and systems design.
• To discuss effective performance measurements How to effectively survey our client base (especially complainants ‐when their expectations differ from our legislative authority) How to interpret survey results
• In conjunction with a course that I am currently enrolled in, I would enjoy learning more about what an Ombuds is and does, as well as learning about the global market for services of this kind.
![Page 107: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/107.jpg)
Did we:
• The Ombudsman's work is unique in nature. The Ombudsman is possessed with certain characteristics above all those under his/her jurisdiction. But at the same time, the Ombudsman has to balance this in terms of effectiveness, productivity and efficiency etc. I expect from the assessment course to give some answers on how one can assess and measure the Ombudsman's work. Whether it is useful to have one in the absence of any assessment tools? Should the office remain complaint driven? As a public office, accountability of the office is necessary.
![Page 108: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/108.jpg)
Did we:
• A primer on the roles and responsibilities of Ombudsmen. Networking and possible employment / consulting opportunities. Ability to develop new means for evaluating the effectiveness of systems.
• that the time together will provide groundwork for the future
• To obtain more practical knowledge. Learn from experienced professionals in the filed of dispute system design.
• Learn best practices for assessing the effectiveness of Ombudsman's Office.?
![Page 109: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/109.jpg)
Did you learn:
• How to best assess a dispute resolution system
• to effectively survey complainants
• To upskill myself and impart the knowledge gained to others in the office
• How to complete my work for your company in a satisfactorymanner
• Assessing a system with information that is often subjective, personal and often confidential.
• metrics that others use successfully
• Theory transforms into practice
• Providing upward feedback?
![Page 110: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/110.jpg)
Questions?
I’d appreciate your feedback.
‐Tools generally
‐The Guide
‐Satisfaction gap theory
![Page 112: Ombudsman Operations: Evaluation Planning and Basics · • # of people using Ombudsman services (call centre, website, complaint form etc) • # and type of complaints • # and](https://reader034.vdocuments.us/reader034/viewer/2022042101/5e7e19c13bf39b5134340636/html5/thumbnails/112.jpg)
Contact Us
Frank Fowlie
•310‐301‐5826
•www.icannombudsman.org
John Zinsser
•240‐401‐8742