USING THE METHODOLOGY FOR EXTERNAL LEGAL EDUCATION QUALITY ASSESSMENTTraining on behalf of USAID FAIR Justice project – 27th and 28th May 2015

Participants will be able to…

Explain the basic theory behind legal education quality assessments

Utilize the relevant basic research methods Conduct an external legal education quality

assessment using the methodology Devise an internal legal education quality

assessment for their own institution Present the results of a legal education

quality assessment accurately and effectively

What we mean by “Quality”… Assurance? Control? Enhancement? Assessment? Audit? Standards? Culture? Accreditation?

Two different historical models of quality assurance…

French External control, under religious authority Focused on accountability More “extrinsic” idea of education

English Self governing by society of fellows Focused on peer review “Intrinsic” idea of education

A general model of higher education quality assurance…

An independent “meta-level” managing agent for the quality assurance system

System based on “self-evaluation” External assessment/peer-review Reporting focused on helping

institutions improve

Why externally assess legal education quality?

Standards For safeguarding national academic standards For accreditation purposes

Market reasons For public provision of independently-verified information

(quantitative and qualitative) about programmes or institutions

To allow for comparison in the law school market Internal self assessment

To encourage reflection and internal self-assessment To contribute to institutional improvement and strategic

planning To give credit for good work

Aspects of quality…

(1) quality as exceptional (e.g. high standards);

(2) quality as consistency (e.g. zero defects);

(3) quality as fitness for purpose (fitting customer specifications);

(4) quality as value for money, (as efficiency and effectiveness); and

(5) quality as transformative (an ongoing process that includes empowerment and enhancement).

Understanding the Methodology

Based on the European standards, guidelines and recommendations

This is a methodology for external assessment, but also useful for institutions devising internal assessments

It should continue to be developed and improved for Ukrainian context

Participatory in nature

Risks

Institution not fully cognizant of methodology

Institution sees assessment as a threat instead of an opportunity

Institution is not open to genuine full assessment

Institution seeks to offer “window-dressing”

Elements and Criteria of Assessment

Elements Basic categories around which the

assessment is structured

Criteria Expressed as outcomes under each

element

Elements of the LEQ Assessment

1: Policy and Procedures for Internal Quality Assurance: 

2: Approval, monitoring and periodic review of programs and awards:

3: Admission and Assessment of students 4: Quality assurance of Teaching Staff 5: Curriculum, teaching methodologies,

learning resources and student Support  6: Administration and Information Systems  7: Public information

Purpose of research…

To get a full, accurate “3D” picture of the institution, its strengths, weaknesses From multiple perspectives Using multiple sources

Respondent groups

Administrators Teachers Students Alumni Employers

Research Methods

Methodology specifies various methods to gather quantitative and qualitative information about the effectiveness of the institution Document analysis Surveys Key Informant interviews Focus group discussions Classroom observation

Document analysis

Important for showing “formal” reality Wide variety Undertaken prior to visit Informs questions

Key informant interviews

Semi-structured format, with one (max two) people Establish context for further research Advantages

Good for establishing basic facts People may be more honest one on one

Limitations Time consuming. Might succumb to social desirability effect, answer in

ways they think they interviewer wants them to answer, which can affect whether the information gathered is valid.

Limited number of people you can interview, so they may not be generalizable.

Interview best practices…

Devise protocol of relevant core standard questions for each respondent group 

Avoid asking yes/no questions. Use caution when asking “Why”

questions… Focus on “What and “How” questions

Focus Groups

Normally have between seven and ten participants Why are they useful?

participants insights tend to “trigger” the sharing of others’ personal experiences

Provides diverse opinions and ideas. What are some limitations of focus groups?

Susceptible to facilitator bias, which can undermine the validity and reliability of findings.

Can be dominated by a few vocal individuals. Limited generalizability to a whole population.

Focus group best practices

Interviewer as moderator Take place following interviews Use to address “What could we do”

questions Areas of inquiry/development decided in

advance Open, informal atmosphere Establish rules at beginning of group Establish rapport

Traditional teaching methods

Teacher as information giver Teacher-centered ; students passive Emphasizes “what” questions

Memory is vital Leads to conformity, students learn pre-

determined outcomes.

Modern teaching methods

Teacher as facilitator Active learning, student-centered Stresses “Why” and “how” questions

Encourages investigatory and explanatory work

Allows students a degree of freedom an creativity

Uses variety of methods

END OF DAY ONE