United Kingdom of Great Britain and Northern Ireland: Consultant - QLE Monitoring Framework Review

Terms of Reference – Save the Children’s Quality Learning Environment Monitoring Framework

End of Strategy Review

Background

Measuring and promoting evidence-based approaches to improve programme quality is an on-going priority in various sectors, especially where measuring change has traditionally been challenging. This includes measuring the extent to which educational learning environments are “of quality,” often an end to itself given the global agenda on quality and inclusive education (reiterated at the World Education Forum). Previous work in sectors such as School Health and Nutrition and Child Protection and the recent 2015 Education for All Global Monitoring Report provide numerous examples of the direct and indirect contribution of learning environments to improved learning outcomes.

Save the Children’s Quality Learning Environment (QLE) monitoring framework can be used to inform a broad and multi-dimensional assessment of the learning environment (i.e. pre-schools, schools and other learning contexts) and its ability to meet standards in promoting psychosocial and physical well-being of learners and staff, active teaching-learning methods and engagement with the community. Data collection tools derived from the framework are employed through a mixed methods approach, including interviews and focus group discussions with students, teachers, parents and community members, as well as classroom and school observations of teaching and learning activities and the presence (or absence) of teaching and learning materials and physical infrastructure.

The QLE monitoring framework is a core component of Save the Children’s 2016-18 education strategy, serving as an indicator to measure the quality of Save the Children’s education programs at the sub-national, national, regional and global levels. The QLE monitoring framework has been used in over 35 country programs, both for Basic Education and ECCD from 2012 to the present. The QLE frameworks are primarily used by country offices for education program design and planning and program implementation, and increasingly as a guide for advocacy and policy initiatives. QLE data are reported to donors, to the SCI CEO and Board; equally important is sharing this data back with (and holding Save the Children accountable to) children, schools, communities and governments Save the Children supports. It is aligned with Save the Children’s MEAL approach, wherein monitoring and evaluation are strongly linked with learning and accountability throughout the project cycle. As such this framework and resultant data are increasingly relied upon as critical program management tools within countries, and as a check on program quality at regional and global levels.

The QLE review will consist of two phases. The outputs of this consultancy represent phase 1. Findings and recommendations will form the basis of discussions amongst Save the Children’s Education Global Initiative (EdGI) on the way forward for the QLE framework in our next strategy period, taking into account Save the Children’s investments to date in QLE. Phase 2 will consist of implementing the agreed upon recommendations.

Purpose of the QLE Review

To review and assess the utilization, design/methodology, and application of the Basic Education and ECCD QLE monitoring frameworks by country programmes between 2012-2015, to inform subsequent revision and updating of QLE framework, methodology and associated tools.

Approaches/Criteria for Assessment of the QLE Framework

Specific objectives and the activities that will address these objectives can be categorized as follows:

Objective of QLE Review

Activity

History and Utilization: How has the QLE framework been used in the sampled countries over the last four years

Map, review and assess SC country office application and use of QLE framework against criteria/approaches listed below.

Must include history of the QLE framework, including the evolution of the tools and adaptations that have taken place to changing contexts and priorities.

Design: Given the current design and methodology of the QLE framework, what is the framework most suited for E.g. program planning and monitoring.

Review and assess the QLE framework, methodology and associated tools against a set of criteria/approaches (see below) and determine strengths and weaknesses of the framework and associated materials when considered against different measures of learning environments and means of measurement.

Include focus on if/how programming and policy practices are adapted based on QLE data analysis and recommendations to improve use of data throughout program cycle.

Application: What does Save the Children want to be the primary use(s) of the QLE in the future (e.g. establish links to learning outcomes) Given the possible additional applications of the QLE framework, what changes are needed

Provide recommendations informing revision and updating of QLE framework, methodology and associated tools.

The consultant is expected to assess the QLE framework against several criteria:

1 SPICED methodology: subjective, participatory, interpreted and communicable (learning), cross-checked and compared, empowering (accountability) and diverse and disaggregated. Includes non-psychometric reliability.

2 Efficiency and Effectiveness of QLE framework: have the processes been adopted for data collection, entry and analysis been cost and time effective What alternatives should be considered How effective have these approaches been in achieving program outcomes (e.g. changes to the school environment, creating a participatory process for defining and assessing quality which promotes local ownership and a voice for multiple stakeholders)

3 Psychometric Reliability: Establish availability of data to assess psychometric properties of QLE (and appropriateness given applications of QLE framework)

1 Internal consistency/reliability of items within and across sub-domains

2 Inter-rater reliability of the observation based criteria in the QLE

3 Adequacy of the tools developed by country programs to collect data on the QLE items

1 Validity:

4 Content validity: MECE (mutually exclusive and collectively exhaustive) assessment of QLE framework. Comparison to other measures of the quality of learning environments, including indices or composite measures.

5 Criterion validity: Relationship of QLE framework to related learning outcomes

1 Sustainability: opportunities for cooperation and alignment of QLE framework with Ministries of Education (MoE) at national and sub-national levels; extent to which communities and schools can “own” the QLE (to enhance accountability, sustainability); understanding and utilization of QLE data by stakeholders such as schools and MoE.

2 Relevance of QLE Framework: does the QLE framework suit the priorities of Save the Children’s education interventions, in different contexts Note: QLE in humanitarian responses is out of scope for this review.

Methodology

• Review of relevant internal and external documents (to be provided by Save the Children).


• 
  

  Skype/email consultation with diverse sample of country programs:

  
  

  1 Sampling: Purposive stratified sampling based on a list of countries that have been grouped according to their use, success with or challenges with the QLE framework and level of investment by SC.
  
  2 Number of case study countries and methodology for collecting data to be agreed upon with consultant.

  
  


• 
  

  Skype/email consultation with SC Member education and M&E advisors (as well as School Health & Nutrition teams, Child Protection Global Initiative), consultants, fellows and external experts (as appropriate).

  
  


• 
  

  Interview list to be developed by the EGI M&E Work Group and reference group for this review.

  
  

Management of QLE Review

The overall QLE review process will be managed by the EGI M&E Advisor and an EGI Steering Group member, with the support of the EGI M&E Working Group. Active involvement by all EGI Work Groups is expected.

Time Frame (30-40 working days). Duty station: Flexible (desk work).

Task

Deadline

Recruit consultant

15 September, 2015

Inception Report (including proposed methods, prelim literature review)

01 October, 2015

1st Draft Report and presentation of results to date

25 October, 2015

Final Draft Report

05 November, 2015

Presentation final draft report via teleconference

09 November

Required Skills and Experience

• Advanced university degree (Master’s degree or equivalent) in the social sciences (e.g. international education or development), or related technical field.


• Extensive experience in application of quantitative and qualitative data collection and analysis methods, particularly its application in monitoring and evaluation of international education/development interventions.


• Knowledge of and use of participatory M&E methods. Previous experience constructing and analyzing indices, composite measures.


• Proven ability to consolidate diverse views into feasible, pragmatic recommendations.


• Excellent English-language written communications skills. Proficiency in French or Spanish is desirable.


• Strong organizational skills and ability to work to tight deadlines as part of a team.


• Demonstrable computer literacy and knowledge of analysis and database software (e.g. Excel, Access, SPSS).

To apply, potential candidates must submit:

1 Past evidence of similar work witin a CV

2 Provide a costing bid to carry out the tasks outlined herein.