Monitoring and Evaluation

The community score card process is a community-based monitoring tool that is a hybrid of the techniques of social audits and citizen report cards.The CSC is an instrument to exact social and public accountability and responsiveness from service providers. By linking service providers to the community, citizens are empowered to provide immediate feedback to service providers. [from author]

Improving service delivery for the poor is an important way to help the poor lift themselves out of poverty. This resource presents and evaluates tools and techniques to measure service delivery and increase quality in health and education.

Family Medicine training and assessment is becoming more formalized and developed in South Africa. Assessment of competency in relation to clinical skills can involve observation in the clinical setting, but is more usually assessed in an examination. Summative assessment of family physician’s clinical skills now usually includes an Objective Structured Clinical Examination (OSCE). Standardisation of the OSCE is required to define the pass mark above which a candidate performs at the level expected of a family physician. [from abstract]

This case study describes how five Integrated Management of Childhood Illness (IMCI) trainers and supervisors conducted an assessment of provider knowledge and skill to carry out IMCI at 38 facilities in two districts in Kenya. [author’s description]

The Quality Assurance Project investigated how job aids could increase compliance with guidelines for the Integrated Management of Childhood Illness (IMCI) in cooperation with the Zambia Central Board of Health beginning in 1999. One of the first countries to introduce IMCI, Zambia had a large number of IMCI-trained providers, and several IMCI job aids were already in use: a chartbook, recording form, poster, and mother card. The study proceeded in three stages: an initial assessment of job aid usage in 1999, the design and introduction of a new IMCI job aid (called “new outpatient department book” or “NOPD book”) that also served as the patient record, and a second assessment of job aid usage in 2000 after introduction of the NOPD book.

This presentation was part of the International Conference on Global Health session, “Integration and Application: Successes and Challenges in Health-Worker Training.” It talks about the need for supervision, the supervisory problems at the Health Center, the approaches taken to address the problem, improving supervisor skills, and monitoring.

As the complexity of the challenges facing the public health workforce has increased, many have argued that insufficient resources have been devoted to the preparation of the workforce, including its leaders. Here we describe the growth of national advocacy for public health leadership and workforce development. We discuss the creation of the National Public Health Leadership Development Network (NLN), a consortium of institutes providing a system for leadership development, and we review the network’s creation of the Leadership Competency Framework for core curriculum design and development of performance standards for public health practice.

This study examines the relationship between IMCI compliance and three cost issues: the length of client-provider consultations, treatment drugs, and workload. The setting was 26 health clinics in Niger, the only developing country setting where QA was implemented before IMCI. [publisher’s description]

The Population Council and The Pubcomm Group, Inc. have developed simple, inexpensive, user-friendly computerized job aids to assist supervisors in improving the quality of family planning, maternity care, and postabortion care services. The software is free to download. [publisher’s description]

This cost analysis is based on QAP research on the effectiveness of two interventions (self-assessment and peer review) in sustaining or increasing the effectiveness of interpersonal communications training that midwives had taken. The research had measured the effectiveness of the interventions in terms of the number of utterances midwives made during family planning consultations, and this case study followed on, measuring the cost of each intervention in terms of the number of utterances generated. Activities/tools: Sample provider self-assessment form, sources of costs, evaluation of marginal benefit.

Nurses provide the bulk of primary care services in South Africa. Post-apartheid health legislation envisions the provision of comprehensive primary services at all public clinics, which implies the need for a cadre of primary care nurses able to render such services. The objective of this study was to identify core competencies of clinic nurses and develop an evaluation tool for primary care nursing in South Africa. [from abstract]

Problems related to health worker motivation are remarkably pervasive, but to-date little attention has been paid to them in developing and transition countries. Basic tools to measure the determinants and consequences of motivation have not been adapted to contexts outside the industrialized world. This paper assesses the feasibility of transferring psychometric tools, typically used in industrialized countries to measure motivational processes, to other contexts. The paper draws upon two field studies conducted in two hospitals in the Republic of Georgia and two hospitals in the Hashemite Kingdom of Jordan.

In both Senegal and Mali, JHPIEGO used a “jumpstart” training approach that combines both basic and intermediate training to ensure that participants are not only competent but are also confident in the skills or procedures acquired during training. High caseloads are needed so that trainers can have enough practice to achieve both competency and confidence in method provision skills—insertion and removal. The Klinik Raden Saleh (KRS), in Indonesia, was selected as the jumpstart training center for the Senegalese and Malian clinical teams because of its large Norplant implants clinical caseload, extensive clinical training, programmatic and research experience, large number of master trainers and clinical coaches, and extensive use of midwives as trainers and service providers. [publisher’s description]

The main objective of this study was to determine the perceptions of Iranian nurse educators and students regarding the evaluation of teaching effectiveness in university-based programs. [from abstract]

This paper evaluates quality of nursing care in the context of an evaluation of the practice of contracting out district hospital services in South Africa. [from abstract]

This study seeks to describe the existing systems for measuring and monitoring staff performance in the clinical setting and covered public and para-statal hospitals in Ghana. [author’s description]

This document evaluates the Support and Treatment for Antiretroviral Therapy project (START) program, a joint initiative of Family Health International (FHI) and the Government of Ghana, to integrate antiretroviral therapy into comprehensive care for people living with HIV/AIDS in Ghana. START helped establish voluntary counseling and testing centers, prevention of mother-to-child transmission activities and clinical care services. Key components of the START program include home-based care (HBC), referral networks and linkages to such existing services as spiritual and social support, and support for orphans and other vulnerable children (OVC).

The focus of this guide is the measurement of capacity for the purpose of monitoring and evaluating capacity-building interventions. It responds to a demand among public health planners, evaluators, and practitioners for advice on assessing the many aspects of health programming that fall under the rubric of capacity building. [author’s description]

This rapid-assessment guide is designed to help users arrive at a global overview of a country’s HRH situation. The guide is designed to help users assess current HRH constraints and challenges to “scaling up” health interventions. HRH main issues include: Policy, regulation and planning; Management and performance improvement; Labour market; Education, training and research; HRH and priority health programmes; and Monitoring and evaluation. [author’s description]

Using all of the tools in the document will provide a comprehensive assessment of skills and service delivery. Each tool, however, may be used separately or combined with others to create a document appropriate for the content of a specific maternal and newborn health training course. [author’s description]

In the context of the critical role that human resource play in delivering health services to populations and as human resources represent a critical constraint in achieving the targets of the Millennium Development Goals, it becomes imperative that effective systems for regulation, education, research and performance management be strengthened in order to achieve the required improvement in health outcomes. The guidelines will provide a framework to improve health programs and the services for delivering them, and to guide in allocation of human and financial resources in current and futures programs and services.

This manual describes the purpose and processes for developing and using a system of Human Resources for Health (HRH) performance indicators in order to monitor the activities of a public sector health service. It is particularly aimed at enhancing the management process in developing countries and draws on experiences of pilot work. [from purpose]

This document intends to provide a policy and strategic framework for management of health information, use of information in planning and management of health services and monitoring health sector performance. [from preface]

This easy-to-use web-based tool compiles and analyzes country data from multiple sources, provides charting options, and generates automated country fact sheets, helping users to assess the performance of the country’s health systems.

Human resources (HR) constraints have been reported as one of the main barriers to achieving the 2005 global tuberculosis (TB) control targets in 18 of the 22 TB high-burden countries (HBCs); consequently we try to assess the current HR available for TB control in HBCs. A standard questionnaire designed to collect information on staff numbers, skills, training activities and current staff shortages at different health service levels was sent to national TB control programme managers in all HBCs. [From abstract]

This document, published by the Central Board of Health of Zambia, describes the implementation of the country’s health management information system (HMIS) during 1995 to 2003, and considers the system’s achievements and the challenges facing it. The new HMIS was a response to weaknesses of the previous system, which was highly fragmented but with centralised data management. The aim was to establish a self sustaining monitoring and evaluation system that would improve decision making at all levels of the health care system with timely, valid and appropriate information. [publisher’s description]

This field guide has been designed to be used by [those] who are considering using community mobilization to improve health at the individual, family, and community level.The guide guide contains illustrative examples and lessons learned in community mobilization experiences from around the world, focusing on working with disadvantaged or marginalized groups in developing countries. The characteristics of each country and each region are very different, so the field guide does not recommend a unique strategy for community mobilization but offers instead general principles and methods which will need to be adapted to local settings and conditions. [author’s description]

This audit tool has been developed to help HR functions and local HR communities in planning and organizing HR services locally. It has been designed to be used flexibly. At one end of the spectrum the audit tool can be used as a simple self-analysis and, at the other end, it can facilitate a full and rigorous assessment of a function’s capacity and capabilities. [from foreword]

This article assesses interventions aimed at improving family planning mechanisms and reviewing the problem-solving processes to build an effective monitoring system of the interventions at the local level of the overall system of the Ministry of Health and Family Welfare of the Government of Bangladesh. [adapted from author]

Research on the Integrated Management of Childhood Illness (IMCI) shows that it is a scientifically sound way to treat sick children, but ways to ensure that it is implemented properly are lacking. This 1997-98 study examined and compared three implementation approaches: structured feedback of performance data, structured feedback of performance data where quality improvement (QI) teams were in place, and the formal World Health Organization training in districts with QI teams. The report details the impact of each intervention, noting that sustaining performance is problematical under any intervention. The cost of IIMCI training was four times that of performance feedback. [publisher’s description]