Evaluating Niger's experience in strengthening supervision, improving availability of child survival drugs through cost recovery, and initiating training for Integrated Management of Childhood Illness (IMCI)

Background WHO and UNICEF have recently developed the "Integrated Management of Childhood Illness" (IMCI) as an efficient strategy to assist developing countries reduce childhood mortality. Early experience with IMCI implementation suggests that clinical training is essential but not sufficient for the success of the strategy. Attention needs to be given to strengthening health systems, such as supervision and drug supply. Results This paper presents results of evaluating an innovative approach for implementing IMCI in Niger. It starts with strengthening district level supervision and improving the availability of child survival drugs through cost recovery well before the beginning of IMCI clinical training. The evaluation documented the effectiveness of the initial IMCI clinical training and referral. Conclusions Strengthening supervision and assuring the availability of essential drugs need to precede the initiation of IMCI Clinical training. Longer term follow up is necessary to confirm the impact of the approach on IMCI preparation and implementation.


Background
In 1995, WHO and UNICEF initiated the Integrated Management of Childhood Illness (IMCI) to assist developing countries reduce childhood mortality caused by most childhood killer diseases: diarrhea, acute respiratory infection, malaria, measles and malnutrition. IMCI provides an efficient approach that is likely to have a higher impact in reducing childhood mortality compared with previous vertical disease-specific programs. The IMCI strategy has three components: improving health workers skills including training of first line health work-ers; systems support including improving availability of essential drugs, strengthening supervision and referral; and household and community [1].
To date, IMCI has been adopted by over 60 developing countries. Early experience with IMCI implementation suggests that many countries have put emphasis on the first component of the strategy without adequately strengthening health systems needed for supporting IMCI implementation. When health workers returned back to their health facilities after completing IMCI clin-ical training they were often faced with the realities of inadequate drug supply and weak and infrequent technical support through supervision [2]. This paper presents the results of the evaluation carried out by the authors of Niger's experience in introducing IMCI after applying interventions to strengthen supervision and improve the availability of child survival drugs through cost recovery system. This approach was introduced between 1997 and 1999 by a combined effort of two USAID funded projects: Basic Support for Institutionalizing Child Survival (BASICS) and the Quality Assurance. The merging of the two projects in 1997 allowed benefitting from quality assurance interventions to ensure the effectiveness and regularity of district supervision system and improve the availability of essential drug supply before the beginning of IMCI clinical training.

Methods
The approach was implemented in two districts: Konni district, in Tahoua Region, with eight health facilities, and in Boboye district, in Dosso Region, with five health facilities. In the beginning of 1997, quality assurance measures were introduced to strengthen supervision at the district level. Interventions included technical assistance to revive the district supervision system and develop a district level management team. The latter included a "coach" to help primary health care staff identify and solve problems using the local resources. District supervision teams received training in quality assurance including data collection and utilization, monitoring key health indicators, problem identification and solving techniques such as brain storming and matrix analysis for prioritizing identified problems and solutions [3]. The outcome of the training was a specific supervision plan covering all health facilities in both districts. In addition, the district supervision teams participated in the clinical IMCI training course. The project also introduced two tools to increase the effectiveness of supervisory visits to health facilities: 1) a Rapid Assessment Tool, including a check list focusing on child health services; and 2) a Supervisory Check List which is an integrated list including all aspects of maternal and child health services delivered by the health facility. In addition, limited financial support for transportation was provided by the project to support supervision.
To improve the availability of essential drug supply, the project provided technical assistance to support the Government of Niger to implement a cost recovery program. While partial cost recovery attempts have been tried in Boboye district since 1993, full cost recovery was introduced by the government in Konni and Boboye districts in late 1997 with the aim to fully replenish drug supply, cover the salary of one stock keeper per health facility, and purchase limited supplies. The project trained district level and health facility level stock keepers in Konni district in coordination with the European Union funded "Pharmaciens Sans Frontiers". It involved the community in drug supply management by selecting trusted members of the community as stock keepers. Service fees for cost recovery were decided at the level of health facility with community participation. They were based mainly on the cost of replenishing drug supplies. Fees for one curative visit for a child under five years old were CFA 350, (USA $1= CFA 500), an amount which is equivalent to half the average household daily income [4]. The availability of selected essential drugs was investigated before and after the implementation of cost recovery utilizing standard stock management indicators [5,6] The initial IMCI clinical training for health workers in Konni and Boboye districts took place in August and September, 1998 using the nationally adapted guidelines and training materials and the WHO 11-day training course. Evaluation of the effectiveness of the initial IMCI training included: 1) direct observation of health workers while examining a limited number of children under five years utilizing the IMCI guidelines and 2) a review of 269 IMCI forms filled in September and October, 1998 for children 2-60 months of age by trained health workers in six health facilities in the two districts.

Improving district-level supervision
For years before the intervention, no supervisory visits had been conducted from district level supervisors to the health facility level for any of the 13 health facilities in both districts. Beginning 1998, after the intervention, the district supervision teams in both Konni and Boboye districts carried out all scheduled supervisory visits on time. Each health facility in each district was visited four times a year. The supervisory visits were carried out utilizing the two supervisory tools. However, interviewed staff mentioned that the supervisory check list, while comprehensive, was too long. Supervisors explained to health facility staff and recorded their findings on a special health facility register for follow up and action. Problem solving techniques, particularly brain storming for prioritizing problems and matrix analyses for selecting feasible solutions were well understood and utilized in all health facilities in the two districts. The cost of conducting one supervisory visit was calculated to be $29, including per diem and transportation cost from district to the health facility and back.

Improving the availability of child survival drugs through cost recovery
The cost recovery system succeeded in eliminating the chronic state of lack of essential drugs that existed before the implementation of the system. Examination of the stock of selected child survival drugs in Konni and Boboye districts revealed that cost recovery has resulted in fewer stock-outs, shorter duration of stock-outs and higher average monthly consumption, an indication of the general availability of drugs. An exception has been observed in Boboye district where there was a four month long stock-out of cotrimoxazole after cost recovery. This was due to an overall shortage at the central store level. Stock keepers in Konni district, who received stock management training, were able to spare their district such a long stock-out by wisely keeping a safety stock (Table 1).
In spite the success of cost recovery in increasing the availability of essential drugs, there was an associated drop in the health facility utilization rate. Data from both Konni and Boboye districts revealed a drop in the number of out-patient visits beginning 1998, after the introduction of cost recovery. In the first half of 1998, new curative visits for children less than five years dropped by 33% and 22% of their 1997 levels in Konni and Boboye districts respectively. Such a drop in the number of health facility visits did not only affect curative visits but also preventive visits such as neonatal visits, where immunization is initiated, and antenatal visits, where tetanus toxoid is administered along with other antenatal services (Figure 1). At one health facility in Konni district, Badaguichiri, where data were available to monitor out patient visits for a longer period of time, the utiliza-tion rate started to show a strong recovery in the third quarter of 1998. The rise did not quite bring the utilization rate to the pre-cost recovery level, yet it brought it almost to the same level of another comparable health facility which had not yet introduced cost recovery ( Figure  2).

Effectiveness of the initial IMCI clinical training
An evaluation of the effectiveness of the initial IMCI clinical training revealed that trained health workers, in general, were capable of examining children using the adapted IMCI standards. However, 6% of the examined forms failed to indicate whether the visit was an initial or  follow-up visit. Furthermore, the nutrition section was not completed in 21% of forms. These findings suggest that health workers need to pay more attention to correctly filling out the IMCI forms and in assuring that the nutritional status of all children is checked and noted. The future monitoring of the same indicators can help in assessing the effectiveness of the supervision system to improve health workers' accuracy in completing IMCI forms and assessing the children's nutritional status (Table 2).
The IMCI standards for referral of very sick children and for follow up visits were largely ignored. Only six (23%) of children with a general danger sign were actually referred by health facility staff. Four of them were referred to a private hospital in Konni district (Galmi Hospital).
Out of a total of 182 follow-up visits required, only 11 (6%) actually returned to the health facility for follow up.
Of children who had their nutritional status checked (212), 21% and 34% suffered from low weight for age and anaemia respectively (Table 2). Health workers who were interviewed mentioned that finding adequate time to counsel care takers about the nutrition of their children is a major limitation.

Discussion
Niger's approach in applying quality assurance measures at district level succeeded in better preparing the target districts before the implementation of IMCI clinical guidelines at the health facility level. Improvements in supervision and availability of drug supply were documented. Exposing district supervisors to two types of training, quality assurance and clinical IMCI, was important in qualifying supervisors to support IMCI implementation. However, longer term follow up is necessary to confirm the long term impact of applying quality assurance measures to better prepare of IMCI implementation. In addition, the intervention included external financial support to revive district-level supervision. To sustain the achieved improvement, this cost needs to be transferred gradually to the district's health budget.
Since the project has provided financial support to revive district level supervision, to sustain the achieved success, it is important that the cost of conducting supervision be transferred gradually to the district's health budget.
Community involvement in cost recovery through participation in determining cost for service and in managing drug stock was key in assuring community's acceptance and trust in the system. Cost recovery was more effective when combined with stock management training. In Konni district, where cost recovery was combined with stock management training, results for improving availability of essential drugs were better than in Boboye, where such training was not conducted.
The observed drop in the number of outpatient visits induced by cost recovery is of particular importance, since the impact of IMCI is related to its ability to minimize missed opportunities of preventive and curative care at the outpatient level. Experience with cost recovery in other countries suggest that such a drop could be temporary [7]. The strong recovery of outpatient visits in Badaguichiri health facility during the third quarter of 1998 could be a sign for the beginning of the reverse in the outpatient trend. Other factors, such as the rainy season in June to October and the harvest around the third quarter, could have contributed to the fluctuation in the number of outpatient visits. Longer monitoring of the level of health facility utilization is needed to fully understand the effect of cost recovery on health service utilization.
The observed difficulty in complying with IMCI standards for referral and follow-up visits presents a challenge not only to health authorities in Niger but also to the IMCI standards themselves. The current IMCI referral and follow up standards may not be suitable for a country like Niger with such a widely dispersed population and only 30% of the population living within five kilometers of a health facility. Health facility staff mentioned that referral was not feasible because of the lack of transportation and the long distance between health facilities and the district level hospital. The distance between health facilities and the district hospital ranged from 25 -85 km. In addition, district level hospitals in both Kon- ni and Boboye were not well equipped to deal with very sick children which further discouraged both health workers at the health facility level and mothers to use them as a referral site. Referral sites need to be included in future IMCI programs.

Conclusions
In Niger, quality assurance measures to strengthen supervision and improve the availability of child survival drugs resulted in better preparation of the target districts for IMCI. Both the frequency and the effectiveness of supervisory visits improved. Cost recovery combined with drug supply stock management resulted in improving the availability of child survival drugs. Community participation in managing cost recovery was key in assuring acceptance and trust in the system. However, cost recovery induced an initial drop in the number of outpatient visits which requires longer monitoring. Health workers trained in IMCI were capable of examining children using the IMCI guidelines and in identifying very sick children. However, compliance to the guidelines standards for referral and follow-up visits was weak due to the wide dispersed nature of Niger's population, large distance between health facilities and referral sites, and lack of transportation.
Niger's experience suggests that it is important to strengthen the supervision system at the district level and assure the availability of essential drugs before the implementation of IMCI. However, due to the short duration between the initial IMCI clinical training and this evaluation, longer term follow up is necessary to confirm the impact of such quality assurance measures on the implementation of IMCI. The evaluation also revealed that while IMCI clinical guidelines are understood and followed by primary health care workers, standards for referral should consider the particular country situation.