Study description
Positive Outcome for Orphans (POFO) is an ongoing longitudinal study following a cohort of children, starting ages 6 to 12, who live in institutional or community-based settings in 5 low income countries: Cambodia, Ethiopia, Kenya, India, and Tanzania. This analysis used 3 years of data from the community-based sample to address the relationships stated above.
Study sample
The detailed sampling strategy and general characteristics of the sample have been reported elsewhere [26, 27]. The following describes the elements of the sampling strategy applicable to this analysis.
The POFO study utilized a two stage random sampling methodology to identify a representative sample of 1,480 orphaned and abandoned children living in community-based settings in six sites across five low and middle income countries. Within each site, geographic or administrative boundaries were used to define sampling areas (clusters), 50 clusters were randomly selected at each site, and up to five eligible children ages 6-12 years were selected from each cluster. Eligible children were defined as follows: orphans were those children for whom one or both parents had died [1] and an abandoned child was one whose parents had left with no expectation of return. Eligible children were randomly selected from available lists or through a house-to-house census. One child per household was selected to participate in the study. For households with multiple age-eligible children, the child whose name started with the earliest letter in the alphabet was selected to participate. Additionally, each site enrolled 50 community-based children who were not orphaned or abandoned at baseline as a comparison group.
Data collection protocol
As previously published [22, 26], the following describes the procedures of data collection relevant to this analysis. Children and each of their self-identified primary caregivers were contacted and interviewed twice per year for up to 3 years. Baseline and annual follow up surveys collected data on numerous characteristics including the child’s exposure to traumatic events, symptoms of emotional and behavioral difficulties, cognitive development, and educational attainment. Additionally, caregivers reported on household socioeconomic characteristics. Ethical approval was obtained from the Duke University Institutional Review Board (IRB) and from local and national IRB’s in each participating country.
The primary measures utilized in this study include child self-reports of emotional difficulties, tests assessing cognitive abilities, and child and caregiver reports of exposure to potentially traumatic events, all reported at baseline, and at 12-month, 24-month and 36-month follow ups. The measures used for trauma, emotional difficulties and cognitive development were previously validated for use across cultures (see below) and field-tested using pilot interviews.
Study measures
Measuring cognitive development: kaufman assessment battery for children and the california verbal learning test
The KABC-II is an individually administered test of intelligence and achievement that was developed with the intention of “building in sensitivity to preschoolers, minorities and exceptional populations” [28]. Three nonverbal subtests from the Second Edition of the Kaufman Assessment Battery for Children (KABC-II), Hand Movements, Triangles and Pattern Reasoning, were assessed at annual child interviews in the POFO study. The KABC was chosen because it is one of the most frequently used tests of learning ability internationally. The three non-verbal subtests were chosen to be used across the five countries as they are less dependent on language differences.
The California Verbal Learning Test (CVLT-C) is a test of verbal memory, used by POFO researchers as an indicator of memory, attention and motivation [15]. POFO interviewers modified the memory list, referred to here as the Market List (ML). Locally relevant lists were developed at each site that contained 15 items a child might see at their respective markets. The test required children to encode and store information in order to repeat back what was read to them. The Market List was chosen based on observed variability of children’s engagement with the tests during pilot work in East Africa, suggesting that a tool that reflects motivation and attention would be an important addition to the learning tasks on the KABC-II [15].
Previous analyses by POFO researchers validated these tests as measures of learning and performance for children living in LMIC [15]. The findings of this previous analysis provided support that across the five countries, the subtests functioned as one would expect measures of learning to function, that is, raw scores increased with chronological age. These tests were also strongly associated with years in school for age. Hence, the KABC II scores used here can be seen as an effective tool for measuring learning, which also reflects experience in the learning environment [15].
This analysis used the highest of the three KABC test standard scores (called topscore) for each child at each round as the primary outcome measure for cognitive development. This measure represents the best the child was able to do across the three subtests when tested by the interviewer. Standard scores of the KABC II range between 0 and 19, and each subtest has a mean of 10 and a standard deviation of 3 [28]. In this analysis, scores were scaled to US age standards to enable comparison across children and these five settings. The average number of items recalled in the first three repetitions on the Market List was used as an ancillary measure of learning, attention, and motivation.
Measuring psychosocial well-being and emotional difficulties: the strengths and difficulties questionnaire
The Strengths and Difficulties Questionnaire (SDQ) is a behavioral screening tool, designed for children ages 4-16, that measures psychosocial well-being across five dimensions: (1) emotional symptoms, (2) conduct problems, (3) hyperactivity/inattention, (4) peer relationship problems, and (5) prosocial behavior. Each subscale has 5 items, scored on a 3-point Likert scale (0-2). The four difficulties subscales add up to a Total Difficulties Score, while the fifth subscale provides assessment of prosocial behavior. POFO researchers chose the Strengths and Difficulties Questionnaire “for its brevity, its psychometric properties, and its frequent use in other international studies” [22]. The questionnaire can be completed in two versions, either by parents, teachers or caregiver report, or, for children ages 11 and older, by self-report [29]. With scoring from 0-2 on each individual item, the Total Difficulties scale ranges from 0 - 40. This analysis used the Total Difficulties score as a continuous variable, rather than using a clinical cutoff, which is not available across these sites. The validity of the self-reported Total Difficulties scale has been assessed and confirmed in multiple contexts (Cronbach’s alpha ranging from 0.73-0.89), indicating that the scale itself is internally valid [1]. In the POFO sample, Cronbach’s alpha was 0.73. These analyses used the Total Difficulties Score self-reported by the child as the primary measure of emotional difficulties. Limiting the SDQ self report to ages 11 and older is in line with the recommendation of the SDQb.
Measuring adverse childhood experiences: the life events checklist
This analysis used the Life Events Checklist, first created by the National Center for Posttraumatic Stress Disorder (PTSD) to aid in the diagnosis of post traumatic symptoms [30]. This checklist, which inquires about exposure to potentially traumatic events such as natural disasters, witnessing someone being hurt or killed, experiencing physical or sexual abuse, or being forced to leave home, is one of the most commonly used research instruments to evaluate exposure to trauma across countries and cultures [31]. Caregivers and children were independently asked at each interview whether the child had ever witnessed or experienced each of 21 types of events. A child was counted as having experienced an event if either the caregiver or the child reported it. As described previously, four categories of events were excluded from this analysis [22].c A cumulative traumatic exposure variable was generated for this analysis, which sums the total count of up to 17 different traumatic event categories reported through any given round.
Additional covariates: household wealth, caregiver illiteracy, and relationship to the child
An asset checklist and other elements from the Demographic and Health Surveys (DHS)d of each site were used to derive a wealth index score for each participating household [32–36]. Wealth index scores are continuous, standardized for comparability with wealth index scores in each country’s DHS, and indicate greater affluence as the score increases. Caregiver illiteracy was assessed based on a literacy test administered at the time of each survey. Caregivers unable to read four short sentences in the local language were classified as illiterate. The child’s relationship to the caregiver (parent versus non-parent) and orphan status (single or double orphan v. abandoned) were included in the analysis.
Analyses
A linear regression model was estimated to describe the relationship between emotional difficulties (SDQ Total Difficulties score) and various explanatory variables, including orphan status, exposure to potentially traumatic events, and household wealth. Additional linear regression models with the KABC topscore and Market lists as dependent variables were used to estimate the association between emotional difficulties and cognitive development. These models controlled for age, gender, orphan status, wealth and caregiver illiteracy, and the number of prior administrations of the KABC test to account for child learning over time. Models analyzed up to four time points for each child cross-sectionally; each model specification was run separately by site and jointly for all sites.
Child-level fixed effects models were estimated to describe the relationship between the SDQ Total Difficulties score and cognitive outcomes while controlling for time invariant characteristics of children that may affect outcomes. Models were run jointly across all sites and controlled for age as the only other observed time varying characteristics expected to be associated with the child’s cognitive development during the study period. Additional models, run for sensitivity analysis, evaluated whether the association between emotional difficulties and cognitive outcomes differed by caregiver type (parent versus non-parent), OAC status, or study site. Effect estimates for subgroups were calculated as linear combinations of SDQ main effects and interactions with the respective indicator variables for each subgroup. Two additional fixed effects model analyzed the association of cognitive outcomes with the four SDQ subscales (which comprise the SDQ Total Difficulties score) and with caregiver reports of the SDQ.
All models were estimated with robust standard errors to account for error correlations within sites and between multiple observations from each child. Child-level fixed effects models accounted for clustering at the level of the child. Weights were constructed to account for differences in the number of children and their age and gender distributions across study sites and were used in all models.
Attrition
To evaluate the extent to which attrition may have biased our estimates, bivariable logistic regression models of baseline characteristics analyzed whether children who left the study differed significantly from those who stayed.