Evaluation of Technical Quality The Achenbach System of Empirically Based Assessment (ASEBA)
In Unit 2, I selected the Achenbach System of Empirically Based Assessment (ASEBA) – any forms or age range. The ASEBA is considered to be an in-depth, comprehensive, evidence-based assessment system, which can be administered by group or individually (Achenbach, 2015). The purpose of ASEBA is to assess the adaptive functioning, mental competencies, and problems of adolescents and children, which focuses on the individuals’ ability to cope up with various events happening during stages of life such as illnesses, loss of loved ones and increased dependency (Achenbach, 2015). The ASEBA is typically designed to facilitate assessment, planning of an intervention, and evaluating outcomes among mental health, school, social service, and medical practitioners responsible for dealing with maladaptive behavior in young adults, adolescents, and children. It consists of four levels of assessment, that is, older adults, preschool, school-age, and adults.
Technical Review Article Summaries
Deckers’ et al. (2020) article, Screening for Autism Spectrum Disorder with the Achenbach System of Empirically Based Assessment Scales, explored how to integrate screening possibility for Autism Spectrum Disorder (ASD) within the ASEBA scales. It examined the ability of Teacher’s Report Form (TRF) and school-aged Child Behavior Checklist (CBCL) to screen the disorder in adolescents and children between the ages of 6 and 18. The study compared different screening variants for the TRF, CBCL, and the combination of both TRF and CBCL (Deckers et al., 2020). They included (1) social problems, separate depressed or withdrawn, and thought problems syndrome scales, (2) combination scales, (3) special ASD scales consisting of various relevant items (Deckers et al., 2020).
The study addressed the predictive validity by predicting the clinical DSM-IV Diagnosis and the ADI-R supported DSM-5 Diagnosis. Under clinical DSM-IV Diagnosis, it was found that the thought and social problems syndromes scales did not discriminate between the clinical and ASD control group. However, the ASD profile, depressed/withdrawn scale, and WTP scale showed significant effects (1.07 to 1.22), and associated AUC scores (.67-.68) were fairly small and relatively low, respectively (Deckers et al., 2020). The study examined the ability of the screening potentials further by determining the most optimal cut-off points and exploring their respective sensitivity, primarily negative predictive value and positive predictive value (Deckers et al., 2020). Predictive validity was optimal in a situation where CBCL data were used. For the ASD scale, the CBCL could correctly identify between 73 and 83 percent of those children with ASD (sensitivity) as well as between 69 and 72 percent of those without ASD (specificity) (Deckers et al., 2020).
There was no evidence to support the use of combined TRF and CBCL to enhance the screening potential compared to employing the ASEBA scale. Based on the comparison groups applied, the combined TRF- and CBCL-based ASD scale produced an approximately 1 to 8 percent increase in the accuracy of predicting ASD compared to their accuracy when used alone. Overall, the study indicated that the special ASEBA scales, primarily when completed by parents, were most predictive of ASD (Deckers et al., 2020). Also, the study discovered that after the initial screening using ASEBA scales, patients should be subjected to thorough diagnostic assessment to definitively determine whether children and adolescents are suffering from ASD.
Willems et al. (2018) article, Genetic and environmental influences on self-control: Assessing self-control with the ASEBA Self-Control Scale, developed a new measure from ASEBA. Willems et al. (2018) applied the theoretically-derived set of items of the ASEBA to develop Achenbach Self-Control Scale (ASCS) for a population aged between seven years and sixteen years. Willems et al. (2018) addressed both validity and reliability. The study applied a large dataset of more than 20,000 children to demonstrate the psychometric properties of the ASCS for self-reports, teacher-reports, and parent-reports, primarily by assessing the internal and criterion validity test-retest and inter-rater reliability (Willems et al., 2018). On criterion validity, the study found a cross-sectional association between the measure and various relevant results, including the well-being and achievement in education, which was significant in the hypothesized direction (Willems et al., 2018). The study used the Root Mean Square Error of Approximation (RMSEA), the Tucker Lewis Index (TLI), and the Comparative Fit Index (CFI) to evaluate the goodness of fit. It maintained the RMSEA between 0.05 and 0.08, which was adequate fit based on error approximation, and CFI, and TLI at 0.95 or larger (Willems et al., 2018).
The study’s descriptive statistics of ASCS and cronbach’s alpha coefficients indicated that there was sufficient internal consistency, which ranged between 0.81 and 0.83 for both the teacher- and parent-reports 0.70 and 0.73 for ASCS self-report (Willems et al., 2018). Inter-rater reliability was examined by correlating ASCS measures over time and raters, which indicated significant cross-sectional correlations between informants, with (1) robust and significant correlations between mother- and father-reports (.66-.67), (2) moderate and significant correlations between child- and parent-reports (.40-.44), (3) significant low correlation between self- and teacher-reports (.29), and (4) moderately significant correlation between parent- and teacher-reports (.32-.40) (Willems et al., 2018). The test-retest reliability was assessed using the relations between the measured scores of self-control across time within raters, with the interval of time between three and five years. Test-retest reliability indicated (1) strongly significant correlations from age 7 to 12 between mother reports (.57-.67) and the correlation between father-reports at the same age (.52-.65), (2) moderately or strongly strong correlation between teacher-reports from the same age (.43-.54) and between self-reports from 12 to 16 years old (.35-.55) (Willems et al., 2018).
Overall, the study found that there existed associations between the ASEBA’s ASCS and measures of well-being, substance use, and educational achievement. It was discovered that genetic influence accounted for about 64 to 75 percent of the variance in self-control regarding teacher- and parent-report (7 to 12 years old), and for approximately 47 to 49 percent of the variance in self-control on self-reports (12 to 16 years old) (Willems et al., 2018). The remaining variance accounted for those environmental influences that did not have a shared association.
Willoughby’s et al. (2011) article, Using the ASEBA to Screen for Callous Unemotional Traits in Early Childhood: Factors Structure, Temporal Stability, and Utility, examined a five-item screening measure of Callous Unemotional (CU) trait-based on details drawn from the Pre-school Form of the ASEBA. ASEBA was used for parents to identify and differentiate Callous-unemotional behaviors from ADHD and ODD behaviors(Willoughby et al., 2011). The authors hypothesized that parents would identify CU separately. The results showed that parents were able to differentiate CU from ADHD and ODD in early childhood. However, limitations included identifying more factors to scale. The scale used in this study was the 5-item ASEBA form, with 7-items(Willoughby et al., 2011); this scale may not be sufficient for future studies to single out CU traits to a given point in development.
The behaviors were as stable as ADHD and ODD behaviors measured by the same format across the same time frame. The high levels of stability did not undermine the likelihood of significant mean level change, but the high stability estimates indicated that the rank used to place or order individual differences was preserved over a specific range of time (Willoughby et al., 2011). Although CU behaviors were differentiated from ADHD and ODD and were seen to be stable over time, there was a large latent correlation between factors. The corresponding coefficient alphas for CP, CU, and ADHD were .83, .65, and .83, respectively (Willoughby et al., 2011). However, because of the violation of tau equivalence in the data, that is, the differential strength of association between individual items as well as the underlying construct, the average item-level R2 values were used as indicators of item reliability rather the respective coefficient alphas (Willoughby et al., 2011). All CFAs provided a good fit, and the 3-factor model offered a statistically superior fit (Willoughby et al., 2011). In utilizing the same screening process, there is potential to facilitate and implement early intervention when evaluating early onset of Callous Unemotional traits.
Giana’s et al. (2015) article, Reliability of Child Behavior Checklist and Teacher’s Report Form in a Sample of Brazilian Children, examined the temporal stability of Teacher’s Report Form (TRF) and Child Behavior Checklist (CBCL). They were administered to teachers and parents of school-aged children, where the intraclass correlation coefficient (ICC) was applied to measure the temporal stability (Giana et al., 2015). Giana et al. (2015) addressed the validity and reliability of the measure, mainly using the test-retest reliability for all scales to cover the internalizing behaviors, total behavior problem scale, and externalizing behaviors in both the TRF and CBCL. Using ICC, the test-retest stability of TRF and CBCL was measured, and temporal stability was analyzed based on the optimal cut off points: “ICC <0.4 = poor reliability; ICC 0.4-0.75 = fair to good reliability; and ICC >0.75 = excellent reliability” (Giana et al., 2015). The one-year test-retest reliability was excellent. However, the study noted fair reliability in TRF’s assessment of internalizing behaviors (Giana et al., 2015). The descriptive statistics used to indicate the participants’ traits and behavior problems’ prevalence.
In the article, The validity of the multi-informant approach to assessing child and adolescent mental health, it is clear that informants hold key information for many mental health patients, especially children. Informants for children can range from parents, teachers, and the patient themselves (De Los Reyes et al., 2015). To maintain a complete report, a trained clinician can observe or administer a standardized test. A collection of reports from multiple informants can cause a great deal of concern. It is of concern because it poses the risk of inconsistent conclusions. Discrepancies can lead to misinterpretations of conclusion and delay proper services to patients. The article continues to explain that the multi-informants’ validity may continue to be unclear on expressing the variation of the patients’ mental health, therefore affecting the concerns brought to conduct a mental health assessment (De Los Reyes et al., 2015). The author considers research from multi-informant clinical assessments and their validity, broken down to incremental validity and construct validity.
Incremental validity may be influenced by the criterion being contaminated. In clinical and research settings, the informants reported a variety of measurement methods. In certain conditions, the research was evaluated, and the research supported the construct validity of structured interviews and rating scales. However, the construct validity may still fail to contribute to the information that is contributed to other measurements. The authors had two sets of findings, where Pearsons r held significance between informant reports, in turn, calculated common variances, where the estimated calculations resulted in small variance problems(De Los Reyes et al., 2015). In comparing the findings, the authors saw a higher correlation between multi informants and the approach of services for patients. Overall, there were some implications of error: transient error, random error, and systematic error. Transient error is the characteristic of rater that may hinder (De Los Reyes et al., 2015). Random error poses challenges to reliability between multi-informant reports because reports can only be as reliable as the individual informants. Lastly, systematic error may indicate that multi-informant assessments can bring about differences between individual informants (De Los Reyes et al., 2015). However, if there are consistencies, the differences may be meaningful.
Conclusion
The ASEBA testing forms show in a variety of cases to hold reliability in measuring scales of specific traits. The ASEBA test using the Child Behavior Checklist shows excellent reliability than Teacher’s Report Form. There was no evidence to support the use of combined TRF and CBCL to enhance the screening potential compared to employing the ASEBA scale. The ASEBA scales, primarily when completed by parents, were most predictive of ASD. Predictive validity was optimal in a situation where CBCL data were used. Therefore, if the attributes can be operationally defined, direct and indirect scores can be measured and applied to individuals as part of the population. However, limitations included identifying more factors to scale. In my opinion, the ASEBA test remains a significant test for identifying behavior problems in children diagnosed with Autism, along with identifying developmental delays these children might have.
References
Achenbach, T. M. (2015). Multicultural evidence-based assessment using the Achenbach System of empirically-based assessment (ASEBA) for ages 1½-90+. Psychologia: Avances De La Disciplina, 9(2), 13-23. Retrieved from http://library.capella.edu/login?qurl=https%3A%2F%2Fsearch.proquest.com%2Fdocview%2F1793554225%3Faccoun
Achenbach T. M. (2019). International findings with the Achenbach System of Empirically Based Assessment (ASEBA): applications to clinical services, research, and training. Child and adolescent psychiatry and mental health, 13, 30. https://doi.org/10.1186/s13034-019-0291-2
De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. G., Burgers, D. E., & Rabinowitz, J. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858-900. doi:10.1037/a0038498
Deckers, A., Muris, P., & Roelofs, J. (2020). Screening for Autism Spectrum Disorder with the Achenbach System of Empirically Based Assessment Scales. Journal of Psychopathology and Behavioral Assessment, 42 (1): 25. https://doi.org/10.1007/s10862-019-09748-9
Giana, B. F., Juliana, R. P., Daiane Silva, d. S., Denise, R. B., & Juliane, C. B. (2015). Reliability of Child Behavior Checklist and Teacher’s Report Form in a Sample of Brazilian Children. Universitas Psychologica, 14(1), 149-156. Retrieved from http://library.capella.edu/login?qurl=https%3A%2F%2Fsearch.proquest.com%2Fdocview%2F1771625150%3Facco
Willems, Y. E., Dolan, C. V., van Beijsterveldt, C. E. M., de Zeeuw, E. L., Boomsma, D. I., Bartels, M., & Finkenauer, C. (2018). Genetic and environmental influences on self-control: Assessing self-control with the ASEBA Self-Control Scale. Behavior Genetics, 48(2), 135–146. https://doi-org.library.capella.edu/10.1007/s10519-018-9887-1
Willoughby, M. T., Waschbusch, D. A., Moore, G. A., & Propper, C. B. (2011). Using the ASEBA to screen for callous-unemotional traits in early childhood: Factor structure, temporal stability, and utility. Journal of Psychopathology and Behavioral Assessment, 33(1), 19–30. https://doi-org.library.capella.edu/10.1007/s10862-010-9195-4