Jump to:Page Content
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
Volume X, Number 4, Winter 2004/2005
Issue Topic: Evaluating Family Involvement Programs
Veronica Thomas and Velma LaPoint, from Howard University’s Center for Research on the Education of Students Placed at Risk, describe the Talent Development approach to evaluating an urban family-school-community partnership program.¹
Evaluating family-school-community partnership programs for urban communities, especially low-income and underresourced communities, is a daunting task. Through our work with Howard University’s Center for Research on the Education of Students Placed at Risk (CRESPAR), we incorporated a Talent Development evaluation framework to help us design and implement evaluations of our school-based family and community partnership initiative for a public school in a high-poverty area in Washington, DC, which serves predominately African-American students.
The Talent Development evaluation approach is rooted in several evaluation traditions that intentionally seek engagement with contexts of practice. These traditions include responsive, participatory, empowerment, and culturally competent approaches to evaluation. Further, the Talent Development approach seeks to be practical, useful, formative, and empowering for the many individuals served by our evaluations, and to give voice to persons whose perspectives are often ignored, minimized, or dismissed in urban school settings.
Our family-school-community partnership program (FSCPP) sought to improve participants’ knowledge of, attitudes toward, and participation in FSCPP activities. The program consisted of a coordinated set of five interventions: (1) a family resource center located in the school building and used by family members, students, teachers, staff, and community members; (2) a high school action team (HSAT), which included students, teachers, staff, Parent Teacher Association representatives, family and community members, and CRESPAR staff; (3) a Talent Development attendance program, comprised of HSAT members, attendance staff, volunteers, and CRESPAR staff; (4) a Talent Development student team, consisting of students, teachers, school administrators, and CRESPAR staff; and (5) the Newsletter Communication Network, a publication, written and produced at the school, which provides information to students, family members, school staff, community businesses, and local organizations related to school and community activities.
In addition, five overlapping themes central to the Talent Development evaluation approach were incorporated into our evaluations of the FSCPP: (1) engaging stakeholders, (2) co-construction, (3) responsiveness, (4) cultural and contextual relevance, and (5) triangulation of perspectives. In designing and evaluating the FSCPP, we found that early and ongoing engagement of key stakeholders was critical. We thus started out by conducting needs assessments to better understand what stakeholders wanted and needed related to family-school-community partnership programming.
Boykin, A. W. (2000). The Talent Development Model of schooling: Placing students at promise for academic success. Journal of Education for Students Placed At Risk, 5(1/2), 3–25.
Thomas, V. G., & Stevens, F. I. (Eds.). (2004). Co-constructing a contextually responsive evaluation framework: The Talent Development Model of School Reform [Special issue]. New Directions for Evaluation, 101. San Francisco: Jossey Bass.
Co-construction calls for evaluators and school stakeholders to be ongoing collaborators in the framing of evaluation questions, development of instruments, collection of data, and interpretation, use, and dissemination of findings. We therefore valued stakeholder views in defining problems and solutions—an approach that empowered stakeholders and was effective in obtaining initial and ongoing participant buy-in. We provided multiple opportunities for FSCPP stakeholders to ask questions, critique our efforts, and provide input.
The program and its evaluation were constructed in ways that were directly responsive to stakeholders’ needs. Responsiveness was operationalized by consideration of our urban school stakeholders’ perspectives prior to planning, implementing, or evaluating any interventions. As for addressing cultural and contextual relevance, we view cultural competence as an essential element for working effectively with diverse populations. We have found that, as African-American evaluators working in a predominately African-American urban school setting, sharing the racial and ethnic background of stakeholders increases our ability to engage stakeholders and better understand the verbal, as well as nonverbal, behaviors being observed. However, we were also challenged to be ever so self-reflective and open to examining our own assumptions about urban schools and their stakeholders.
Triangulation involves using more than one perspective to study the same thing. In our work, triangulation of perspectives occurred in multiple ways, including, for example, investigator triangulation, methodological triangulation, and data-analysis triangulation.
Participants in the high school action team became “assistant evaluators” to CRESPAR staff and received ongoing training in their evaluation role. HSAT members were involved, for example, in developing evidence-based program goals, objectives, and outcomes; reviewing data collection instruments; and assisting in the development, screening, and refinement of items for surveys, focus groups, and interviews, with particular attention to the contextual and cultural responsiveness of data collection protocols. During the formative evaluation, HSAT members also provided feedback on FSCPP implementation, allowing evaluators to make immediate changes for program improvement. They also reviewed written documents, such as students’ school attendance logs and logs that recorded student, family, and community members’ attendance at school events; conducted interviews with program participants; distributed and collected surveys from stakeholders; and participated in quarterly focus groups.
The complex and dynamic nature of urban schools and communities makes the implementation and evaluation of reform efforts a formidable challenge. However, we view the Talent Development approach as particularly invaluable when participants are low-income persons of color who are oftentimes disenfranchised from both evaluation and service-delivery systems. Based on our experiences working in low-income urban school settings, we believe that program planning, implementation, and evaluation can be significantly enhanced by front loading implementation and evaluation efforts, engaging stakeholders in meaningful ways throughout the entire process, and co-constructing intervention and evaluation activities. Giving back to the community in both tangible and intangible ways, ensuring that field implementers and evaluators are culturally competent, and showing a high tolerance for ambiguity and change are also critical.
¹ The work reported herein was supported by grants from the Institute of Education Sciences (formerly the Office of Educational Research and Improvement), U.S. Department of Education. The findings and opinions expressed in this article do not necessarily reflect the position or policies of the Institute of Education Sciences or the US Department of Education.
Veronica G. Thomas
Center for Research on the Education of Students Placed at Risk
Holy Cross Hall, Room 427
2900 Van Ness Street, NW
Washington, DC 20008