Jump to:Page Content
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
Volume X, Number 4, Winter 2004/2005
Issue Topic: Evaluating Family Involvement Programs
Scott Rosas, from the Nemours Foundation, discusses the potential of concept mapping for the design and implementation of family support evaluations.
Although family support has gained widespread acceptance as a viable human-service delivery approach, particular characteristics of family support models often present challenges to determining clear program outcomes. Family support models have emerged from a variety of theoretical and nontheoretical approaches, reflected in the wide array of service mechanisms and diverse perspectives on the scope, content, and effects of programs. The absence of a clear empirical foundation on which to base decisions has led to confusion among developers and evaluators about which aspects of these programs to assess.
Moreover, because family support programs interact uniquely with the local context, they are constantly adjusting to environmental influences, and the same model can vary widely across contexts. Finally, these programs cannot be separated from the numerous individual characteristics and perceptions of those who develop and manage them. Although these issues are not exclusive to family support, they highlight the need for clear conceptual frameworks to guide the design, implementation, and evaluation of family support programs.
The need for conceptual clarity has led developers and evaluators to search for ways to develop adequate program theory. One promising approach to theory development is concept mapping.¹ Concept mapping is a multistep process that helps to describe and delineate concepts and their interrelationships through group process, multivariate statistical analyses, and group interpretation. It involves the following four steps:
Family support practitioners, for example, may brainstorm specific benefits for participants, such as peer support, gaining confidence, and communication skills. They then sort each of the brainstormed items into groups based on similaritysocial relationships, personal development, resource controland rate each item on a scale reflecting their relative importance. The analysis produces a concept map that visually represents each items place in the hierarchy; for example, relationships may score higher than services. In labeling and interpreting the map, practitioners discuss how it represents their ideas of what happens as a result of participation in the model.
Potential Benefits for Family Support
Concept mapping can assist family support developers and evaluators in articulating a framework of central program concepts. Using concept mapping to guide theory-driven evaluations can help strengthen the conceptual and empirical base of family support. Concept mapping can also help support sound implementation of family support programs. At regular intervals, concept maps could be compared with original program design to monitor the degree to which programs shift or adapt to their environments. Key differences between staff and participant orientations have important implications for program design and evaluation; regular review of concept maps can enable stakeholders to identify disparate views before they lead to implementation problems.
Building and maintaining good stakeholder-evaluator relations is another area in which concept mapping can be of use. Family support evaluations are often collegial and participatory, and require data-gathering techniques to facilitate the socialization of evaluators and agency staff. The application of concept mapping in evaluation can build value for stakeholders, as their input is incorporated throughout the entire process, increasing the likelihood of ongoing engagement in programming and evaluation.
Output from concept mapping can help family support developers and evaluators make informed decisions about design, measurement, analysis, and interpretation. These decisions are crucial; Olds³ found that the inability to specify the benefits families might receive from family support initiatives has been due to an absence of investigations grounded in a clear theoretical framework.
Finally, concept mapping can help facilitate the development of cumulative knowledge about social problem solving. Comparing concept maps across family support programs can highlight common program elements, thereby enabling practitioners and evaluators to more readily identify similarities and differences in application. Thus, concept mapping could be used to identify under what conditions and contexts family support program features contribute to the effectiveness of a particular model.
¹ Trochim, W. M. K. (1989). An introduction
to concept mapping for planning and evaluation. Evaluation and Program Planning,
² Multidimensional scaling is a multivariate statistical technique used to place items spatially on a map according to their relative similarity. Hierarchical cluster analysis is another multivariate statistical technique, used to group points into clusters representing higher order aggregates of the brainstormed items.
³ Olds, D. (1988). Common design and methodological problems encountered in evaluating family support services: Illustrations from the prenatal/early infancy project. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family programs (pp. 239264). New York: Aldine de Gruyter.
Scott Rosas, Ph.D.
Senior Program & Policy Analyst
Nemours Division of Health and Prevention Services
Christiana Building, Suite 200
252 Chapman Road
Newark, DE 19702