You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

Visit the Subscription Center

Program Description

Overview The Communities Organizing Resources to Advance Learning (CORAL) initiative in California works to link communities, institutions, and residents around the common goal of improving youth academic achievement through the provision of structured literacy programming and enriching out-of-school time opportunities. Communities with low-income, low-achieving schools in five cities (Sacramento, San Jose, Fresno, Pasadena, and Long Beach) are part of the initiative.
Start Date Programming started in 2001 in Long Beach and Pasadena, 2002 in Fresno, and 2003 in Sacramento and San Jose.
Scope state
Type after school
Location urban
Setting community-based organizations and public schools
Participants elementary through middle school students (grades 1–8 with the majority in grades 1–5)
Number of Sites/Grantees 31 school-based sites and 6 community-based sites in 2004
Number Served 5,321 children in 2004
Components A lead agency (a local nonprofit organization) was selected by the initiative funder to plan and run CORAL in each city. The lead agency works with schools and community-based organizations (CBOs) to provide after school programming for youth. In two of the cities, the lead agency directly administers programming at all sites. In the other three cities, the lead agency contracts with local CBOs to operate some sites, although the lead agency typically also directly operates at least one site. Each site is overseen by a coordinator, who manages site staff and daily activities.

CORAL cities initially implemented a variety of programming. The main components in common across sites were homework help and enrichment programming, which included activities such as art or dance, usually without an academic focus. In the 2004–05 school year, CORAL began to provide more consistent academic programming as part of a new “balanced literacy” approach. This approach is shaped by research suggesting that reading skills can be improved when children (a) are exposed to “read alouds” that model fluent reading; (b) practice writing and talking about books and vocabulary; (c) practice phonetics and “word attack” skills; and (d) read books they enjoy at an appropriate reading level. CORAL provides its cities with information on two balanced literacy program models, Youth Education for Tomorrow (YET) and Kidzlit, but cities are free to implement other programs of their choice.

Regardless of which model is used, cities are required to include balanced literacy programming 3–4 days a week for at least 75–90 minutes a day and must follow certain guidelines. Each lesson has to include, at minimum, an opportunity for staff to read aloud to youth and independent reading time with access to books organized by specific reading levels. The program also must include other key balanced literacy strategies: book discussion, writing, vocabulary building, and “fun” activities to encourage literacy skills development.

CORAL programs are located in neighborhoods with high levels of poverty and very low-performing schools. CORAL staff work closely with school staff to select youth who could benefit most from the program, including those who are struggling academically or socially and/or who are English learners. With the shift to balanced literacy, two CORAL cities began to work with schools to intentionally target children who were well behind in reading based on their standardized test scores.
Funding Level Each city received $2 million per year through 2003 from the James Irvine Foundation (Irvine). In 2004, Irvine reduced the annual funding to $1.6 million per year per city.
Funding Sources James Irvine Foundation; four CORAL cities also received United States Department of Education 21st Century Community Learning Center Program grants, and one city received state CalWORKS (the California Temporary Assistance for Needy Families program) child care dollars.

Evaluation

Overview The evaluation examined the implementation of sites’ literacy strategies and their early effectiveness in retaining children and in producing literacy gains during the 1st year after literacy program efforts were introduced (the 2004–2005 school year).
Evaluator Public/Private Ventures (P/PV)
Evaluations Profiled Launching Literacy in After-School Programs: Early Lessons from the CORAL Initiative
Evaluations Planned Evaluation activities will continue through June 2006. A final report at the end of the evaluation period (after children have been in the program for approximately 18 months) will combine 1st and 2nd year findings and will examine the initiative’s longer term effectiveness in providing high-quality programming and contributing to positive changes in children’s reading skills, reading attitudes and behaviors, general school attitudes and behaviors, and improved standardized test scores.
Report Availability Arbreton, A. J. A., Goldsmith, J., & Sheldon, J. (2005). Launching literacy in after-school programs: Early lessons from the CORAL Initiative. Philadelphia, PA: Public/Private Ventures. Available at www.ppv.org/ppv/publications/publications.asp?search_id=20

Contacts

Evaluation Amy J.A. Arbreton
Director of Research
Public/Private Ventures
Lake Merrit Plaza
1999 Harrison Street, Suite 1550
Oakland, CA 94612
Tel: 510-273-4600
Email: aarbreton@ppv.org
Program Anne Stanton
Youth Program Director
The James Irvine Foundation
575 Market Street, Suite 3400
San Francisco, CA 94105
Tel: 415-777-2244
Email: astanton@irvine.org
Profile Updated April 18, 2006

Evaluation: Launching Literacy in After-School Programs: Early Lessons from the CORAL Initiative


Evaluation Description

Evaluation Purpose To address the following question: Can literacy activities be integrated into existing after school programs with sufficient quality to promote reading gains? In order to answer this question, several additional issues were addressed: Who participated in CORAL? Was there early evidence of reading gains as a result of youth’s participation? Did some youth benefit from CORAL more than others? What strategies seemed to contribute most to reading gains? What practices facilitated quality literacy instruction?
Evaluation Design Quasi-Experimental and Non-Experimental: CORAL staff surveys and participant test scores were collected from all 37 sites. Surveys were mailed to 564 staff members (including team leaders, site coordinators, volunteers, paraprofessionals, educational liaisons, city directors, literacy directors, enrichment providers, and other staff) in February and March 2005 and were returned by 412 staff members, for a response rate of 73%.

Most of the evaluation focused on 4 or 5 “research” sites in each city, for a total of 23 sites. Sites were selected if they served elementary school youth and had been in operation for at least 1 year. Within these sites, the evaluation focused on third and fourth graders, because research suggests that this time period is crucial for reading skills development.

Observations of balanced literacy activities were conducted at research sites with 56 groups of 12–20 third and fourth graders. Also at these sites, evaluators interviewed approximately 25 key informants per city (including CORAL staff such as team leaders, city directors, board members, and school staff such as principals) and conducted 19 parent focus groups with approximately 170 parents across groups.

Parents of 738 CORAL third and fourth graders at research sites agreed to allow their children to participate in the evaluation, out of 762 who completed permission forms, for a participation rate of 97%. Of youth with permission, 635 were randomly selected to be part of a youth survey and reading assessment cohort (approximately 125 per city). The survey was administered in fall 2004 to 515 youth, consisting of 280 third graders and 235 fourth graders. (Some of the original cohort left CORAL before the survey, were absent on survey days, or were designated special education and determined to be inappropriate participants for the study.) Also in fall 2004, an informal reading inventory (IRI) was administered to 520 youth, consisting of 281 third graders and 239 fourth graders. In spring 2005, IRIs were administered to 383 youth still attending CORAL, for a follow-up rate of 74%.

Teacher rating forms of children’s behaviors during the school day were also gathered in spring 2005 from teachers of 288 of 418 (69%) children attending the school-based CORAL sites. In addition, data were collected from a Management Information System (MIS) that tracked enrollment and attendance; these data were collected from all 37 sites, but since some sites did not complete data in time to be included in the evaluation, only MIS data from the 23 research sites were used in the evaluation.
Data Collection Methods Interviews/Focus Groups: Interviews with CORAL staff and stakeholders asked about staffing structures and training, participant recruitment and targeting strategies, lesson and activity planning, obstacles to implementation of the balanced literacy model, relationships with schools and other partners, and plans or goals for the future.

In focus groups, parents were asked three primary questions: Why did they choose to send their children to CORAL? How, if at all, had their children benefited by participating in CORAL? What was the quality of their interactions with CORAL staff?

Observation: Structured observations assessed the extent to which CORAL programming successfully incorporated the six balanced literacy strategies; provided high-quality instruction (e.g., clearly presented and organized), group management, and connection-building strategies (e.g., related texts to youth’s experiences); and offered examples of positive adult support (e.g., acted in a responsive way). Each group was observed 2–4 times during the year, with most observed 3 times (n = 48).

Observations measured quality dimensions in the areas of Support and Instruction and Balanced Literacy Strategies. Scores of 1–5 were assigned to each dimension, with 1 signifying extremely negative behaviors and little to no positive strategies and 5 signifying consistent use of strong strategies.

The Support and Instruction area included:

Adult Support: Did staff help all youth succeed? Were they interested in youth’s thoughts and ideas? Did they display warmth toward youth and develop supportive relationships?

Instructional Quality: Did staff use successful instructional strategies? Did they motivate youth to participate and push them to move beyond present skill levels?

Group Management: Did staff manage youth behavior in an appropriate way for the age and activity?

Connection Making: Did staff make links between youth and the text and between parts of the lesson?

The Balanced Literacy Strategies area included (a) read aloud: Were read alouds interactive and engaging to youth? Did staff introduce text in a way that interested youth? Did they encourage youth involvement?; (b) book discussion: Did staff ask clear questions to guide discussion? Did they lead youth to draw connections between their lives and the story? Did they encourage all youth to participate?; (c) writing: Was the activity flexible so all youth could work at a level appropriate to their skills? Did it provide an opportunity to write about topics relevant to a book they read, personal experience, or current events? Did staff interact with youth throughout?; (d) independent reading: Were youth focused on reading? Were staff involved with youth?; (e) vocabulary: Did staff use strategies to increase or reinforce youth’s vocabulary?; and (f) skill development activities: How did staff reinforce literacy skills? Did they make activities fun and engaging?

To measure implementation quality and link it to reading level changes, a Literacy Profile was developed that assigned a composite score of 1–5 to literacy activities experienced by each of the 56 groups of youth observed. The Profile considers the quality of the primary balanced literacy strategies (read alouds, independent reading, writing, book discussions, vocabulary, and skill development) and whether each strategy was observed during at least half of the observations. Profile 1 indicates a group that implemented read alouds and independent reading during fewer than half of the observations or did so at a low quality. Profile 2 indicates a group that implemented read alouds and independent reading at a quality rating of at least 3 during at least half of the observations. Profile 3 indicates a group that implemented read alouds, independent reading, and one other literacy strategy at a quality rating of at least 3 during at least half of the observations. Profile 4 indicates a group that implemented read alouds and independent reading at a rating of at least 4, as well as all of the other four strategies at a rating of at least 3, during at least half of the observations. Profile 5 indicates a group that implemented all six balanced literacy strategies at a rating of 4 during at least half of the observations.

Secondary Source/Data Review: Demographic records from each school district and MIS enrollment and attendance data were gathered from each site in each city.

Surveys/Questionnaires: Youth surveys asked about program experiences and developmental outcomes, including sense of safety, social support, interesting activities, and conflict management skills. Several questions addressed attitudes toward reading and school, including enjoyment of reading, sense of efficacy as a reader (feeling that they can learn to read and that reading is easy), and effort and interest in school.

The staff survey included questions about staff’s educational background, experience, training, time with CORAL, and responsibilities within CORAL.

The teacher assessment asked teachers to assess CORAL participants’ school-day behavior, including aggressiveness, positive social relationships, and cognitive concentration.

Test/Assessments: est scores for the spring administration of the California Standards Test (CST) English Language Arts section were gathered from seven of the nine school districts serving CORAL participants in 33 of the 37 sites.

The Jerry L. Johns Basic Reading Inventory (Johns, 1997) was the IRI administered to research site third and fourth graders at pretest and posttest. The inventory assigns a grade-level reading designation, based on youth’s reading of a series of graded word lists and paragraphs, and their responses to comprehension questions after each paragraph.

Reference:
Johns, J. L. (1997). Basic Reading Inventory: Pre-Primer through Grade Twelve & Early Literacy Assessments (7th ed.). Dubuque, IA: Kendall/Hunt Publishing Co.
Data Collection Timeframe Data were collected during the 2003–2004 program year.

 

Findings:
Formative/Process Findings

Activity Implementation CORAL programs tended to be open 4 days (in one city) or 5 days (in four cities) a week, for about 3 hours a day. Youth were expected to attend every day the program was open.

Participants were generally divided by grade into groups of 12–20 for all activities; these groups remained with the same team leader over the course of the program. Except for one city, where certified teachers led literacy activities, team leaders taught literacy-related lessons.

Youth who attended CORAL in 2004–2005 participated in (a) balanced literacy activities, offered about 5 hours a week on average, divided across 3 or 4 days; (b) homework help, which varied from 5 minutes a day in a few sites to up to 60 minutes a day; and (c) cultural/academic enrichment activities, such as art, science, dance, and cooking, that generally lasted 60–90 minutes, occurred 2–5 days a week, and generally rotated on a 6–8 week basis so that youth participated in a variety of enrichment activities during the year.

During read alouds, staff sometimes introduced stories by asking youth questions related to the topic or having them predict what would happen based on the pictures. Staff read a variety of books of varying lengths on diverse topics.

Read alouds were sometimes followed by book discussions, which took a variety of formats. Staff sometimes asked youth to reflect on what they had just read or asked targeted questions triggered by the books.

Writing exercises sometimes followed read alouds or book discussions and were sometimes extensions of book discussions.

Throughout balanced literacy activities, staff sometimes incorporated vocabulary exercises in order to introduce or review words. Many staff devoted a space on the wall to vocabulary and added new words every day. Staff often introduced vocabulary as part of the read aloud, either reviewing words before reading or pausing while reading to define a new word. Sometimes staff devoted more time to vocabulary—for example, having youth record words and their definitions in journals or act out the meanings of new words.

Independent reading exercises usually occurred at the beginning or end of literacy activities and lasted an average of about 15 minutes (ranging from less than 5 minutes to more than 30 minutes). In most cases, youth chose from books at individualized reading levels, mostly fiction but also some nonfiction and comic books.

Some balanced literacy lessons included skill development activities, which were opportunities for youth to practice particular literacy skills, such as learning to write in complete sentences and practicing specific letter sounds. These activities occurred less frequently than other balanced literacy activities.

Because parents and school staff in some cities saw homework help as a key benefit of CORAL, these sites devoted a large amount of non-literacy program time to homework. In most cases, though, cities devoted much of this time to enrichment.

Cities had varying philosophies on the relationship between balanced literacy and enrichment activities. Some cities intentionally provided enrichment with a strong literacy focus. In other cities, the emphasis was on connecting topics covered in balanced literacy lessons to the topics of enrichment activities.

Independent reading took some time to “get off the ground,” with an average quality rating of 2.97 in the fall and 3.14 in the spring, as sites faced challenges such an insufficient number of books to meet all youth’s reading levels, unclear procedures for sharing books across classrooms, too little time scheduled for this activity, and staff taking a passive role and failing to ensure that youth were engaged. Many of these issues, particularly related to materials and scheduling, improved by the time of later observations.

The following quality ratings improved from fall to spring: read alouds (from 3.35 to 3.58), book discussions (from 2.82 to 3.14), and writing (from 3.02 to 3.12). Ratings decreased for skill development (from 3.20 to 2.71) and vocabulary (from 2.61 to 2.03).

Read alouds and independent readings were observed more often than other strategies. Read alouds occurred during 80% of observations and independent reading during 88%, as opposed to writing during 62% of observations and book discussions during 57%.

Staff felt that using a literacy model (either Kidzlit or YET) contributed to effective implementation of balanced literacy programming; models brought consistent, articulated goals and strategies and allowed training to be focused and concrete.

During the 1st year of balanced literacy programming, 59% of the groups (n = 33) were assigned to Literacy Profile 1, indicating that they failed to implement moderate-quality read alouds and independent reading during at least half of the observations. Of the remaining groups, almost all (36% of the total, n = 20) fell into Profile 3, indicating that they implemented moderate-quality read alouds, independent reading, and one or two other strategies at least half of the time. The remaining two fell into Profile 2, indicating that they implemented read alouds and independent reading at a quality rating of at least 3 during at least half of the observations.
Recruitment/Participation Some parents noted in focus groups that they chose CORAL specifically because of the literacy focus, while others identified enrichment or homework help as a key attraction.

Total CORAL enrollment for the year was 5,321, ranging from 585 to 2,081 across cities.

Most participants were in elementary school, mainly first through fifth graders (81%), with a smaller proportion in middle school.

The racial/ethnicity breakdown of participants was 68% Hispanic, 14% African American, 10% Asian, 4% multiracial, 3% White, and 1% “other.” More than half of participants (53%) were English Learners and 89% received free or reduced-price lunch.

CST scores for spring 2004 indicated that only 16% of CORAL children met the grade-level standards for proficiency on the English Language Arts portion of the test.

Of the CORAL third and fourth graders administered the IRI, 70% read below grade level in fall 2004 (with 50% of these two or more grade levels below).

Teacher assessments indicated that, in their daytime classrooms, 94% of third and fourth graders in the CORAL research sites were never or rarely overly aggressive, 99% sometimes to almost always engaged in positive social relationships, and 90% sometimes to almost always displayed cognitive concentration.

Surveyed youth reported having positive adults in their lives who cared about them and were available for support (98%) and liking school (90%). They reported that they enjoyed reading (mean = 4.4 on a scale of 1–5) but were less comfortable with their ability to do well in reading; the mean for reading efficacy was 3.7 (on a scale of 1–5).

For the 2004–05 school year, the 1,120 CORAL third and fourth graders at research sites attended on average just under 3 days per week for just under 3 hours per day. The average number of days attended was 81, with 51% attending more than 75 days. The average attendance rate (days attended out of available days the program was open) was 62%.

Retention rates showed that 79% of youth remained in the program as of June 30, 2005.

More than half (53%) of the third and fourth graders who were administered the IRI in the fall and again in the spring attended literacy programming 3 or more days a week between IRI assessments; 36% attended 2 to fewer than 3 days per week during that period; 10% attended 1 to fewer than 2 days per week; and only 1% attended less than 1 day per week. These youth attended an average of 2.31 days per week of literacy activities.

Third and fourth graders attended literacy and homework help at almost equal rates, at an average of about 63 days, between October 4 and June 8, or roughly three fourths of the time they were at CORAL. They attended enrichment programming slightly more than half of the days (45 days) they participated in CORAL. In addition, they attended physical education an average of 26 days, other academic activities an average of 10 days, and field trips an average of 5 days.
Staffing/Training Average staff age was 27 years old, and three quarters of staff were female.

Forty-six percent of staff were Hispanic, 17% were African American, 16% were Asian or Pacific Islander, 13% were White, 6% were mixed race/ethnicity, and 2% were “other.”

Of direct staff, 49% had a high school diploma but had not yet completed college; many were studying to be teachers.

Across cities, 65% of direct staff reported speaking a language other than English sometimes or always with CORAL youth and their parents. If a staff member did not speak the same language as a parent, he or she often had a colleague act as a translator.

Site staff had worked an average of 17 months in the program, and 22% reported that they had worked in CORAL for longer than 2 years.

In all cities, enrichment was provided by a variety of staff: school day teachers, CBO staff, community members, and team leaders.

Adult support was the strongest aspect of activities across all observations, with an average quality rating of 3.53 in the fall and 3.76 in the spring. In general, staff were observed to be skilled at responding effectively to youth and interacting with them in a warm and engaging manner. Many positive instances were observed of staff providing extra help for struggling youth, taking an interest in youth’s interests and thoughts, smiling and laughing with youth, and being responsive to youth’s individual needs.

Instructional and group management strategies were implemented at a moderate level (on average, instructional quality ratings were 3.03 in the fall and 3.08 in the spring; group management ratings were 3.46 in the fall and 3.54 in the spring), with large variations across groups. In the highest rated groups, staff provided clear instructions, taught organized lessons, employed strategies to motivate and challenge youth, and had activities ready when youth finished a lesson. In other cases, staff did not use strategies effectively. For example, a few instructors struggled to manage youth behavior. In these cases, staff spent their time attempting to focus and quiet youth and prevent outbursts, sometimes to such an extent that they were unable to implement any balanced literacy strategies.

Successful connection-making strategies were observed the least frequently of the quality areas observed, with an average rating of 2.61 in the fall and 2.71 in the spring. In the stronger cases, staff devoted more time to this element, such as having youth role play scenes from read aloud texts. Observations with low ratings generally fell into two categories: either the connection-making activity was so brief that it was unlikely to have much impact on youth, or it was well-intentioned but not strongly implemented.

A major adjustment needed in order to implement balanced literacy was hiring a literacy director (an expert with a solid literacy background, skills to train and support site staff in literacy programming, and ongoing time and authority to monitor quality). Two cities filled this position early in implementation, while the other three did not do so until later. Early hiring cities benefited from having a literacy director on board before literacy programming began. In those early months, the literacy director was able to develop and implement a balanced literacy training curriculum, oversee lesson-planning development, and provide early monitoring and feedback. The literacy director’s role in monitoring and quality control was a key factor contributing to higher-quality implementation in the early hiring cities. Monitoring elevated program quality when the literacy director made frequent site visits to observe programming.

Staff survey results indicated that 83% of direct staff attended training in the balanced literacy model. Scheduling training was a challenge, because staff who provided literacy activities were employed part-time and had availability constraints. Moreover, in some cities, programming occurred 5 days per week, and any weekday afternoon time taken for training would result in reduced youth programming. Cities went to great measures to compensate for these restrictions, including holding training on evenings and weekends.


Summative/Outcome Findings

Academic Third and fourth graders showed significant (p < .0001), but modest, gains between the fall and spring IRI. Overall, youth increased about a third of a grade level (0.31) in reading.

Youth who were two or more levels below their grade level at the fall IRI assessment (n = 189) improved approximately three quarters (0.78) of a grade level. Youth who were one level below their grade level (n = 81) showed an average gain of a little over a quarter of a grade level (0.27). Youth at or above their grade level in reading (n = 113) showed an average loss in reading levels over the time period (-0.43), but the average decline did not result in them falling below their grade levels.

Youth in Literacy Profile 3 groups showed greater reading gains (average gain = 0.45 grade levels) than youth in Literacy Profile 1 or 2 groups (average gains = 0.26 and 0.28 grade levels, respectively).

Youth who felt a strong sense of reading efficacy, as measured on the youth survey, showed stronger reading gains than those who felt a lower sense of reading efficacy (p <. 0001). No other subgroup differences in reading gains were found (i.e., gender, grade, race/ethnicity, English language proficiency, frequency of attendance, balanced literacy participation).

Of youth two or more grade levels behind, those in Literacy Profile 3 groups had greater average reading gains: 1.0 grade levels compared to 0.49 in Profile 2 and 0.73 in Profile 1. Youth who were one grade below grade level showed the greatest average reading gains in Profile 2 groups (1.00 grade levels) and the least in Profile 1 groups (0.13 grade levels); youth in Profile 3 groups showed average gains somewhere in between (0.47 grade levels). Youth who started at or above grade level tended to drop over time but those exposed to higher quality literacy instruction had a smaller drop than those exposed to lower-quality literacy programming; Literacy Profile 3 youth at or above grade level only dropped 0.17 grade levels on average compared to 0.62 for Profile 2 and 0.58 for Profile 1.

Results suggest that how well instructors implemented independent reading strategies was an important predictor of youth’s reading-level gains. However, the strength and consistency of general instructional practices, group management practices, adult support for youth, and connections made for youth between their experience and the text were not predictors of greater reading gains.

Several parents noted in focus groups that they were encouraged by improvements they saw in their children’s reading since enrolling. In addition, several noted changes in their children’s attitudes toward school and reading. Parents said their children were more structured and responsible in their study habits, developed an interest and joy in learning, and began to trust themselves and their abilities to do schoolwork.
Youth Development Some parents noted in focus groups that their children were more passionate about particular subjects than before they participated in CORAL, such as singing or drawing, which they first learned in CORAL.

Parents in focus groups mentioned seeing improvements in their children’s social skills and believed that those changes helped their children become better students.

Parents in focus groups expressed their appreciation for the physical aspect of some enrichment activities, noting that it improved their children’s overall health.

Parents in focus groups spoke often about the positive effects of enrichment programming and expressed appreciation that CORAL exposed their children to activities they could not otherwise have provided and opened up the world to their children.

 

© 2010 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project