Program Reviews a shift from test-centered to program-centered


Art teacher George McKee helps then 4th-grade student Ashley Howell create an African mask at E.P. Ward Elementary School (Fleming County) May 28,  2010. E.P. Ward Elementary is one of 48 schools from 34 districts that piloted Program Reviews in writing, arts and humanities, and practical living/career studies this past spring.  Photo by Amy Wallot
Art teacher George McKee helps then 4th-grade student Ashley Howell create an African mask at E.P. Ward Elementary School (Fleming County) May 28, 2010. E.P. Ward Elementary is one of 48 schools from 34 districts that piloted Program Reviews in writing, arts and humanities, and practical living/career studies this past spring. Photo by Amy Wallot
By Matthew Tungate

Terra Greer, principal at E.P. Ward Elementary School (Fleming County), learned something interesting while piloting Program Reviews, a new way for measuring how well Kentucky schools are teaching students writing and arts and humanities. She could not find evidence to demonstrate how well her school was providing students with opportunities in the arts and humanities.

“In light of the program review results, we will be making some slight changes to the writing program, but our arts and humanities program will be undergoing a whole new scope and sequence,” Greer said.

Senate Bill 1 requires the Kentucky Department of Education to implement a new assessment and accountability model for public schools by 2011-12, and Program Reviews are part of that model. Even though neither the accountability model nor the role Program Reviews will play in it have been determined, 48 schools from 34 districts tested Program Reviews in a combination of writing, arts and humanities, and practical living/career studies, according to Rae McEntyre, assessment liaison with the state Department of Education.

Rather than testing students with paper and pencil to see what they have learned in the three areas, schools gathered evidence about how they integrated the subjects across their curricula and whether they provided students with opportunities to learn, among other things, McEntyre said.

To conduct a program review, a school forms a team of administrators and teachers to look for evidence of specific events or occurrences, called characteristics, expected in a school functioning at high levels. An example would be “students will be taught by teachers who are certified in the subject area they teach,” McEntyre said.

Several characteristics would establish a demonstrator, which is a broader trait describing what is happening in a school considered fully functioning, such as “students have access to a quality arts program,” McEntyre said. Several demonstrators lead into four standards: curriculum and instruction; formative and summative assessment; professional development and support; and administrative/leadership support and monitoring. The standards then establish the quality of the overall program.

The school team will identify evidence and establish a rating at each level from fully functioning to no implementation using a green, yellow and red scale.

“We are telling teachers, ‘Use your professional judgment,’” McEntyre said. “We know that every school is different, and they know what resources they have in their building.”

Greer said her team found the Program Reviews “extremely valuable.”

“We learned we were meeting many of the criteria of the demonstrators, but we may not have been keeping the evidence,” she said. “Or we may not have been meeting a demonstrator and could see immediately areas of need or concern. Schools and districts want immediate feedback; a program review gives you just that, and the next steps must be decided within the school’s leadership team.”

Lincoln County Middle School, which piloted arts and humanities and practical living/career studies Program Reviews, also will be making changes based on its results, according to Academic Program Consultant Sarah Hagans.

“Throughout the process, we had many great discussions that had us really thinking about what and how we teach as well as what and how we assess,” Hagans said. “The reviews have sparked many thought-provoking conversations that have impacted our improvement plans for our programs in our school. We’ve already begun making changes based on our results.”

As part of the pilot program, Montgomery County High School’s team found it has areas for improvement in its practical living/career studies department, according to guidance counselor Lacy Carrington-Gross.

“We need to equally distribute better technology throughout the different departments, work on having more collaboration going on throughout the school between different departments and seek more opportunities to have outside agencies, business and community partners get involved in our programs,” she said.

The Program Reviews help schools improve more than seeing results from the Kentucky Core Content Test, Hagans said.

“Because the Program Reviews are evaluations rather than tests, the data that you get is more about how to improve your programs rather than a report of how students did and what content strands are weak,” she said. “So our ‘data’ was a list of program characteristics and demonstrators that we did well on as well as those that we needed to target for improvement.”

Hagans, a former arts and humanities teacher, said one of the biggest changes teachers will see is that writing, arts and humanities and practical living/career studies are no longer broken down to just test results.

“We’re all used to prepping all year and administering a test at the end, getting a score months later and equating that with school performance,” she said. “With the Program Reviews, the focus is on what characteristics of your program are your strong areas and which ones are your weak ones, and then taking that information to begin improving your program periodically throughout the year.

“Overall it’s a shift from being test-centered to being program-centered.”

Schools also need to be prepared for how long it takes to complete the reviews, Carrington-Gross said.

“It involves a lot of time outside of the classroom and meeting together to go over each standard,” she said. “Sometimes we feel that something like this is another thing that we had to complete, but our district was willing to support us and provide subs for the teachers for the days we needed to complete the review.”

Greer said schools also will be challenged to compile evidence if no system is already in place.

“It’s not enough to just assume identified characteristics are occurring in a program,” she said. “Evidence should be available to show continuity across grade levels.”

Aside from evaluating their own programs, schools also were evaluating the program review process, McEntyre said.

She said she expects revised versions of the Program Reviews to be available in the next month or two. Schools may begin using the Program Reviews to rate their writing, arts and humanities, and practical living/career studies programs before they are held accountable for them in the 2011-12 school year, McEntyre said.

She encouraged school administrators to begin early and to work with all teachers in their respective buildings.

“It’s not just art-specific or PE (physical education)-specific or career studies-specific. All the teachers are actually working together to give students those skills to be successful,” McEntyre said. “It’s not a matter of these other content teachers teaching the arts or teaching health, but how can they make connections to those other programs in their own content, where applicable.

Program Reviews
Rae McEntyre,, (502) 564-2106