Editor’s note: Senate Bill 1 (SB 1), enacted in the 2009 Kentucky General Assembly, requires a new public school assessment program beginning in the 2011-2012 school year. Kentucky Teacher is doing a series of stories explaining the Unbridled Learning: College/Career Readiness for All assessment and accountability system this month. This article focuses on high school assessment. A future story will focus on accountability. The system is subject to United States Education Department approval and may be changed prior to adoption.

By Matthew Tungate
matthew.tungate@education.ky.gov

Sophomores April Luttrell, Micah Coppage and Destiny Davis listen to Chris Carter teach about different cells types at Ohio County High School. Photo by Amy Wallot, Dec. 8, 2011

Sophomores April Luttrell, Micah Coppage and Destiny Davis listen to Chris Carter teach about different cells types at Ohio County High School. Photo by Amy Wallot, Dec. 8, 2011

Following English II, Algebra II, Biology and U.S. History classes, high school students will take end-of-course assessments that will be used in the new Unbridled Learning: College/Career Readiness for All assessment and accountability system – and count as part of their grade.

This is the first time Kentucky public school students’ performance on the state’s assessment and accountability tests may affect their grades. The end-of-course assessments for high school students may count at least 20 percent of a student’s overall course grade or school districts will have to explain why not.

Commissioner Terry Holliday told Kentucky Board of Education members in April that Kentucky was the only southern state that didn’t use end-of-course tests. Board member Roger Marcum, a former teacher, principal and superintendent, said a lack of student accountability has caused complaints for a long time among educators.

The idea for using end-of-course tests is that all Kentucky students will have the four courses during their high school careers, said Rhonda Sims, a director in the Kentucky Department of Education’s Office of Assessment and Accountability.

“It’s a really different place than we have been before, because at high school it used to be you herd all your 10th graders in a room and you give them a test, you herd all your 11th graders in a room and you give them a test,” she said. “And now, it’s linking testing to what is happening instructionally for that child.”

But testing is only part of the Unbridled Learning: College/Career Readiness for All assessment and accountability system. The accountability model incorporates all aspects of school and district work. It includes student data from testing, gap, growth, college/career readiness and graduation rate (Next-Generation Learners); principal and teacher effectiveness (Next-Generation Professionals); and Program Reviews (Next-Generation Instructional Programs and Support). Senate Bill 1 (SB 1), enacted in the 2009 Kentucky General Assembly, requires a new public school assessment program beginning in the 2011-2012 school year.

This year, only the Next-Generation Learners component will be used for school and district accountability.

What are the new tests?

Sims said the new end-of-course exams will have to have a combined score between a multiple-choice test that students can take online and get immediate results from, and constructed-response answers that won’t get scored until summer. The two components will be combined for a score.

Most students will take English II, Algebra II and Biology as sophomores, and U.S. History as juniors, Sims said.

Students also will take the on-demand writing tests and the PLAN test as 10th-graders. The on-demand writing is stand-alone, passage-based and customized for Kentucky, she said. The on-demand writing test will be given at the end of each school year.

PLAN serves to test students in editing and mechanics and also shows their progress toward college/career readiness on the ACT.

Students will take the ACT in 11th grade. The ACT is the capstone measurement for determining whether students are considered college/career-ready in the new assessment and accountability system.

What happens to the results once students take the test?

Students will be identified as novice, apprentice, proficient or distinguished based on their performance on the end-of-course and on-demand writing tests, Sims said.

“The standard-setting of when a student is novice, apprentice, proficient or distinguished has to occur after we have test results, because otherwise we’re arbitrary in setting the definition for novice performance when we  don’t have the current test results,” Sims said. “We can’t assume that on the new test, where we say the standards are higher and tougher and harder, we will be at the same level of performance and the same distribution.”

High school students’ results will be used in two of the Next-Generation Learner areas: achievement and gap.

Achievement is similar to grades in that a minimum score will determine whether a student is distinguished, proficient, novice or apprentice, Sims said.

The Kentucky Board of Education decided last year that schools would receive one point for every percentage of students scoring proficient or distinguished for each of the content areas (reading, writing, mathematics, science and social studies). Schools will receive half a point for each percent of students scoring apprentice and no points for students scoring novice.

Schools also are eligible for bonus points. Each percentage of students scoring distinguished earns an additional half point, and the percent scoring novice loses a half point.

“If you have more distinguished kids than novice, then that difference becomes a bonus that’s added,” Sims said. “If the reverse is true and you have more low performers than high performers, nothing happens. You don’t lose points. It was important to the board, while recognizing the highest performance of our students, that we not lose our focus on improving our lowest performers.”

The second area on which schools will be judged in is gap. Under the new accountability system, African-American, Hispanic, Native American, special education, low income and limited English proficiency students will be combined as one group for calculation of the achievement gap and school gap-reduction goal. However, scores for the various demographic groups would still be reported individually.

“It would hold all districts accountable for all children,” Commissioner Terry Holliday said.

The gap score is based on the percentage of students who score proficient and distinguished. Unlike the previous assessment and accountability system, however, the new system counts students’ scores only once, even if they fit into multiple gap groups.

The department is concerned about the potential masking of individual gap group scores even though all gap groups will be reported, so the accountability portion of Unbridled Learning takes that into consideration. All schools with gap groups underperforming by a certain percentage will face state consequences.

Sims said the consequences for underperforming individual gap groups are a new addition because of Kentucky’s request of a waiver from federal No Child Left Behind (NCLB) Act requirements.

For high school students, growth is determined by how well students do in reading and mathematics on the PLAN and ACT tests. The growth calculation is designed to measure a student’s learning growth each year as compared to the student’s academic peers across the state. So, higher-performing students’ scores are compared against other high-scoring students across the state, while lower-performing students’ scores are compared against other lower-scoring students’ scores.

Points are awarded for the percentage of students who grow at typical or higher levels relative to their peers, with typical growth defined as at the 40th percentile or higher.

A computer program generates the results of all Kentucky students who took the same test, Sims said, so schools won’t be able to calculate their own growth.

The results may surprise some schools, especially those with low expectations who think their students are growing until they are compared to students from schools with high expectations, she said.

“There you might be able to ask instructional questions  about are you teaching the standards, are you linking back to the standards, are you at the level the standards expect, if you’re not seeing growth,” Sims said.

The growth model also will show high achievers who are just coasting, she said.

“Because when I look at him in comparison to all other kids scoring right up there at the top, he might not have shown growth,” Sims said.

The growth part of the Unbridled Learning calculation is “the hardest thing to understand,” she said.

Determining college/career readiness

The next measure that affects high schools’ accountability scores is college/career readiness.

For high schools, readiness for college or career will be calculated by dividing the number of high school graduates that have successfully met one of the indicators of readiness by the total number of graduates. The indicators of readiness include:

  • the Kentucky Council on Postsecondary Education’s Systemwide Benchmarks on the ACT in reading, English and mathematics
  • the Kentucky Council on Postsecondary Education’s College Placement Test Benchmarks
  • the Career Measures as defined by the Kentucky Department of Education

Students who meet an established score on ACT, COMPASS or Kentucky Online Testing (KYOTE) are considered college-ready.

The Kentucky Board of Education also defined what it means to be career-ready.

The board agreed that a student must meet one requirement each in the areas of Career Academic and Career Technical to be considered career-ready. Career Academic includes ACT WorkKeys and Armed Services Vocational Aptitude Battery (ASVAB) assessments; Career Technical includes Kentucky Occupational Skill Standards Assessment (KOSSA) and industry certificates.

Under college and career readiness, schools and districts receive a point for each percentage of students who are considered college- or career-ready. The board agreed to give half a point extra to students who are both college- and career-ready under the Readiness for College/Career indicator, which can total no more than 100.

 The final part of the high schools’ Next-Generation Learners score comes from graduation rate. Kentucky is using the Averaged Freshman Graduation Rate (AFGR).

The AFGR formula divides the average of prior years’ 9th- and 10th-grade enrollment by the number of four-year diploma and more-than-four-year diploma recipients in the current reporting year. Students with disabilities whose Individual Education Plans enable them to take more than four years to obtain a diploma are included in the calculation.

“We know this is not a perfect formula,” Sims said. “If kids move and factories close, you could have a big drop in your student population through no fault of not graduating.”

 Kentucky is moving to a new calculation that tracks individual students across schools and districts to give a more accurate number, beginning with the graduating class of 2013.

What happens to the test results?

Points earned from each of the five areas – achievement, gap, growth, college/career readiness and graduation rate – will be added to calculate a score, each with its own weight. End-of-course exam and on-demand writing results will be used to calculate achievement and gap. Growth will be based on PLAN to ACT reading and mathematics scores.  

 Until the other components are completed, only the Next-Generation Learners component will be used to generate an overall score for accountability. The following chart provides the overall score phase-in for the three components.

 When will the other two areas of accountability be included?

Program Reviews, as part of Next-Generation Instructional Programs and Supports, will be added to the calculation in 2012-13. Rather than testing students to see what they have learned, Program Reviews require schools to gather evidence about how they integrate subjects across curricula and provide students with high-quality learning opportunities. The schools then use the information to improve programs.

Schools piloted Program Reviews during the 2010-11 school year in three areas: arts and humanities, practical living/career studies, and writing. Those Program Reviews are being field-tested and the results will be made public this school year. Full accountability for the three Program Reviews will begin in the 2012-13 school year.

Program Reviews in world language and K-3 will be added in future years.

In the 2013-14 school year, Kentucky will implement its teacher- and principal-evaluation system as part of the Next-Generation Professionals section.

During the December Kentucky Board of Education meeting, Holliday told the board that the system must have six components to meet federal waiver standards:

1. continuous improvement of instruction

2. meaningful differentiation of teacher/principal performance using at least three levels

3. multiple measures of effectiveness including the use of student growth data (both state standardized tests and formative growth measures that are rigorous and comparable across schools in a local district) as a significant factor

4. regular evaluation (most likely annual)

5. clear and timely feedback to include opportunities for professional development

6. use of the system to inform personnel decisions.

Kentucky’s plan meets these standards, including the multiple measures – and the use of student growth data, he said.

“If we get the waiver, there is no system that could ignore student growth on standardized tests as part of the evaluation system,” Holliday said.

The board’s most difficult discussion to come is the weighting of each of the multiple measures and how much flexibility local districts will have to use their own evaluation systems, he said.

“The neat thing about the new system is you’ve got all these different sort of looks or lenses that allow you to look at  kid performance and school and district performance, whereas before it was all very focused just on achievement,” Sims said. “And are these things kind of correlated to each other? Well, of course they are.”