News, data and conversation
about schools in New York City.
WNYC’s participation is supported by

Peer Groups Do Not Improve Grading System

3 Comments
Respond

Oct. 14, 2012, 9:53 p.m.

How similar is your school to mine?

I recognize that this seems like a fairly complex question to answer but it hasn’t stopped the Department of Education from trying. I believe education officials rely too much on the peer indices they create to cluster schools into peer groups for comparison. The result muddies the accomplishments and challenges of individual schools.

The peer index consists of the following components: students with economic need, weighted 30 percent; students with Individualized Education Plans, weighted 30 percent; students who are Black or Latino, weighted 30 percent; and students who are classified as English Language Learners, weighted 10 percent.

Let us first acknowledge the problematic notion that we can place a proportional value on a child’s need; I hope that you share my distaste for any attempt to quantify how much having a disability or being of a particular racial background influences a child’s educational potential.

But let us also focus on how this questionable methodology leads to the comparison of fairly dissonant educational contexts, and how critical the formulation of these peer groups is to a school’s performance on its progress report. The 40 or so schools with similar composite peer indices are grouped together, and 75 percent of a school’s progress report grade is contingent upon the comparisons drawn between the schools within its given peer group.

In other words, the schools that comprise a peer group matter a whole lot, because much of the progress reports are based off of how a given school’s “results” – namely standardized test scores and learning environment survey responses – compare to the “results” of the other schools in one’s peer group.

Given that my school is located in the heart of Chinatown and serves a largely Chinese-American student population, I had initially assumed that my school’s peer group would consist of nearby Chinatown schools, and perhaps other large schools that were predominantly East Asian American and similarly situated within working class, largely immigrant communities.

However, because the peer indices used to assemble peer groups are considered in the aggregate, and because factors like geographic location, school size, or the racial identities of non-Black and Latino students are literally not part of the equation, the distinctions among schools within each peer group can be quite striking.

Within my school’s group there are nearly twice as many Staten Island schools as there are Manhattan schools; only one other Chinatown school made the list, in addition to schools in places like Park Slope, Bayside, and Tottenville. Now, these are by many accounts wonderful places to live and go to school, but they are not likely to be confused with Chinatown.

More noteworthy is the fact that my school maintains an economic need index that is 160 percent of the peer group average, while our percentage of English Language Learning students is 248 percent of the average. This makes me question how salient a comparison one can draw between a school like mine – where nearly a quarter of the students are labeled English Language Learners and where meeting the needs of these students and their largely non-English speaking parents is one of several overriding instructional concerns – and one particular “peer” school where less than 1 percent of the students are labeled as such.

This is not to say that all schools do not face tremendous challenges in working with the diverse ranges of students and families that they serve. In fact, my point is exactly the opposite: If we recognize that schools are incredibly complex institutions that must attend to an equally complex array of individuals and individual needs, then how accurately can we actually quantify the tangible differences between any two schools? And why must our systems for accountability rely so heavily on comparison as a means of measuring proficiency?

I fully recognize the ongoing need to reflect on and reform my practice to best meet the dynamic needs of my kids. My opposition to an over-reliance on standardized assessments and faulty comparisons does not mean I oppose accountability. It just means there are other numbers that are more important to me.

I am proud of the work my colleagues and I have done this past year to support the learning and growth of our students. According to the most recent learning environment survey, 94 percent of the parents and guardians at my school said they were either satisfied or very satisfied with the education that their children received this past year.

And 94 percent is an “A” on my grading scale.

Juhyung Harold Lee is a fifth grade teacher at Public School 124 in Manhattan's Chinatown

3 Comments

Respond
Picture?type=square
Aaron Schumacher October 15, 2012, 2:18 PM

Based on the 2012 School Survey results available here: http://schools.nyc.gov/Accoun... there were 397 elementary schools where 95% or more of parents were satisfied or very satisfied with the education their children received in the past year. 220 elementary schools had 94% or less of their parents responding similarly. In that context, with the result of 94% actually worse than nearly two thirds of elementary schools, should a 94% be an "A" on the grading scale? Comparisons add context. I see in this article some evidence in support of improving the method of identifying peer schools. I'm not sure the point of the title - that peer groups do not improve the school grading system compared to not comparing to peer schools - has been supported by the provided evidence.

3 Replies
Picture?type=square
Juhyung Lee October 15, 2012, 5:38 PM

I think comparisons add context when the entities being compared are similar enough to truly delineate the distinctions between those entities as functions of the entities themselves, rather than the product of some sort of external factor that is not common to all entities in the comparison.

Comparisons also add context when the criteria upon which those comparisons are made is understood and thoughtfully considered by all relevant stakeholders, which, in this instance, I don't believe to be the case.

My intention in bringing up the rate of satisfaction among the parents in my school -- 100% of whom completed their surveys, and 0% of whom were forced to fill their surveys out in front of teachers or administrators -- is that this is one of a variety of more meaningful indicators of the quality of a school than a progress report grade, which I believe we both agree is based on, at least in part, faulty comparisons.

I also believe that the title -- while not selected by me -- speaks to a critique of the peer groups as they stand now, not necessarily to the notion of peer groups in general. While I -- in my current position as a classroom educator eager to consider how to improve my practice -- may not always find the process of comparing certain aspects of schools' standardized testing data to be particularly illuminating, I at the very least suggest that it we are to continue this rather misguided practice, we should at least be a bit more thoughtful about which schools we are comparing to each other. 

Picture?type=square
Juhyung Lee October 15, 2012, 5:39 PM

I think comparisons add context when the entities being compared are similar enough to truly delineate the distinctions between those entities as functions of the entities themselves, rather than the product of some sort of external factor that is not common to all entities in the comparison.

Comparisons also add context when the criteria upon which those comparisons are made is understood and thoughtfully considered by all relevant stakeholders, which, in this instance, I don't believe to be the case.

My intention in bringing up the rate of satisfaction among the parents in my school -- 100% of whom completed their surveys, and 0% of whom were forced to fill their surveys out in front of teachers or administrators -- is that this is one of a variety of more meaningful indicators of the quality of a school than a progress report grade, which I believe we both agree is based on, at least in part, faulty comparisons.

I also believe that the title -- while not selected by me -- speaks to a critique of the peer groups as they stand now, not necessarily to the notion of peer groups in general. While I -- in my current position as a classroom educator eager to consider how to improve my practice -- may not always find the process of comparing certain aspects of schools' standardized testing data to be particularly illuminating, I at the very least suggest that it we are to continue this rather misguided practice, we should at least be a bit more thoughtful about which schools we are comparing to each other. 

Picture?type=square
Juhyung Lee October 15, 2012, 5:41 PM

I think comparisons add context when the entities being compared are similar enough to truly delineate the distinctions between those entities as functions of the entities themselves, rather than the product of some sort of external factor that is not common to all entities in the comparison.

Comparisons also add context when the criteria upon which those comparisons are made is understood and thoughtfully considered by all relevant stakeholders, which, in this instance, I don't believe to be the case.

My intention in bringing up the rate of satisfaction among the parents in my school -- 100% of whom completed their surveys, and 0% of whom were forced to fill their surveys out in front of teachers or administrators -- is that this is one of a variety of more meaningful indicators of the quality of a school than a progress report grade, which I believe we both agree is based on, at least in part, faulty comparisons.

I also believe that the title -- while not selected by me -- speaks to a critique of the peer groups as they stand now, not necessarily to the notion of peer groups in general. While I -- in my current position as a classroom educator eager to consider how to improve my practice -- may not always find the process of comparing certain aspects of schools' standardized testing data to be particularly illuminating, I at the very least suggest that it we are to continue this rather misguided practice, we should at least be a bit more thoughtful about which schools we are comparing to each other. 

Add Reply
Picture?type=square
Ken Achiron October 16, 2012, 5:41 AM

For real? 1% difference? With no thought given to standard deviation, validity or reliability of the instrument used. On this basis we can rank schools? This is exactly the trouble with measuring schools against each other and equating competition with and winning with true improvement. Does a score of 100% favorable really mean the school is better, or does it only mean that those parents were all in agreement perhaps without knowing a true picture of what is happening, while in another school two parents who are upset over a grade or placement issue although the school has it correctly skew the results and lower the rating. But the first question still needs an answer: Is 94% really different from 95% and is it different in any meaningful way?

Add Reply
Picture?type=square
Michael Rosenthal October 18, 2012, 2:31 AM

TO me the biggest problem is the amount and extent to which schools game the system. Whether through "credit recovery programs" that do more to recover the school's credit on the progress report than to help students recover learning they missed from failed classes or exams that have magic erasure marks.

Add Reply
Add a Response
SchoolBook Bulletin Board
Welcome to SchoolBook

Schoolbook is a site dedicated to news, data and conversation about schools in New York City.

Have a News Tip?

Tell us what’s going on in your school. You can e-mail us with your tips or documents, or call 646-801-9698 and leave a voice message.

Contribute to Current & Future News Coverage

Join the Public Insight Network and help our journalists cover education in the city. Your stories and insights can help us create relevant and distinctive reporting. Join more than 100,000 people and become a trusted source.