News, data and conversation
about schools in New York City.
WNYC’s participation is supported by
Explore the News

Test Driving a Pilot Teacher Evaluation System

P.S. 100 Principal Katherine Moloney looks at examples of student work to determine whether fourth grade teacher Arielle Lutzer (in background) is effectively engaging her pupilsBeth FertigP.S. 100 Principal Katherine Moloney looks at examples of student work to determine whether fourth grade teacher Arielle Lutzer (in background) is effectively engaging her pupils
6 Comments
Respond

March 14, 2012, 5:56 a.m.

WNYC Listen to how one school uses the latest method to evaluate its teachers.

As an experienced principal, Katherine Moloney knows that good teachers can make a lesson out of almost anything. So she was delighted when she saw a lively class of fifth graders at Public School 100 The Coney Island School using recipe books to construct their own math problems with fractions.

“We’re making Mexican-style hash,” one girl told the principal. Her group was planning to serve this imaginary meal with chocolate chip ice cream cake, and the kids were writing down how much of each ingredient was necessary.

Ms. Moloney spent about 20 minutes talking to students and looking at bulletin boards. “The work that’s going on in there, the discussion, the excitement, that’s what good teaching is about,” she said, as she left the classroom.

Ms. Moloney has been testing a new framework for evaluating teachers this year at the school, which is actually in Brighton Beach, after receiving training over the summer. It was designed by Charlotte Danielson who wrote a common-sense framework to help both teachers and administrators identify good teaching.

It’s similar to a tool kit, with 22 strategies every teacher should master. The city is trying out the Danielson framework at 107 schools to learn how much training principals need so they can become certified evaluators once the state’s evaluation system goes into effect, said Kirsten Busch, executive director of the Office of Teacher Effectiveness. The city has until next January to negotiate an evaluation system with its teachers’ union.

At P.S. 100, Ms. Moloney and her teachers believe classroom observations are much more valid than a controversial rating system the city used that was based solely on student progress on state exams.

When the city released its teacher data reports, the fifth-grade teacher who was using recipes to teach math, Nicole Weingard, got one of the lowest scores in the school. She received just an 18 for her effectiveness at teaching math, putting her in the bottom fifth citywide. Her English score was even lower. But most of Ms. Weingard’s students easily passed their state exams.

“I’ve been teaching for eight years,” she said. “I can probably count on one hand how many of my students didn’t perform well.”

Ms. Weingard most likely got low marks because the city’s rating system put a greater emphasis on progress than student performance. She teaches the honors class, and there wasn’t a lot of room for her high-scoring students to improve.

Education officials agree that test scores, alone, are not a sufficient way to rate a teacher. That is why New York State is using its $700 million federal Race to the Top grant to develop a new teacher evaluation system in which test scores will count toward 40 percent of a teacher’s rating.

The other 60 percent will come from observing teachers at work.

New York City is considering using the Danielson system for that 60 percent. As a participant in the study, Ms. Moloney visits each of her teachers about four to six times over the year, focusing on a few of Ms. Danielson’s strategies at a time. This month she is looking at student engagement. She carries a clipboard with examples of good and bad engagement techniques and writes down what she sees, talking to students and looking at bulletin boards for examples of their work.

In a fourth-grade classroom, she was struck by the way the teacher, Arielle Lutzer, asked her students to create a “parking lot” on the bulletin board, a place for them to write down whether or not they were confused during a lesson. Later, Ms. Moloney sat down with Ms. Lutzer for about 15 minutes and asked her questions. Ms. Moloney told the teacher that she thought she was “highly effective” at student engagement.

Ms. Lutzer said she appreciated instant feedback. But she acknowledged these observations can be a little nerve-wracking.

“When someone comes in, it might be the 15 minutes that you don’t want them to see, maybe it’s the 15 minutes where you just finished disciplining someone and getting back focus, or starting a lesson and it’s confusing, or in transition,” she said.

Ms. Moloney said the Danielson framework helped take the edge off an evaluation process that could easily seem subjective. If she sees a teacher struggling with classroom management, she can send him or her to watch another teacher who’s doing a good job.

“So it’s not just a rating and leave you hanging,” she said. “It’s a rating with discussion, with feedback and then with next steps.”

Currently, the city’s teachers are formally observed once or twice a year and the vast majority are rated as satisfactory.

When Ms. Danielson’s framework was tried in Chicago, a study found about 8 percent of teachers ranked at the lowest level, more than under the city’s current system. Ms. Danielson said low-rated teachers could get better. Her method was intended to help supervisors figure out how individual teachers can improve.

But as cities and states rush to adopt new evaluation systems, she fears they might lose sight of that goal.

“So that worries me,” she said. “That in their zeal, and under the pressure to do this quickly, that school districts will cut corners and will in fact boil the complex work of teaching down to a simple checklist that they can quickly frame people on.”

Beth Fertig is a senior reporter at WNYC. Follow her on Twitter @bethfertig

6 Comments

Respond
Picture?type=square
Diana Williford March 14, 2012, 4:04 PM

There was a National Test that was supposed to be implemented back in the early 80s...I drove 3 states away to take it...It was a pretty fearsome test...never knew what happened to it...find that arrangement and save some $...of course...testing teachers will not solve our educational problems...we need to eliminate High Schools as we know them...do what other countries do...training programs early before they lose interest...also college coursework for teachers should be revamped...it is an easy degree...a rich girl I went to HS with graduated from college with me and she couldn't spell, write, do math, know any science, etc...the coursework is too easy. HSs are too political and the hiring political...a great teacher friend of mine was about to get tenure back in the day, when a board members daughter graduated with the same degree...they let my friend go and hired another numb blonde.

Add Reply
Picture?type=square
Molly Sackler March 14, 2012, 5:14 PM

Why doesn't Ms. Fertig mention or investigate the following crucial sentence in the newly mandated teacher evaluation plan: "Teachers rated ineffective on student performance based on objective assessments must be rated ineffective overall."
You can read it here: http://www.oms.nysed.gov/pres...

So the seemingly reasonable basis for a teacher's rating -- 40% based on students' test scores, 60% based on observation -- is a fallacy. If a teacher's students perform ineffectively on standardized tests, the teacher will be rated ineffective.

Add Reply
Picture?type=square
Michael Hayman March 14, 2012, 5:51 PM

Notwithstanding the glaring fallacy of failing to write that being rated ineffective in the 40% category of "teach to the test" earns even the best teacher an overall ineffective, this article, as well as most others assumes that the official discussion of teacher evaluations has something to do with evaluating teachers; it does not. The longer teachers and our associations continue to regard this evaluation blather as good coin, the longer we will fail to defend public education and our profession from the political and corporate interests determined to strip away the rights of teacher unions and to funnel taxes to private schooling ventures. As a recently retired teacher who was always keenly and obsessively concerned with the quality of my lessons and my students' understanding the material I worked to convey, I can say without doubt that teachers can certainly be evaluated objectively, but the aims of government and the school administrators currently in the saddle are to whittle down the control teachers exercise in the classroom and use real and perceived failings to eliminate opposition to the attack on public education.

Add Reply
Picture?type=square
Kevin Gray March 14, 2012, 6:46 PM

Just a thought: If I were a school principal, responsible for years for conducting 1 or 2 observations a year of each teacher in my school, and next year the rules have changed so that I must conduct 4-6 observations each year, with more deep and broad follow-up discussions, reflections, and next steps, what happens to all the things I would have been doing with all the time I'd now be spending doing 4 or 5 more observations per teacher per year? If there are 60 teachers in my building (large schools can have as many as 300-350), that's a lot of time spent confirming the good teachers are good, and the ones that need help need help. The follow-up is promising, but without enough clerical support for school leaders, this smells like a fast road to burnout to me. These initiatives are rarely discussed in terms of systemic impacts, and while there are some positives, the negatives are minimized and intentionally ignored. Then, when they predictably and inevitably derail the initiative, we all race to the next spinning top. Let's stop racing, and instead, do one sensibly and with the full insight of professional educator input, at least once. Please?

Add Reply
Picture?type=square
Chloe Graef March 18, 2012, 2:05 PM

First, I agree that a meaningful evaluation system for teachers will be extremely beneficial to all of us who work in education.

However, I also completely agree with the above posts that point out the fallacy of saying that State ED officials recognize that state tests are not as important as observations. There may be some officials who do, but they are not the ones who designed this current APPR system.

Further, the current system will label a teacher as ineffective based on test scores alone (40% of their score) - ignoring the fact that the students taking the tests have no incentive to apply themselves (this is obviously less true of students in high school but absolutely true from 8th grade down.) So beyond the complex and often erroneous nature of the calculation process, the numbers themselves will also be an inaccurate reflection of the student's ability and knowledge and therefore the efficacy of their teacher. Often, the brightest and most prepared student becomes bored in a standardized test and decides to either ignore it or play games with it. A teacher's career should not hang in the balance of a child under 14 choosing whether to take a test seriously or not.

I agree that there needs to be accountability in education, but pushing through a system that will fundamentally harm the people who work in the institutions that it is designed to improve just doesn't make sense.

Currently, the APPR system must be implemented by September 1 without any time for piloting before it will go into effect and be made known to the public despite the acknowledged errors involved in the recent numbers and ratings posted for 4th through 8th grade teachers.

I have taught in public and private schools in other countries and have found the best education situation for teachers and students in a NY State public school. Public education does work when the entire community from the local to the state to the federal government supports it. This new APPR system is doing the very opposite and threatens to fundamentally diminish the quality of education in the places where it is really working.

1 Reply
Picture?type=square
Jakotora Tjoutuku March 19, 2012, 7:46 PM

Add Reply
Picture?type=square
Adam Grumbach March 15, 2013, 1:32 AM

Lots of issues to sink one's teeth into. First, evaluation by students test scores, even under (especially under) a value added model (VAM) is not valid -- the margin of error in NYC was +-30 points. If your doctor weighed you on a scale with a margin of error of 30% of your body weight either way and then recommended a new diet, you'd think he (she) was nuts, and you'd be right.
So, reducing those scores to "merely" 40% of the overall evaluation is ridiculous -- they should play no role at all.
Beyond that, Danielson's system of evaluation may (or may not be) common sense, as Fertig write. But these systems are being put in place in bad faith, by politicians and administrators who have demonstrated repeatedly that they don't really care about educating students as much as they do about "holding teachers accountable." Danielson's rubrics are likely going to be misused, whatever her intention (and once upon a time, before she began raking in millions from various states, she explained that her rubrics were intended for professional development, not evaluation and "accountability"). The problem is not so much the tool, as the folks who are wielding it. That they have insisted on using VAM scores with such a huge margin of error, even if only for 40% (and, as other commenters have noted, that 40% can count for 100% in some cases), should give us all pause before assuming good intentions on their part when they turn to Danielson's rubrics.

Add Reply
Add a Response
SchoolBook Bulletin Board
Welcome to SchoolBook

Schoolbook is a site dedicated to news, data and conversation about schools in New York City.

Have a News Tip?

Tell us what’s going on in your school. You can e-mail us with your tips or documents, or call 646-801-9698 and leave a voice message.

Contribute to Current & Future News Coverage

Join the Public Insight Network and help our journalists cover education in the city. Your stories and insights can help us create relevant and distinctive reporting. Join more than 100,000 people and become a trusted source.