Marshall Memo 589
A Weekly Round-up of Important Ideas and Research in K-12 Education
June 1, 2015
1. Improving teacher evaluation
2. Are teachers with the highest Danielson scores really the best teachers?
3. Using student surveys to evaluate teaching: cautionary notes
4. Keys to effective instructional coaching
5. Measuring students’ noncognitive skills
6. Separating boys and girls for middle-school anti-bullying lessons
7. The Matthew effect with educational technology
8. Which causes more academic loss, snow days or individual absences?
9. Wordless picture books as a key literacy element in kindergarten
10. Career advice from Robert Sternberg
11. George Mitchell reflects on conflict resolution
12. How people handle tensions at work
“If coaches are asked to write reports, develop school-improvement plans, oversee assessments, deal with student behavior, do bus and cafeteria duty, and substitute teach, they’ll have little time left to partner with teachers.”
Jim Knight (see item #4)
“Lately, we seem to have shifted from improving teaching to alternately blaming or idolizing teachers. We are no longer evaluating with the goal of ongoing changes in practice; we’re blinded by science and ‘metrics.’”
Nancy Flanagan (see item #1)
“We talk about how we get into that 4.7-and-above range. We talk about that more than about how to teach.”
A college professor on student evaluations of her teaching (see item #3)
“The receptors in our brain for information contrary to our prior beliefs are very narrow. It requires effort and discipline to get people to consider what the other side has to say.”
George Mitchell (see item #11)
Kentaro Toyama (see item #7)
“Six Questions About Teacher Evaluation” by Nancy Flanagan in Education Week Teacher, May 28, 2015, http://bit.ly/1eLjZye
In this article in Educational Evaluation and Policy Analysis, Rachel Garrett (American Institutes for Research) and Matthew Steinberg (University of Pennsylvania) report on their analysis of teacher effectiveness data from the Measures of Effective Teaching (MET) study. They reached four conclusions:
First, there is a strong correlation between teacher ratings on Charlotte Danielson’s Framework for Teaching (FFT) and student test scores. “On average,” say Garrett and Steinberg, “student achievement is higher among teachers who receive higher FFT ratings.”
Second, there are problems with using this correlation for high-stakes personnel decisions on individual teachers (e.g., tenure, performance pay, or dismissal). That’s because “relying heavily on FFT measures ignores one of the key drivers of this relationship,” say Garrett and Steinberg, “– the systematic sorting of students to teachers. We find consistent patterns of noncompliance with randomization that moves students to teachers with higher FFT scores. Such nonrandom sorting limits the ability of teacher performance measures to provide a valid estimate of a teacher’s contribution to student learning, thereby constraining policymakers’ and school leaders’ ability to identify truly effective teachers.”
Third, Garrett and Steinberg conclude that Danielson rubric data, while an interesting marker of teacher effectiveness, is less useful as an intervention to improve teaching. “Implicit in this distinction,” they say, “is the impossibility of either fully capturing or randomly assigning instructional quality. While better teachers, on average, may receive higher FFT ratings, there are likely other aspects of teacher quality that are salient to student learning but not measured by the FFT… Disentangling the effect of quality instruction on student achievement is further complicated by the fact that instruction undoubtedly interacts with numerous factors, including the composition of students in a given class. The mix of students on observed and unobserved dimensions will vary both across teachers within a school, as well as across classes taught by the same teacher.” That’s why individual teachers’ Danielson ratings fluctuate from year to year.
Finally, Garrett and Steinberg note that the Danielson framework was originally created to coach teachers, encourage self-reflection, and inform professional development, but the MET study and their own analysis of MET data used it only to measure correlations between teacher effectiveness and student achievement. In other words, teachers in these studies were analyzed in a way that didn’t capture the feedback conversations with principals and the subsequent professional development that Danielson envisioned – which any good school implements on a routine basis. Therefore, conclude Garrett and Steinberg, “this study does not speak to either the potential impacts on student and teacher performance when the FFT protocol is fully implemented, or how performance can be shaped over time. Indeed, this potential for professional development embedded within the compete FFT protocol is one of the compelling reasons for its use, as compared with value-added scores, which provide little guidance for teachers on how to improve their practice.”
In this Chronicle of Higher Education article, Stacey Patton reports on the debate around high-stakes use of student evaluations of college instructors. One full-time humanities professor at a West Coast research university said the number 4.7 was burned into her mind: that’s the student-evaluation score (out of 5) she needed to receive in order to feel safe in her non-tenure-track position. “Everybody in my department is obsessed,” said this veteran professor. “We talk about how we get into that 4.7-and-above range. We talk about that more than about how to teach.” She lists some of the ways she and her colleagues have tried to game the system:
“Student Evaluations: Feared, Loathed, and Not About to Go Away” by Stacey Patton in The Chronicle of Higher Education, May 29, 2015 (Vol. LXI, #37, p. A8), http://bit.ly/1LWXgcL
In this Scholastic EduPulse article, Jim Knight (University of Kansas) says instructional coaching has the potential to move schools “from cultures of talking to cultures of doing.” He suggests seven ways that principals can support and enhance this work:
• Protect instructional time. “If coaches are asked to write reports, develop school-improvement plans, oversee assessments, deal with student behavior, do bus and cafeteria duty, and substitute teach,” says Knight, “they’ll have little time left to partner with teachers.”
• Use an instructional playbook. Coaches need to “deeply understand a set of high-impact teaching strategies that will help teachers achieve their goals,” he says, beginning with the “big four”: content planning, formative assessment, instruction, and community-building. Coaches need to know the playbook backward and forward and filter district directives and initiatives to maintain focus on a small number of key teaching strategies.
• Listen to the troops. Teachers’ opinions matter, and they should be making most of the decisions about what occurs in their classrooms, working shoulder-to-shoulder with the coach to reach worthy goals. Knight believes coaching should be voluntary, since requiring it is often seen by teachers as punitive.
• Clarify roles. Coaches shouldn’t be involved in supervisory visits to classrooms or formal evaluation, says Knight: “If coaches are given administrative roles, they need to have the same qualifications and training as any other administrator, and everyone in the school (most especially the coach) needs to know they are in that role.”
• Maintain confidentiality. Trust and transparency are essential if teachers are to be forthcoming about their thoughts and concerns. “What is most important with regard to confidentiality,” says Knight, “is that principal and coach clarify what they will and will not talk about, and that the principal clearly communicates that agreement to everyone involved.”
• Meet regularly. Principal-coach meetings don’t have to be longer than 20 minutes in most cases, but frequency is essential if the principal and coach are to be on the same page.
• Walk the talk. “Principals who proclaim that professional learning is important should attend and even lead professional learning sessions,” says Knight. They might also video-record their own meetings and presentations and model the process of examining what’s working and what isn’t.
“Measurement Matters: Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes” by Angela Duckworth and David Scott Yeager in Educational Researcher, May 2015 (Vol. 44, #4, p. 237-251), http://bit.ly/1AGi9sj; the authors can be reached at [email protected] and [email protected].
In this ASCA School Counselor article, Pennsylvania counselor Lisa Fulton says she used to present lessons about bullying to girls and boys together. Over time, she noticed that many students were tuning her out. “I blamed the students for their lack of attentiveness,” says Fulton, “but I should have been blaming my approach.” Finally she realized that there are big differences in the type of bullying experienced by boys and girls:
In this article in Education Next, Joshua Goodman (Harvard Kennedy School) reports on his study of the impact of Massachusetts no-school days on student achievement. Using data on school closings and standardized test scores, Goodman concludes that individual student absences “sharply reduce student achievement, particularly in math, but school closings appear to have little impact.” He continues: “These findings should not be taken to mean that instructional time does not matter for student learning; the bulk of the evidence suggest it does. A more likely explanation is that schools and teachers are well prepared to deal with the coordinated disruptions caused by snow days – much more so than they are to handle the less-dramatic but more frequent disruptions caused by poor student attendance.”
When just a few students in a class have been absent, teachers have to choose between spending time helping returning absentees catch up, which takes time away from the rest of the class, or letting returning students fend for themselves, which negatively affects their progress. Either way, the class’s achievement takes a hit. A snow day, on the other hand, can be handled by postponing, compressing, or eliminating non-tested material, which is why these lost school days have so little impact on test scores.
“The negative achievement impacts associated with student absences imply that schools and teachers are not well prepared to deal with the more-frequent disruptions caused by poor student attendance,” concludes Goodman. “Schools and teachers may benefit from investing in strategies to compensate for these disruptions, including the use of self-paced learning technologies that shift the classroom model to one in which all students need not learn the same lesson at the same time.”
In this article in The Reading Teacher, Judith Lysaker and Elizabeth Hopper (Purdue University) say they have no problem with the “pushdown” of literacy expectations to kindergarten, noting that “in many classrooms around the world, children read at the age of 5 and 6.” But they disagree with pushing down parts of the primary-grade literacy curriculum that are developmentally inappropriate. “An early emphasis on specific aspects of print processing and reading subskills may crowd out opportunities for children to develop more broadly as meaning makers,” they say. “The intensity of a code emphasis reflects the assumption that print reading is a completely new experience, demanding a distinctively different set of strategies, separate from the meaning making children have been engaged in since birth.”
The bridge, say Lysaker and Hopper, is wordless picture books. When kindergarten teachers use these books well, students get practice at reading images and developing a number of early print-related strategies – searching, cross-checking, self-correction, and rereading. Lysaker and Hopper share this selection of wordless books for young readers:
In this Harvard Business Review interview with Alison Beard, former U.S. Senator George Mitchell, 81, reflects on what it took to successfully mediate the crisis in Northern Ireland. “The challenge is not to get them to talk, but to get them to listen,” says Mitchell. “The receptors in our brain for information contrary to our prior beliefs are very narrow. It requires effort and discipline to get people to consider what the other side has to say.” Here are his pointers for working with people who are at loggerheads with each other:
In this Harvard Business Review sidebar, Mark Goulston shares data from a recent survey on how the magazine’s subscribers say they communicate during conflict:
© Copyright 2015 Marshall Memo LLC
About the Marshall Memo
Mission and focus:
This weekly memo is designed to keep principals, teachers, superintendents, and others very well-informed on current research and effective practices in K-12 education. Kim Marshall, drawing on 44 years’ experience as a teacher, principal, central office administrator, and writer, lightens the load of busy educators by serving as their “designated reader.”
To produce the Marshall Memo, Kim subscribes to 64 carefully-chosen publications (see list to the right), sifts through more than a hundred articles each week, and selects 5-10 that have the greatest potential to improve teaching, leadership, and learning. He then writes a brief summary of each article, pulls out several striking quotes, provides e-links to full articles when available, and e-mails the Memo to subscribers every Monday evening (with occasional breaks; there are 50 issues a year).
Individual subscriptions are $50 for a year. Rates decline steeply for multiple readers within the same organization. See the website for these rates and how to pay by check, credit card, or purchase order.
Website:
If you go to http://www.marshallmemo.com you will find detailed information on:
• How to subscribe or renew
• A detailed rationale for the Marshall Memo
• Publications (with a count of articles from each)
• Article selection criteria
• Topics (with a count of articles from each)
• Headlines for all issues
• Reader opinions (with results of an annual survey)
• About Kim Marshall (including links to articles)
• A free sample issue
Subscribers have access to the Members’ Area of the website, which has:
• The current issue (in Word or PDF)
• All back issues (also in Word and PDF)
• A database of all articles to date, searchable
by topic, title, author, source, level, etc.
• A collection of “classic” articles from all 11 years
Core list of publications covered
Those read this week are underlined.
American Educational Research Journal
American Educator
American Journal of Education
AMLE Magazine
ASCA School Counselor
ASCD SmartBrief/Public Education NewsBlast
Better: Evidence-Based Education
Center for Performance Assessment Newsletter
District Administration
Ed. Magazine
Educational Evaluation and Policy Analysis
Educational Horizons
Educational Leadership
Elementary School Journal
Essential Teacher
Go Teach
Harvard Business Review
Harvard Educational Review
Independent School
Journal of Education for Students Placed At Risk (JESPAR)
Journal of Staff Development
Kappa Delta Pi Record
Knowledge Quest
Middle School Journal
Peabody Journal of Education
Perspectives
Phi Delta Kappan
Principal
Principal Leadership
Principal’s Research Review
Reading Research Quarterly
Reading Today
Responsive Classroom Newsletter
Rethinking Schools
Review of Educational Research
School Administrator
School Library Journal
Teacher
Teaching Children Mathematics
Teaching Exceptional Children/Exceptional Children
The Atlantic
The Chronicle of Higher Education
The District Management Journal
The Journal of the Learning Sciences
The Language Educator
The Learning Principal/Learning System/Tools for Schools
The Reading Teacher
Theory Into Practice
Time
Wharton Leadership Digest