Effective School Leaders


>>ELIZABETH GRENINGER: Good
afternoon, everyone, and thank you for joining us for today’s
teacher effectiveness webinar on effective school leaders. I’m Elizabeth Greninger,
and I’ll be facilitating today’s webinar. Joining me as our featured
presenter for today is Dr. Eric Hanushek from
Stanford University. Before I give you a more formal
introduction to Rick, I would like to find out who’s
joining us on the webinar today. You should see a pop-up box come
onto your screen, and I’d like you to select the role that
best fits the one you serve. Nice to see a diverse
audience with us today. Several principals are here. Lots of researchers. We think that everyone who’s
attending today will get a lot out of the session, and hearing
about the research that Rick has done on effective school leaders
and get some real practical tips from the conversation with your
colleagues on the line about how to use this
information in your context. So at this time, I’d like to
introduce Dr. Eric Hanushek. He is the Paul and Jean Hanna
senior fellow at the Hoover Institution at
Stanford University. His research spans areas such
as impact of teacher quality, high-stakes accountability,
equity and efficiency in school finance, class size reduction,
and the role of cognitive skills in international
growth and development. Rick pioneered the analysis
for measuring teacher quality through growth in student
achievement that forms the basis for current research
into the value-added of teachers in schools. He serves as the chairman of
the Executive Committee for the Texas Schools Project at the
University of Texas at Dallas. And he’s authored or been the
editor of at least 20 books and 200 articles. At this time, I’d
like to welcome Rick.>>ERIC HANUSHEK: Thanks,
Elizabeth, thanks for having me. The presentation today that I’m
going to give comes as-from the standpoint of a researcher,
and many of you have much more practical experience than I do. But from a researcher
standpoint, I have spent many years looking at teacher
quality and measuring teacher effectiveness in all the debates
about evaluation and personnel systems but had largely
neglected the role of school leaders, at least in any
of the research I’ve done. There was-as all of you know,
there’s lots of anecdotal stories about the really good
principal or the really bad principal, and there
are movies about both. And I was never convinced that
that was all that important in a systematic way, as opposed to a
few isolated incidents of people that are really good. And then with some colleagues in
Texas, I tried to look at the impact of principals on
student outcomes and student performance, much the way that
I had looked at the impact of teachers on student performance. And the-without wanting to hold
the surprise for very long, the answer simply is that principals
are really important in schools and there are big
differences among principals. And it’s something that’s
large enough that we should pay attention to. Now, I want to warn you before
I give you the discussion of principals and their effects,
that I’ve spent so much time looking at teachers that I
sometimes, instead of saying “principals” when I should, I
say “teachers.” So unless I underscore it or something, you
should always assume any time I say “teachers,” I really meant
“principals.” So that’s just my problem with getting
the language right. So let me talk about
today’s activity. I want to review
research on principals and effective school leaders. This research is all pretty
new, at least in a systematic, quantitative sense. There are lots of anecdotal
stories and case studies, but this is an attempt to
look quite broadly at the effects of principals. And I have done some of the work
with my colleagues Greg Branch and Steve Rivkin in Texas. There are other people that
we’ll cite along the way that are doing similar work. But it’s all within the last
five years that people who have looked at quantitative
research have looked at all at principals. Secondly, which I think that
many of you are better able than I am to talk about how school
leaders can influence school culture, that it seems like
much more of a question of what actually goes on within schools,
but I will talk a little bit about what the research says on
how effective school leaders impact on teacher quality, and
necessarily, that’s going to be rather restricted, because we’re
going to be able to look at how principals manage the teaching
for us, but not necessarily about how they make the
existing teachers better. Those are the goals for
today, to go through those. The specific questions that
we’re going to look at are-the simple one that I’ve already
given you the answer to, is there substantial variation
in principal effectiveness? What I’m going to try
to do is show you how large that variation is. Does this difference
in teacher-principal effectiveness-see, I’ve
already done it once-does this difference in principal
effectiveness become more important in schools serving
disadvantaged populations? Thirdly, do more
effective teachers make better personnel decisions? And fourthly, I’ll talk a
little bit about the movement of principals that we see, do
they-do good or bad principals, effective or ineffective
principals, tend to leave poverty schools more
frequently or not? So those are the questions that
I hope to give you some answers to today largely from the
research that I’ve been doing across the state of Texas, but
it’s also backed up by other work in-particularly in Florida.>>ELIZABETH GRENINGER: Great. Rick, thanks for that
overview of today’s webinar. We want to open up with a
question for our audience and get a sense from folks of what
you all think is the most important role of
a good principal. So before we get started, we’ll
put out a poll, and you can type in an open-ended
response to that question. Instructional leadership
seems to be the front-runner here so far. Clearly, instruction is a
central component of schools, and really important for leaders
to have some skill in providing instructional
leadership to teachers. Leadership in general is coming
up quite a bit, and I’m sure that means a lot of
different things to a lot of different people. We’ll get into a little bit
of that today and what the different skills that
are necessary are. Talking about the impacts
on student performance. Culture building, Rick already
mentioned that that’s something we’ll be thinking about today. This is great. A lot of very broad concepts
here, and I think in the webinar today you’ll get some more
specifics, but also I think, as Lauren already showed us in the
parking lot, the great place for you to exchange ideas and think
about some more strategies and specifics of how school
leadership looks in the place where you work. And I think that sharing
of ideas will be really an important outcome
for today’s webinar. So feel free to use that parking
lot to ask questions and to share any ideas as they come
up through the conversation. All right. So, Rick, I’m going to turn it
back to you so you can give us background on effective
school leaders.>>ERIC HANUSHEK: Well, the
responses to that, I think, first show a lot of insight
from your own experiences. They don’t necessarily match
survey evidence that I have seen on principals or time use
studies, because the emphasis at least originally on
instructional leadership is one component. But the surveys and the actual
time use diaries of principals suggest that that’s just one
small part of many, many roles. I’m going to try to go into just
the outcome effects first, and then we’ll probably have to
come back to the roles that principals have, but not
necessarily from the research that we can do. The first sort of anecdote-I
don’t think it’s actually necessary, motivational slide,
is that if you ask anybody in any kind of meeting about
schools, they immediately say, having a good leader
or good principal is absolutely essential. And as I say, that’s something
that we have tried to look at in very detailed ways. The emphasis that we’ll place is
on the achievement of students, fitting in with the idea
that this is the era of accountability, and the first
thing that matters is whether students are learning and
whether we can see the effect of learning. So what we’re going to, sort of
the simplest conceptual model that we started with was just
that principals can have quite varied impact on the school,
decisions about-decisions and influences on teacher retention,
helping teachers to grow and do their job better, and setting
the overall school culture, were the background ideas that
we had going into the study. But we saw this largely at this
stage as rather a black box. And what we wanted to see is
if you put all those elements together, could you in fact
find differences in student achievement gains. Now to give you a lead-in to
some of the evidence, I’m going to start with a study that’s
not by me or my colleagues. It’s by actually one of my
Stanford colleagues, Susanna Loeb, and some of her former
students who spent a lot of time looking at what happened in
Miami-Dade County schools, and serving different
populations there. What it comes out across the
board, and we’ll reinforce this, is that principals in lower
achieving schools or more disadvantaged schools tend to
have less experience, less educational background. They’re just newer to the
job, and that happens rather systematically. As a result, you see that these
new principals tend to turn over also more rapidly than
more experienced teachers. And they’re likely to have a big
effect on who the teachers are and the teacher development. This comes from observational
studies, where these people actually spent a lot of time in
the schools observing what the principals are doing and then
trying to put that together with some of the data on schools. What they did find was that with
less effective principals, that they tended to essentially drive
teachers out, make them more likely to go to a different
schools and to other districts. That effective teachers
had a big impact on student achievement, within as short
a time as one year, as did ineffective teachers have
an almost immediate impact. That was something that we
didn’t know before they started putting this together with the
data on principal movements and on teacher-teacher movements
and student achievement. The work that we’ve done
goes on and tries to do it in a much broader scale. I guess I shouldn’t have
switched quite to that slide. What I’m going to do is show
you not the study of one school district, but what I’m going to
show you is what we see across the entire state of Texas, which
as you know is one of our larger and more diverse states, so
that we have lots of experience to draw upon. And I think it goes
over to you, Elizabeth.>>ELIZABETH GRENINGER: Yes. Let’s talk a little bit
more to get folks engaged in this question. What impact do you all see
principals having in your school or your district or your
state, wherever you are most closely familiar with? We’d like to know about the
outcomes that you see principals having an effect on. You’ll see another poll. We’d like to get this
conversation started. And Kristen’s comment in the
parking lot, maybe you missed the comment Rick made at the
beginning, but he mentioned that it’s a frequent error he makes
in interchanging the terms “teacher” and “principal,”
because there is a heavy focus on work with teachers. So not intentional, but he
did provide that disclaimer at the beginning. Just so folks are aware of that. Looks like folks are seeing
principals having a really big impact on the
environment and the culture. Spending time with data and
making decisions based on data. Setting expectations and vision. We know those are
all important things. Rick, do you have the sense just
like with the last question that some of the things that folks
are mentioning here are the ones that principals are truly
spending their time with? Or what is the research showing
us, without –>>ERIC HANUSHEK: I think what the observational
studies that go into schools show is that principals have the
kinds of impacts that are shown here, but it doesn’t necessarily
match up to the way they’re allocating their
time and energy. So you see that some principals
are doing many other things, you know, down to small
administrative tasks and so forth, as opposed to really
taking ownership for the culture in schools or making the system,
the whole school work better. The observational studies, I
think, suggest that if these are the important things, which
I believe that they are in general, that are being listed
here, if these are the important things, these are not
the way all principals are spending their time. But they’re doing
quite different things.>>ELIZABETH GRENINGER: And I
think that begs the question. I think most folks on the call
today would agree that if these are the outcomes that we want,
we want people to be spending their time there. We need to have some focused
decision-making about what expectations for principals are. So that’s kind of some
food for thought for folks. But good ideas here. We can move this along, and get
you into the next section to talk to us a little bit more
about the details of the Texas Schools Project and the
study that was done there.>>ERIC HANUSHEK: Well, let me,
what I want to do is give you just a little bit of background
about what this research that I’ve done is all
about and how it’s done. I’m not in any way trying to
teach you the details or show you all what’s involved, but I
want to give you an overview, so that you have some sense of what
we can and cannot say with the data we have. The overall work is part of
what we call the Texas Schools Project at UTD, or the
University of Texas at Dallas, and UTD and the Texas Schools
Project has worked together with the Texas Education Agency,
which is their state department of education, to put together
data that matches students and teachers and
principals across time. And it uses the administrative
data that many of you who are working the schools are
supplying to your state, but it tries to put it together
in a way that we can use it for research purposes. What we have done is to
examine principal quality and effectiveness from a value-added
model, which looks at how does an individual principal add or
subtract from the learning gains that you see in students. The whole idea behind this work,
which in the background is a series of statistical analyses,
the whole idea behind this work is to try to separate the
role of the principal or what difference a principal makes
from the other background factors of students and what
they started the school knowing and so forth. So if you think that, as most
research confirms, that families are perhaps the most important
element of a student’s learning, this research tries to separate
the inputs of families from what does the principal add to the
learning of this student. So, since the mid-1960s when
this work really began (not the principal work but the work on
looking at student achievement), we know that families
are very important. We know that the other
students in the school are very important, and then comes the
schools, and the teachers and the leadership in the schools
can have an influence on top of that. And so the whole research is to
try to find out how important is it to know which principal you
had in a school, how important is it for the student outcomes. So that’s where we’re
going with all of this. The way we do this is that we
can actually put together all of the administrative data over a
number of years, I’ve called it a stacked panel of data. What it is, is that I can take
an individual student in, say, 2000 and trace the path of that
student from 2000 through 2005, 2010, and look at the
achievement path and outcomes of that student. But I also then get to see some
students in 2001 who start out in maybe fourth grade, and then
I get to watch them as they progress through school. So it’s a panel data in the
sense that I follow individual students over time, and it’s
called “stacked panel” just because I look at
different cohorts of students. The students, the class of 2014,
or the class of 2013 or 2012. Now, the advantage of doing
this work in Texas is that Texas is a huge state. So we have a large
number in our analysis here of different principals. We have 7,420 individual
principals that we can observe for some period of time
over the period 1995 to 2001. For technical reasons of getting
access to the data, this is a little bit old, and we’re right
now expanding it right up to the current day. But I can’t give you
those results today. I just can give you the
historical information. But we get to observe
28,000 principals and year observations, and try to, out of
that, say something about how different are the
principals that we see. We’re going to spend a lot of
time looking at whether there are differences among
high-poverty schools or lower poverty schools, largely
because much of today’s policy discussion centers on
whether we are helping our most disadvantaged students
sufficiently, or what it is that the schools that are holding
them back, or is it something outside the schools. If it’s something outside the
schools, can the schools make up for the problems that
poverty people in general have? So throughout this webinar,
we’re going to come back to poverty and disadvantage. So to get some starting point,
we know that principals change schools a lot, and I’m sure that
all of you have seen that in your own environments. What this chart does is give a
little bit of data, where we break out schools by their
poverty quartile, and we have the lowest poverty quartile,
which is the most advantaged students, compared to the
highest poverty quartile, the one-schools with the most
concentrated incidence of poverty among the families
of the students there. If you look at the first
numerical column, what you see is that 17.8 percent of the
principals in the-in the lowest quartile poverty schools
are in their first year. They’re just brand new. Slightly higher, 19 and a half
percent are in their first year if it’s a high-poverty school. Similarly, the far right column
shows you a sense of longevity, how many of the principals have
been in the schools six or more years, and what you see is that
principals tend to stay longer in low-poverty schools, and tend
to go-turn over more rapidly in high-poverty schools. So this matches a lot of
what the common perception is. The one thing I want to point
out though is that you get much larger differences, if I don’t
look at poverty per se, but I look at the overall achievement
levels of the students in the school, and these just
aren’t the same thing. So if I do the same analysis
where I divide schools up by the lowest quartile in terms of math
achievement-and this is on the standardized Texas test of
mathematics here we’re using-and we divide schools by their
average achievement in math and the lowest math achievement, we
find that over 20 percent, 22.7 percent of the principals
are actually in their first year of teaching. And only 26 percent stick
around for six or more years in low-achieving schools, as
compared to high performing, students with, schools where the
students are high performing, the highest quartile in terms
of math achievement of schools, only one-sixth of the
teachers are new to the school in their first year. And 38.8 percent have
been there for a long time. So it looks like from these data
that principals like to have schools with higher
performing kids. Now, this is a
little bit confusing in the following sense. I’m going to show you also
that principals have a big effect on achievement. So it’s not that they’re
just choosing to be in high-performing schools. They’re having an influence
about whether these schools are high or low performing
through their own actions. And that’s what we hope to
parcel out as we go along here. So let’s go on to talk a little
bit more about what the analysis is, because understanding
the analysis is important. If you, many of the common first
reactions to this work is, well, did you take into account the
fact that some schools have kids that are less prepared than
others, and things like this. And what I want to do is walk
you through a little bit of what we did to try to get you
some-give you some idea of the things that we have
taken into account. That’s not to say that we’ve
taken everything into account, but I’ll try to show you what
we have taken into account.>>ELIZABETH GRENINGER: Rick,
before you move into the next section, just a quick question. You have a few folks asking,
just a clarification, and I think you said it but we want to
make sure, when you’re talking about principals in their first
year, are you talking about first year ever as a principal? Or first year at that
school in particular?>>ERIC HANUSHEK: I’m talking
about first year at that school in particular. And we’ll look a little
bit more at teacher tenure. But this is essentially a
measure of how much turnover, how much churn is there in the
principals of these schools, and is that churn related to
either poverty or to the achievement of students.>>ELIZABETH GRENINGER:
Okay, great. Thanks.>>ERIC HANUSHEK: What we’re
trying to do, this is a kind of a problem that researchers get
heavily involved in, because it’s a complicated problem. It’s not easy to pick out the
separate impact of principals from all the other
things that influence kids. But we know that neither
principals nor students are randomly assigned to schools. Parents are making choices
about what schools they’re in, principals and school leadership
district personnel are making choices about what specific
school an individual principal is in, and some of these things
sort of go along all together, so that we could get confused
about, is it just that good kids selected into this
school and it’s nothing to do with the principal? Or is it that the principal
is having a big impact. So what we try to do, the main
approach to try to take care of this, is that we try to first
control or allow for differences in all of the observed student
characteristics and in their prior achievement. So when we look at a principal,
in say the year 2000, what we start with is what was the
achievement of the kids in that school in the year 1999? So we’re going to try to ask,
does that principal, does the principal influence where we see
student achievement at the end of the year or not? So we’re going to try to
eliminate all of the differences in the preparation and then
see what happens after that. And we’re also for the most part
going to try to standardize our results for how long the
principal has been in the school, because a principal
who has been in a school for a longer period of time,
obviously, has a cumulative effect on what’s been
going on in the school. And so we’re trying to look at
people essentially at the same amount of time that they’ve been
in the school, to try to find out inherently how
much difference is there in a principal. So those are the things
that we’re going to do in the background. I have-I’m going to walk you
through or give you just a slightly technical
version of this. Some of you are in fact
interested in the technical aspects, and for the others, I’m
going to try to introduce you to some language that you might
hear or might see if you read any articles about the effects
of principals or value-added and so forth. I’ll just give you a
quick overview of that. What we’re going to be doing is
using statistical analyses to try to separate out the
independent influences of all of these different factors, and
in reality, when one does a statistical analysis, it’s hard
to say that you have absolutely the best way to
perform this analysis. What we have done is use some
alternative approaches that make different assumptions about the
importance of other factors in order to get essentially a bound
on the impact of teachers. We want to know sort of what’s
the largest and what’s the smallest potential
impact of principals. I said “teachers” again-
please translate whenever I say “teachers,” unless I say
specifically teachers, translate that to “principals-what I want
to do is use some alternative ways to look at the potential
range of impact of principals, and if that, if the lower part
of that range is still a big number, which I’m going to show
you it is, we can be pretty certain that no matter
how we look at this problem, principals are important. And that’s going to be one of
the conclusions, of course. So let me provide you a brief
technical description, if I can. So, we’re going to use
alternative approaches. They all use a shorthand, a
statistical shorthand which is called fixed effects
for principals. This is just researchers trying
to mask very simple concepts. What the fixed effect for
principals is, is really the average impact on achievement
gains of each principal. And so, it has to do with the
way it’s estimated, which is unimportant for our discussion. What it really is, is I’m going
to try to say on average across all the kids in a school,
what was the separate impact of a principal. And then I’m going the try to,
I’m going to try to see whether that estimate of the average
impact of the principal varies across individual principals. And of course it does. What this does is try to control
for all the background if we can of the student, where they
started, whether they were in special ed, whether they were
ELL students and the like, things that are not the
impact of the principal, but of outside factors. And then I’m going to try to
get the average effect of each principal, and then I’m going to
compare these average effects across all of the
principals that I see in Texas. There’s a second way that this
is done, which we will use as we go along, which is
called fixed effects for principals in schools. What this says-what this really
means is I’m going to look at the average effect of a
principal, holding constant other things, but I’m just going
to compare one principal in a school, in a given school, to
another principal who comes into the same school. So I’m going to compare the
effect of a principal at North High School or North Middle
School with another principal in the same North Middle
School in the same town. Now what this does is, attempt
to take into account the selection of both teachers,
principals, and students into an individual school and say, well,
if we look at all the kids that went to North Middle School this
year compared to all the kids that went to North Middle
School last year, they are roughly similar. There is not big changes over
time for an individual student. Then I’m going to try to look
at whether the-having a new principal changed the
achievement and growth in that school. So this is a really stringent
test of differences among principals, because I’m
just comparing two different principals in the same school,
and trying to say, did the first principal seem to do better than
the second principal at that school, where they are
separated by a year or two years. finally, what I’m going
to do is something that I’m not even going to
explain to you in detail. I’m going to try to look at how
much of the variation in student achievement seems to be
related to a principal and do that directly. If I get a new principal in a
school, and it increases the average achievement of everybody
in the school, what I’m going to see is that the variation in
test scores across the state tends to rise a little bit,
because I’ve had this new principal, has managed to
push up achievement compared to other schools. But I’ll just give you
the answers to that. That-the details are beyond any
simple explanation in a webinar. But they’re a really
conservative way of trying to estimate the impact
of principals. So this is going to actually
give us our lower bound, the sort of lowest reasonable or
plausible estimate of the impact of principals. And then, what I’m going to
ultimately do is try to validate the different estimates of the
effects of principals by looking at what teacher
turnover is related to an individual principal. So that if a principal has a big
effect on a school, either plus or minus, we could plausibly
expect to see that teachers are changing at the same time
related to that principal. So that’s the overview of
what we’re going to do. There are more details on this
that I don’t want to go into, to details, other than to say
that the middle one is the most important, is that in all of the
estimation, we take into account what was the starting math
score, lagged math score of the kids that I observed this year,
and as much of the student demographics as I can get from
the administrative data, race, ethnicity, free lunch status,
special education participation, gender, ELL, and so forth. And then I’m also going to try
to allow for any changes in tests across the state and
other things each year. So what I’m again trying to do
is to hone in, as best I can, on what difference does it
make which principal shows up in the school. So let me show you some
results for that, if I can.>>ELIZABETH GRENINGER: Rick,
before you move on, one question that’s related to the slide
that you were just on. Frances is asking if you
took a look at the rural versus urban, the context. I don’t see it there. But->>ERIC HANUSHEK: We
haven’t spent a lot of detail on that, and that is actually on
our research agenda, to look at whether there are
big differences. The only difference that we have
is if there’s differences in the racial concentration of
schools, and free lunch status, and so forth. But we have not specifically
allowed for urban/rural. We recognize that that’s an
important issue in Texas. There are 1,000 school
districts in Texas. And about half of those are what
you might call rural districts. And so we’re going to
come back to look at that. But I can’t give you any
specifics right now, Frances.>>ELIZABETH GRENINGER:
Great, thank you for responding to that.>>ERIC HANUSHEK: So here’s the
first set of estimates of how different are schools. So what we’ve done here is to
measure student achievement in standardized forms, so we’ve
taken the Texas test, and transformed them into ones that
have a mean of 0 and a standard deviation of 1. What this does is allow me to
say how much does a principal move a student in terms
of their position in the overall student distribution. Because we know, the following
are characteristics of the student distribution. If I have a student at the mean
of the distribution versus one that’s one standard deviation
ahead, the one at the one standard deviation ahead is
at the 84th percentile of the distribution. So we’re moving students
from the middle to the 84th percentile, if we moved an
entire standard deviation. That’ a big, big number. It’s something you see in
most schools, of course, those kinds of differences. But you don’t see schools that
are that different, nor do you see the impact of principals. But what you do see is that on
this slide, I’ve given you an overview of the average, the
variations that we see in the average impacts of
principals on student gains. It says the first three years of
tenure, again, I just want to look at people in the same kind
of time period of being in the school, because a principal
that’s been there 15 years in a school has obviously changed a
lot over that time, and I want to look at more a short
period of time to see how much influence changing
a principal has. And what you see in the lowest,
let’s look at the top line of this table at the bottom. It says the lowest
poverty quartile. Again this is dividing all the
schools in Texas up by what percentage of free and
reduced-price lunch students there are in the school, and the
lowest poverty quartile are the most advantaged students
or schools with the most advantaged students. And the impact there,
that’s identified as 0.16 standard deviations of
student achievement. What this says, if I take an
average principal and compare them to a very good
principal, one that’s at the 84th percentile, in each year
that we observed the principal we should see that
students on average come out .16 standard deviations ahead. Now, what does that mean? That means if I started with a
school with students at roughly the average achievement
level, the 50th percentile, and moved them up .16 standard deviations of
student achievement, I would move them up to somewhere around
the 56th percentile of students. This is each year. This is a big effect, because
the good principal is actually moving kids quite a bit
ahead each year, 6 percentiles in the distribution. But as big as that effect,
it gets bigger as I go to higher poverty schools. So if I go down that column
until I get to the greatest poverty, that’s the most
concentrated poverty schools, the difference between an
average and this good principal is now up to .26 standard deviations, which
means that I’m moving somebody from the middle of the
distribution, now up to this 60th percentile in the
state, in state distribution. So this is, again, a huge impact
that we see estimated for the impact of principals. Now, the opposite is also true. And we don’t often like to
talk about the opposite. But if I move, if I’m in a
high-poverty school, and I move from an average principal to
one at the 16th percentile of principals at the bottom end of
the distribution of principals, I subtract from
student achievement. I lower student achievement. And I would move a school that
started with the kids on average at the middle of the
distribution, I would move them down to the 40th percentile
in that next year. So what this is saying is that
there are really, really big differences that we observe in
student learning growth, gains in student achievement,
depending upon whether they have a really good or really
effective principal or whether they have quite
a poor principal. On average, the
impact is about .2 standard deviations, .21 which means that I’m moving
kids from the 50th percentile of the distribution to the 58th
percentile of the distribution. So that’s the way that we try to
characterize how much difference does it make which
principal is in the school. Now, these are subject to lots
of uncertainties, and I’m giving you precise sounding
answers, that say .21 standard deviations. The research itself, and
something that’s important when we think about policy,
is in fact affected by different things. I put up on the slide
random measurement error. It’s just that the tests as
you all know are not perfect. They do not perfectly
measure student outcomes. We have to allow for the fact
that you know some tests aren’t very good or some students
aren’t prepared for a particular test, and we want to take
that out of this problem. And so, what this does is make
principals look more different than they really are, because
what we saw in the difference across schools achievement gains
was a combination of the impact of the principal plus this
random error, this noise, this test measurement error that
compounds, makes some principals look better than they really are
and some look worse than they really are, just because they
got a good set of tests for a given year. Now, there are formal
ways to take care of this. I’ve just communicated
a formal treatment. It’s the Bayesian
shrinkage estimator. What it really does is say
that we want to control for principals that look like they
are far out in the distribution, have had too much influence. And we sort of move them back to
the center and see whether that has an influence on achievement. There’s a second set of problems
that are particularly important for Texas at this time period. Texas in the time period that
we’re looking at was late ’90s, early 2000s, was using a
particularly-I’ll call it an easy test. It was a test that was designed
to measure most carefully basic skills, the really fundamental
skills, but not be too good at the top end. It wasn’t very good at
distinguishing students that knew a lot more than
expected for a given grade. It was particularly good at
looking right around the expectations which were, at
this time, fairly low in Texas. As you all know, there’s a lot
of debate nationally about the tests, whether some tests are
too easy or hard, we’re changing all of our tests and so forth. But it does in fact affect the
research we’re doing also, and the reason it affects the
research is that we can tell a lot of difference with the Texas
test at schools where many of the kids are low achievers. We have a more difficult time
looking at schools where the kids are achieving more. The tests aren’t very
discriminating at the high end. So that means that if a
principal comes into, I’ll call it a low-poverty school, because
on average they are higher achieving, they might look-have
a hard time showing that they’re really good, because it just
doesn’t show up on the test, because they’re basic
tests that are being given. So what we’re going to do is
actually take all the students in the school, and pretend that
we have the same distribution of high and low achievers in all
schools to start with, and then we can come back and see whether
some of the variations we saw in principals like we saw the lower
variation in principals in the low-poverty schools, if you
remember, part of that could be that they just have a hard time
showing up on the test, that the low-poverty schools are scoring
high on the test, and the tests aren’t very discriminating. So, we’re going to try to make
sure that the differences we saw cross schools was a real
difference, and not just some artifact of the test. Now, again, I have no intention
to teach you statistics. But what I want to do is to just
suggest that we’ve spent a lot of time trying to take these
factors into account, and the slides that I just put up at the
bottom, tries to show you that this isn’t the driving force. These issues, these concerns
aren’t the dominant factors. Unadjusted column at the top is
what I showed you before, that on average, the difference
between a good and an average teacher is .21 standard deviations
of student achievement. When I allow for the fact that
test measurement errors might come in, it doesn’t
change it much. It changes it from .21 to .20. It makes a little bigger
difference when I allow for reweighting to allow for
whether the tests are measuring performance at different points
in the distribution well. And that makes a little
bit bigger difference. But when I put it all
together, the difference is .21 to .24, in the bottom column there. And we basically conclude
that’s not what is important. Our results are not some
statistical artifact, but they are real. And Elizabeth?>>ELIZABETH GRENINGER: Sure. Before I jump into this
question, I think we have a relevant question from Nick. He’s asking about the relevance
of the standardized tests. For example, were they required
for graduation, promotion, or included on transcripts? If you look at that, does
that make any difference?>>ERIC HANUSHEK: Very
good question, Nick. The tests we’re using as a
standard state test used for NCLB accountability purposes,
they were actually-Texas started having test-based
accountability before NCLB. I’m actually looking at the
entire pre-NCLB period since NCLB came into effect in 2001. But the-or enacted in 2002
is when it really started. The students were not held
responsible in some sense. So if students were not taking
them seriously because it didn’t affect their grades, that
could enter into this. But what it does come into
account for promotion. Texas was moving toward more
grade retention at this time period, and so some of
these tests came into play for promotion purposes. And other kinds of similar tests
that we’re looking at lower grades, where we test lower and,
primary, middle grades, where we have the annual testing in math,
that we’re concentrating on, there were other tests
at later grades that did in fact affect graduation. So these were earlier in the
career, where in part, taking advantage of the fact that
younger kids tend to take things more seriously than
older, or tend to work on them more seriously. But there are the possibilities
that in some schools the kids just blew it off and weren’t
paying attention to it.>>ELIZABETH GRENINGER:
Thanks for responding.>>ERIC HANUSHEK: Let
me actually answer Andrew’s question too. We’re at a little bit
of a juncture here. These are annual effects. These are one-year measures. These are the effects of a
principal on the growth in student achievement
over a one-year period. Now we don’t have
particularly solid evidence on how these compound. But roughly, these add
up and go over time. So a principal has a continuing
impact on the performance of the student, at least over
the three-year period that we’re studying here. What we’ll see, later on, this
impact actually looks like it continues as long as the
principal is in the school, that they have the plus
or minus impact.>>ELIZABETH GRENINGER: Great. Thanks, Rick, for
attending to those questions. We have-I did note that there
were some questions regarding incentives, and I
want to get to that. But I want to open up
this next question so folks can start responding. And then I’ll get into some of
the other questions that are happening in the parking lot. So for all of you out there,
we’d like to open this one up and have you start thinking
about what other factors in your schools or the research you
might be doing could you see affecting estimates
of principal quality. So other than some of the things
we’ve been talking about, what do you see as having
an impact on teacher, or principal quality outcome. While you all start typing that,
Rick, I’m going to talk to you about some of the conversation
that was happening way back here, a way back with David,
and, I think, a few others. Regarding principal movement,
so two kind of parts to this question came up in reference
to monetary incentives. And so I don’t know if you want
to touch on this now, but I’ll ask you, and you can defer to
later if you want to wait and talk about it in a little bit. They’re wondering, are there-is
there a correlation between incentive pay for test score
increases to principal turnover. So is there a relationship, if
principals are moving schools because of an incentive. So whether there’s, I think two
situations, one is that there is an incentive to go to a
high-poverty school, or there might be kind of an informal
incentive to go to a school that is high performing that
might just have higher salaries in general. I think I’m understanding that
conversation that was going on. But maybe if you
can speak to that.>>ERIC HANUSHEK: Well, we’ve
tried to look some at salaries, and we’re going to
look more in the future. The difficulty is that we do
not know, for example, what the contractual payments or salaries
are for principals, and when we’ve looked at this in Texas,
we find that it’s often not even written down, but that it
seems to be all over the board. What we found is that salaries
to principals don’t seem to be very closely related to how
individual principals choose to move across schools or not, but
this is looking at the level of principal salaries and we cannot
actually observe or we haven’t been able to yet observe whether
they, for example, get a bonus to move to a particularly
difficult school or not. Those are questions we would
hope to be able to look at in the future, but frankly they’re
also really hard, because most principals don’t have set salary
schedules, but in fact it’s much up to the superintendent
and the district how they reward principals.>>ELIZABETH GRENINGER: Okay. That’s helpful
information, I think. That piece about incentives
and compensation is a whole different topic but
certainly related. Let’s get in here to the
question that we just asked and start thinking about some of
the responses folks are having. Give us a moment to
glance through those. Andrew makes a good point about
staffing and the principal’s role in staffing schools. I think we’ll talk a little
about that, but it’s an important role that
principals play. Holly brings up the point of
changing state and federal mandates, something I alluded to
a little earlier, in reference to the principals having to meet
a lot of varied requirements. So when we talk about our
outcomes and some of the responses we got earlier related
to communication and school culture and driving student
achievement, sometimes those are not in line with the mandates
that principals are under. So that certainly
is a challenge.>>ERIC HANUSHEK: I should say
that, at least a portion of these observations, that I think
are all very important, will be taken care of by where I’m going
to take you in terms of a little bit more analysis of the-how
much variation there is in student achievement.>>ELIZABETH GRENINGER:
That’s great. So I think we can
move right along. There will be more opportunities
for folks to weigh in. And feel free to continue to
use the parking lot if you have questions or comments
you want to share.>>ERIC HANUSHEK: So, let me
return back to, again, some of the questions we’ve been asking
and also try to get at the ones that were just brought up, the
factors of is it the school board, is it state policies, is
it the selection into schools, and so forth that’s
driving part of this. I’ll try to give you some
different views on that. The answer at this point, is,
yeah, there’s a big difference in student outcomes that’s
related to the principal in the school. And that the variation in
principal effectiveness looks like it’s very large. Again, these are annual impacts. It’s actually larger
than 4 percentile. It’s about 8
percentile difference. I think we have a little mistake
there from what we had before. And this is the effect that is
about the same size as has been estimated for
teacher differences. Except the principal works on
the whole school, not just one individual classroom. So when we see that these
kinds of variations in student performance, and sort of try
to compare it to, well, is the principal more or less important
than a teacher, what we see is that in many ways, for
principals, the principal is more important because the
principal affects the entire school all at once. Let’s come back to both
low-income schools, and it gets back to some of the
questions that came up in the open-response part. Couldn’t part of this estimate
have to do with just the kids that a principal happens to have
in the school and the selection into the school? Couldn’t it also be that the way
that district leadership treats individuals, principals, or
schools, isn’t it something about the school
board and so forth. We all know that politics
and other things are really important in many of the
decisions, particularly about who is going to be a
principal and how long they stay and so forth. So this is where we go. I gave you an overview of
alternative ways to try to estimate the impact
of principals. The one that I gave you just
sort of says if I take into account all of the prior
achievement of students and the characteristics of the students,
what’s the average relationship between, what’s the average
gains that we see in a school that’s leftover, that appears
to potentially be related to the principal. The alternative that is superior
in the way in the sense of trying to make sure that it’s
not other things influencing the impact of principals is to go to
what we said before, and we’re going to look at principals
within the same school and what happens to student achievement
growth in a given school when the principal changes. So I’m going to compare the
principal this year, with the principal we had last year, and
say how much difference do I see in the achievement growth
this year and last year. And that’s going
to be my measure of the effectiveness of principals. Now, this is not-here is the
way we’re going to do that. The first column is basically
what I gave you before, of the effect of a principal by the
poverty status of the school, the lowest to highest, and
“all” means the poverty status of the school. I’ve now, instead of just
including principals in the first three years, I’ve
included six years. But you see the same exact
thing we saw before, slightly different numbers, but the
lowest poverty schools, most advantaged schools, have less
impact of the principal on the student achievement, 0.18
is that first number in that first column. And as the poverty goes up,
principals seem to have a much larger impact on the growth
in student achievement. Overall, the estimate here is .22 for not including this
larger number of principals. Before I had .21. It’s exactly the same number. Think of it as the same number. The right-hand column though
is something quite different. That’s where I compare a
principal this year with the principal last year and look
at the differences in student achievement growth. What this is doing is, unless we
have had a change in the school board or a change in the
politics this year to last year, or so on, what I’m trying to do
is look at a given school where all those things are roughly the
same, this year to last year. And the only thing that’s
different is the principal. And in this case, I get
estimates that have the same pattern, but
they’re much smaller. They’re roughly going to be
about half of what I had before. What I’m seeing is that in the
lowest poverty schools, the impact of a really good
principal compared to an average principal is much smaller, .08 standard deviations. It’s less than a half
of what I had before. I have the same pattern that
the principal looks to be more important as the
school has higher concentrations of poverty. And on average, the estimate of
the impact of principals is cut in about half. But let’s think a minute again
about the average effect, the 0.11 that I get here, is a much
more conservative estimate because it just looks at
differences among consecutive principals on their learning
growth of students, but it says that a good principal, again
at the 84th percentile, top principal compared to an
average principal, gets .11 standard deviations of
achievement gain each year, higher in the school. Now, I’m not sure if it was
Andrew or somebody else who asked the question, “Well, if
this cumulates, can’t this close the achievement gap?” And the answer is yes. Because the average achievement
gap between high-poverty and low-poverty students in
Texas at this time was about .6 standard deviations. So it says if you have a really
good principal for five years in a row or so, you can hope
to close the achievement gap between this high-poverty school
and a lower poverty school. So these are in fact substantial
impacts that principals have on the potential for
closing achievement gaps. That’s the sort of more
conservative-even in the more conservative estimates, we
do in fact see a big impact. Now, we tried to look at what
kind of impact a principal has on a school, and there
we’re restricted to the administrative data. So we don’t have the depth and
color and details that were brought up by your
questions and comments early on. What we have is the
administrative data, and what we can do is just look at
transitions of principals and then try to figure
out what’s going on. The what we’re going to do-what
we’ve done throughout here is observe the difference in
mathematics achievement gains across different
principals, and they obviously have different cohorts. So we have looked at overall
finding that a good principal compared to an average is
roughly moving from the 50th to the 58th percentile or a good
principal to a very poor principal moves them from the
50th to the 42nd percentile. We’ve done some other estimates
that try to validate this in other statistical methods. And I’m just going to give you
the answers, I’m not going to talk about it in any detail. But what we call direct
estimates in the far right-hand column only looks within
schools, and it cuts the estimates again in half
from the middle column. I think if you wanted to put a
realistic bound on the impact of principals, you would say it
was somewhere between the .11 in the middle
column, at the high end in .05 standard deviations of
student achievement at the low end. But what I’m going to again just
reiterate is that these are big effects, because they
are annual effects. They apply to the average
for the entire school. So that they’re going beyond
individual classrooms, but saying how much difference does
it make that you’re in this school with this principal, as
opposed to this school with a different principal. This is-this is where I turn
it over to you, Elizabeth.>>ELIZABETH
GRENINGER: Sure thing. So we’ll take a breather for a
minute, and ask folks to think about this next question. So you’ve heard a lot about the
study and how the researchers took a look at the poverty
schools, and we were wondering what improvements do you think
would be needed to increase principal effectiveness especially in high poverty schools? So we’ll open up
a poll for that. And for any folks who are
interested, we didn’t put it in the resources box, but you see
that Lauren posted a link in the parking lot. That’s a great article that
summarizes some of the work he just described. So if you want to take
a look at that, it’s some great reading material. So, moving back to the question,
Rick, let’s take a look at some of the responses there. Ways folks are characterizing
improvements that could help increase principal
effectiveness. Talking a bit about
communication, I know a little while back in the parking lot
there was some talk about engaging parents in different
strategies to do that. Understanding the culture
and the specific needs of the school community.>>ERIC HANUSHEK: So, I will
give you a little bit of response to this. But in general, I think Andrew’s
characterization is correct, that what we’ve been doing is
looking at a black box here. Something is happening in that
box that leads to different principal performance, and many
of the suggestions that are up above seem plausible to me. We don’t have the research
evidence, or I don’t, to support which ones to pursue. But the overall answer of this
research is, we have to take this much more seriously. We just cannot let the
leadership of schools be-sort of float along without being
actively involved, because it’s having a huge impact, at least
by our estimates, on where students are learning and
how much they’re learning.>>ELIZABETH GRENINGER: And I
think to piggyback off of that, Elizabeth makes a great point
about the fact that there are so many extraneous tasks that
principals can become consumed with, that take them away
from the work that focuses on student achievement. So we’ve had a lot of people
mentioning the buzzword of instructional leadership, but
in looking at where principals really spend their time, it
seems that their time is divided, and their focus
has to be pulled in a lot of different directions.>>ERIC HANUSHEK: That’s
precisely the case. That’s precisely what I was
saying before, that the people have done the research by going
into the schools buildings and finding out how principals spend
their time and so forth find that they are distracted by all
kind of things that are quite different than instructional
leadership, or many of the items that are listed here.>>ELIZABETH GRENINGER: We’ll
give folks about 30 more seconds to write responses there. And while they’re doing that,
Rick, there’s a question over here in the parking lot
from Renee, if you want to look at that. She is wondering about how
students with behavior problems-I lost it.>>ERIC HANUSHEK:
I have it here.>>ELIZABETH GRENINGER: Okay,
I think essentially what she’s asking is how can effective
principals support the situation where there may be schools with
students with a high degree of behavior problems, no parent
engagement or little parent engagement, and how can a
principal, how does a principal support that type of situation.>>ERIC HANUSHEK: When you,
when it gets down to how can a principal do it, it’s actually
getting beyond my research, and it’s more into the realm of many
of you who are actually out into the schools. What we’ve found is that there
is a larger variation in effectiveness of principals
in low-poverty schools. Now, that could be a reflection
of the fact that there’s just many bigger differences in the
principals that are assigned to poverty schools, that some are
really good and some are really bad, and that shows
up in difference in student effectiveness. Alternatively, it could be that
it’s just that the job is so much more difficult or requires
so much more in high-poverty schools than in low-poverty
schools, so that if we had two equally as good, equally as
prepared principals, they would have very different effects in
low- and high-poverty schools. Our research really can’t
differentiate that. All of our research is saying,
we see huge differences depending on which principals
are assigned to the school, the differences I’m going to claim
are, I’m going to claim are large enough that we ought to be
thinking about the policies and the training and the background
and the preparation and the mentoring and the decision
making on retention of principals much more than we do
today, because these differences are so large that it looks like
we’re letting important issues of school performance
slip by the wayside, by not addressing them.>>ELIZABETH GRENINGER:
Thanks, Rick. I think that’s helpful. As you stated up front, this
goes well beyond what your research looked at. I think some of the practical
solutions and conversations that are happening here in the
webinar and can continue to happen beyond today’s webinar
among colleagues, really help to craft a solution, and think
about in reality, and in each individual context, how to
tackle some of these issues. We’ll have a little bit of time
at the end for folks to share some of your own strategies, more than what you’ve already been sharing. We’ll take this poll off, and
we’ll continue on with the last portion of the webinar.>>ERIC HANUSHEK: Coming back to
what we’ve seen in Texas, the data part of it, one of the
things that we want to look at was how much principal turnover
was related to poverty schools, and whether the principal has an
effect on teachers, in addition to just moving themselves. And so one of the ways that we
looked at this was to look at teacher departures. And this is a very complicated
diagram that I have for you. But what we’re trying to do is
to first, each of the blocks are quartiles of the poverty
distribution of schools, so the upper left-hand quartile are the
most advantaged schools, the ones with the lowest percentiles
of poverty children and down to the bottom right hand
column, which is the most disadvantaged schools. Then what we have done here are,
the rows in each box, if we just concentrate on the most
advantaged schools in the upper left-hand column, the rows are
rankings for the quality of principals based upon student
achievement from the bottom, the least qualified principals, to
the top which are the best, most effective principals. And what you see is in the
schools with the poorest principals, the bottom of the
distribution of principals, we get a lot of teachers who just
disappear from the schools, and we get a lot more teachers
that just change districts. So that the biggest difference
is between the least effective principals and all the rest, but
they’re showing that teachers are both leaving teaching and
leaving the school district much more frequently with
poor-quality principals. Now, if we go down to the most
disadvantaged schools in the bottom right-hand column, we see
some of the same differences, that again, the poorest
principals have a harder time retaining teachers. We have more teachers that are
changing districts and that are just leaving
teaching all together. So there’s some direct
relationship between the quality of the principal and what’s
going on with teachers. We actually also look at
whether better principals lead to better teacher transitions. Not all teacher
transitions are bad. If a principal makes a decision
about removing an ineffective teacher, that’s a good decision. If a principal makes a decision
or causes an effective teacher to leave, that
would be a bad one. What we see is that better
principals, at least in Texas, seem to lead to the better
kinds of teacher transitions. What we see is also that there
is much more transition of teachers out of a school if-in
the grades where the students are performing the poorest. In other words, it looks like
from the data that principals concentrate their attention,
or good principals concentrate their attention on grades where
there’s achievement problems, and there they look to replace
the teachers in grades that aren’t working very well. We’ve tried to put together all
of this evidence about teacher turnover with
performance in the schools. And do principals make
better personnel decisions? Well, this evidence is a little
bit limited, but the answer seems to be yes, that we
are seeing principals-good principals making better
personnel decisions. The more effective principals
who place a greater portion of teachers in grades where the
gains-you’ve got to do the translation on my slides also,
I’m sorry-but the more effective principals replace a greater
portion of teachers in grades where the achievement gains are
the less, and what we’re seeing is that less effective teachers
tend to leave schools where the principals are better. Now, that’s indirect evidence. We don’t know who’s
making the decision. It could be that poor teachers
realize that they are not up to the game and leave
on their own account. Or it could be that the
principals are counseling them out or pushing them out
in one way or another. But overall, what we see is that
teacher transitions look better when we have a highly
effective principal. So let me try to talk a little
bit more about the interaction of poverty with this. What we see is that, in terms of
principal quality, that both the most and least effective
principals tend to depart from the toughest schools, the
schools where the population is the most disadvantaged. So that the middle-quality,
middle-effective principals are the ones that are more likely to
stay in schools, particularly disadvantaged schools. We do see in the lowest
poverty schools that effective principals tend to be up to the
challenge and tend to stay there most often in the
low-poverty schools. What we see if we look at where
principals go when they move, we see a substantial number of
least effective principals actually just go to another
school, which we think is potentially pointing
to a poverty problem. If we have the dance of
the lemons with respect to principals, it would not be
particularly good for their overall system, that even if an
ineffective principal leaves, they turn up someplace else. This is especially true
in high-poverty schools. And so we think that that’s
something that needs to be looked at more. We don’t have enough detail here
to know what’s behind that. But it could well be that
principals in a high-poverty school that do badly are sort of
given a second chance, sort of not being sure whether it’s the
principal or just the tough environment, or what. But we do see that low-effective
principals in high-poverty schools tend to go to other
schools, and that worries us. So, that’s, you know,
the summary I think is pretty obvious. The research is hard
to be precise about. But I’ve tried to give you a
sense of where we’re confident and where we aren’t
confident, and what could influence the data. But what we’re pretty sure about
is that there really are big differences in
principal quality. And that these differences, at
least, have a bigger impact on high-poverty schools, where the
challenges are most important, they’re most important to
society in solving the problems of high-poverty schools,
and that principals are an element of this. And right now they’re
contributing potentially to some of the results we see. I’ve said this enough that I
don’t have to repeat that we think that the impact of
principals is very large. Let me just end this portion by
telling you where we’re going with this work. We’re actually working quite a
bit on this right now, as we speak, my coauthors are working
on this whole problem of getting more precise and
more detailed answers. What we’re doing is going from
our seven years of data that we had for this study to 18 years
of data, which gives us enough detail that we can look at a
large number of the questions about just what are the
transitions of principals and where are they going, how does
the clear path of a principal unfold as we look at it. And we also are going to look in
a lot more detail at the role of salaries that was brought
up earlier and of the accountability system and
whether that’s having an influence, positive or negative,
on the movement of principals. What we can do is look at the
Texas ratings of schools and how that interacts with the
mobility of principals. What-then there are some other
details of looking at how the impact of a principal evolves
over time, when that principal has a longer tenure within
the individual school. So those are the
places we’re going to go. To us, this is research that
points to an important issue, but it leaves lots of open
questions, as we’ve seen in the parking lot, where people have
great questions that we hope to be able to answer somewhat. But many of these questions are
actually going to be answered by you participants in the webinar,
who are closer to making some of the decisions and the
policies that are important with school leaders.>>ELIZABETH GRENINGER:
Thanks, Rick. Before I move on to the next
poll, there’s two questions that relate somewhat back to the
discussion you were just having over here in the parking lot. Andrew is asking if you
simulated or modeled the pattern of teacher departure to estimate
what share of the observed impact could be explained by
differential teacher departure.>>ERIC HANUSHEK: We
don’t have that right now. The way we’ve done this is by
looking at across different grades in a school and whether
principals are making better decisions across
different grades. What we hope to do in some of
our work is to go into much more detail about precisely which
teachers are leaving by essentially estimating the
value-added of individual teachers and looking at how
movement of teachers with different value-added or
different effectiveness move with different teachers. But we haven’t been
able to do that yet. It turns out that that takes a
lot more data, and we’re just in the process of being able to
look at the questions like that.>>ELIZABETH GRENINGER: Steven
asks a question that, I think, my hunch is that you haven’t
studied it, but I’m going to pose it so that you can share
any insight on this question. Whether you were able to study,
your study, to show the impact of an effective principal on
improving the poor teacher to becoming a better teacher. So we know you talked about
movement of teachers out of schools, in or out. But how about just improvement
in growth in a teacher? Do we have->>ERIC HANUSHEK:
That’s an excellent question. That’s basically what-one of
the things we want to look at. I’ve answered the prior question
by saying that we want to look at the value-added of
teachers that move or stay with different principals. One of the things we also would
like to look at is whether the value-added of teachers changes,
according to whether they have a good or bad principal. And we haven’t been
able to do that yet. And that takes much more
detailed information about precisely who the teachers are
and what the impact of each teacher is, in addition
to who the principal is. And we’re trying to
correct that problem. But it really takes a
lot of data to do that.>>ELIZABETH GRENINGER: Okay. One more question here
from Liz before we move on. She’s wondering what are your
thoughts on using school-level value-added as a measure of
principal effectiveness as opposed to using a
model that calculates principal fixed effects.>>ERIC HANUSHEK: Well,
this is getting at the same kind of issues. But it’s-we haven’t had enough
time to try to sort out different ways of
estimating these. The principal fixed effects
look a lot like school-level value-added during the tenure of
the principal, during the spell of the principal. But we haven’t solved some of
the details of estimation that you’re bringing up. And I’d be happy to talk to you
offline about that, Liz, because that’s getting it beyond what
I suspect most of the audience wants to get into.>>ELIZABETH GRENINGER: Great. And we’ll share Rick’s contact
information at the end of the webinar, so that folks, if they
have more specific questions, can reach out. So, at this time, let’s
move on to the next question. And we’ll be interested in
finding out from all of you what patterns of principal mobility
have you seen in your research or in your state or district? So has it mirrored what we have
discussed today, that we have seen in Texas, or have you
seen different patterns? And Rick, while folks are
answering that question, we did receive some questions prior to
today’s webinar that folks sent in, and one of those is
pretty simple to answer. But I’m hoping you can share
with folks what standards they can refer to that qualify
school leaders as effective. So, describing the standards
for school leadership.>>ERIC HANUSHEK: This
research borders some on that. But what we have been unable to
do is to pinpoint background or preparation of principals that
relates to their effectiveness. This research on principals
looks a lot like the research on teachers. In the teacher work, what we
find is that effectiveness is largely unrelated to the degree
levels of teachers, mostly the experience at least past the
first couple years of teacher experience, but their
professional development and their certification and so
forth, and that’s been a pretty strong finding, of
the teacher research. To the extent that we have
looked at that so far, principals look
exactly the same. Some people are really good
at it, and some people aren’t good at it. But it is not very closely
related to the background and preparation of the principal. And so, to the extent that
“standards” mean what do you want a principal to look like
before they can be a principal, I don’t think that we have
any good answer to that. In fact, our answer is a little
bit to the negative in that the simple things that we’ve looked
at about background in terms of preparation and so
forth don’t seem to be very related to effectiveness. So that suggests that you would
not want to use those things as the standards for what a
principal should look like. If you think of standards
in terms of outcomes and effectiveness, I think that this
research is probably too crude to give you actual measures. You wouldn’t want to write into
the Texas state education code that you can’t be a principal
unless your fixed effect is .05 standard deviations or
something like that, because we’re just not at that point to
use any of these measures in terms of the direct evaluations. What it does point to is that
decision making about principals should be more strongly related
to student outcomes and student performance and that that’s what
you should pay attention to.>>ELIZABETH GRENINGER:
That’s really helpful. One thing I know, and just
guiding folks and many leaders on this call are probably
aware is ISLLC standards as a practical guide for school
leaders and a standard that they can shoot for-I think Lauren
just put that link up there-so kind of characterizes
some of the description you just gave us. So let’s move over here to the
poll question and take a look at some of the responses there,
what folks are seeing in terms of principal mobility
in their areas. Holly is noting that in Maine,
they’re seeing a lot of mobility among their
high-poverty schools.>>ERIC HANUSHEK: I’m sort of
looking at Kevin’s question, which is the last on there, that
suggests that paying attention to value-added of teachers may
in fact be just putting too much stress on them and
demoralizing them. You know, exactly how you use
quantitative assessments, those teachers, or other evaluations,
I think is still an open question about both, for both
teachers and principals. I think the evidence suggests
that the impacts of principals and teachers are large enough
that you want to pay attention to who are really effective and
who are really ineffective. But we obviously have to be
careful of how we incorporate that in the evaluation system
and making career decisions about teachers and principals.>>ELIZABETH GRENINGER: A couple
folks are commenting about some of the more effective principals
moving from the low-performing schools, either into higher
performing schools or out of schools in general. And you know, that’s something
we see with teachers as well. This exodus from the teaching
profession because of a lot of the things that we’re
already discussing.>>ERIC HANUSHEK: Absolutely. That corresponds also to the
data that we put right up at the very beginning of this
presentation, which shows that in high-poverty schools are much
more likely to-high-poverty schools are more likely to have
new teachers and more turnover, but high-low-achieving schools
are really almost an extreme, that low-achieving schools have
lots more principal turnover and principal exodus, and that is
consistent with the fact that principals are looking to move
to districts that have higher achievement and are doing
better, and that’s something that we actually have to
deal with is, from a policy standpoint, we have to find ways
to ensure that highly effective principals and highly effective
teachers are in fact willing and able to perform in our most
disadvantaged schools.>>ELIZABETH GRENINGER: One
question I want to raise but I don’t think we have really
time to get into it. Both Andrew and David talk a
little bit about some nonpublic schools, so maybe charter or
independent schools and the success of principals
in those settings. And I’d be curious to
what the research shows. It’s clear yours focused on
public schools in Texas. But you may be able to point
people to a direction to learn more about that.>>ERIC HANUSHEK: So, our data
will include charter schools in Texas, as other public schools. We haven’t separately isolated
them and looked at them in terms of either leadership or in
terms of teacher movements. We have been doing other work on
charter schools in Texas right now, and it’s not
publicly available yet. But what we’ve seen in Texas is
that the charter schools have in fact improved over the
last decade noticeably. We tried to get at part of this
by actually surveying individual charter schools and to talk to
them about what they thought was most important in changes
over the last ten years. And in virtually every survey we
conducted or each interview we conducted, somebody mentioned
at one point or another that leadership is important, and
that they in fact, in the charter school sector, appear to
spend more time both screening and developing leaders for their
schools, and that appears to be the case in the
traditional public schools.>>ELIZABETH GRENINGER: Well,
the 5 o’clock hour is creeping right up on us here
on the East Coast. So we’re going to
move right along. We have a few more slides. Then one more final question for
folks, try to wrap up so people get out of here in
under two hours. We have some
general conclusions. I’m not going to spend a whole
lot of time on these two slides, because Rick has already really
gone into the summary and kind of what the bottom line is here. It’s clear that leaders matter. We’ve talked about that
from beginning to end in today’s webinar. And we need to spend some time
focusing on principal evaluation and rewards, take a look at
credentialing and how that relates to performance, again
linking back to standards and our recent discussion of that. And to do some more research on
the development of principal training and mentoring. Another summary of some of the
things we’ve been talking about. And, so now we really want to
ask you all what you gained from today’s webinar, and what you
intend to take away back to your context, your school, your
district, your research setting. And your answers could really
help others in thinking about how they could take
action that’s meaningful. I’ll mention a few things as
folks are typing, just so we can get through all of the
housekeeping-type things at the end of the webinar. We have the PowerPoint down
below for you to download. You’ll be receiving an e-mail
in the next couple of days that asks you to take our survey,
if you don’t take it today. I will provide the link
for you in just a minute. If you feel that this is a
webinar that you would like to share with others who may not
have been able to attend today, you’ll have a link to the
archived webinar, and you can feel free to share
that with others. And you will see in just a
slide or two Rick’s contact information, as well as mine. If you have questions for either
of us, Rick more on the topic of effective school leaders and if
you have general questions on the REL webinar series, you can
feel free to reach out to me. We hope you will plan to attend
some of our upcoming webinars. You’ll see in one of the
upcoming slides dates and information on those webinars,
both the Teacher Effectiveness and our Ask an Expert series. I’ll shift back to taking a look
at some of these responses. Looks like across roles folks
are going to share information with their principals or with
principals that they work with. Rick, do you have any final
thoughts or comments on the takeaways that folks had?>>ERIC HANUSHEK: Support your
good principal and get rid of your bad principals (chuckles).>>ELIZABETH GRENINGER:
Easier said than done, right?>>ERIC HANUSHEK:
No doubt about it. I say that easily, from the
standpoint of researcher. But at the same time, I think
it’s something that we have to start having a serious dialogue
about, of how we do this.>>ELIZABETH GRENINGER: Well,
at this time, we’re out of time for today. Thank you all for
the great engagement. I’d like to give a virtual round
of applause to Rick for his time and his effort in sharing
his expertise and knowledge on this topic. As I mentioned, I’ll advance
to the slide where you can see his information. Here is the way you can get in
touch with him or with me or any general questions about the
work the REL Mid-Atlantic does. And we have a forum on teacher
effectiveness, which you can continue the dialogue. As I mentioned, here
are upcoming webinars. We hope to see many of you
back for one of these events. And you should see the
evaluation has opened in a browser on your screen, and we’d
love you to take that feedback survey, so that we can continue
to improve our webinar series, and meet your needs. And thank you all. We hope you have a great night.>>ERIC HANUSHEK: Thanks,
everybody, for attending.

Leave a Reply

Your email address will not be published. Required fields are marked *