Helpful Guidance for an Effective Search Process


Good afternoon, everyone. I’m Judy Singer. I’m the senior Vice Provost
for Faculty Development and Diversity. And I want to thank
you all for joining us this afternoon for this event
on the faculty search process. I think we have a
very interesting panel of experienced search
committee members and chairs. We have a lot of people in the
room who, themselves, have been on many search committees,
are department chairs, deans, a lot of people in
positions of responsibility. And I just want to start out
with my first introduction to the importance
of faculty searches. So in the mid ’80s, I met
Derek Bok for the first time. I was a junior faculty member,
I was an assistant professor. And I think I might have been
half an inch taller than I am now, but for those of you
who remember Derek Bok, in his heyday he was like 6’6″. And there was easily a
foot and a half between us, as well as other
power differentials. And I met him at
a cocktail party, and where– and I’m
not even sure what the purpose of the party was. I was just totally tongue tied. Here I am with the president
of Harvard University, and my research collaborator,
John Willett, was with me. And he came up with a
question to ask Derek Bok. He said, Mr.
President– which is what you would call
Derek Bok– is it true that you review every tenure
case at Harvard University? And he looked down–
John wasn’t much taller than I am– looked
down the two of us, and he said– I can’t
imitate his voice– in a very low base, well yes. That’s the most
important thing I do. I determine the Harvard
faculty of the future. And I thought that was
such an interesting view from the perspective
of the president, that in making decisions
about tenure, in determining who the
Harvard faculty are, that he was really setting
the course for Harvard well beyond his own
tenure as president. And I think for everybody
in this room who’s participating in
a faculty search, you are carrying that mantle. Part of why you’re
here is a commitment to doing searches well, and
also thinking a little bit differently than might have
been thought about in the days of Derek Bok’s presidency. The Harvard of
the future we want to be as excellent,
or more excellent, than the Harvard of the past. We also want the
Harvard of the future to be more diverse than
the Harvard of the past. And one way to get
that excellence is to look at that diversity. Just to cite a
statistic, in 1970 there were no tenured
women in the faculty of Arts and Sciences at Harvard
University, zero, in 1970. That’s right– I was in
high school at the time. Some people have
been in college– no women were considered
acceptable members of the faculty of Arts
and Sciences in 1970. I’m very pleased to say that
that has really changed, and I think the commitment
of people in this room to putting energy into searches,
to thinking a little bit outside the box. To not always say, well,
we’ve always done it this way, which is too often
a Harvard refrain. So I just want to thank
you all for being here. And thank you for your
efforts in being on a search. How many of you are actually
on a committee this year? Yeah, a lot of people. Being on a search is an
enormous amount of work, and I just want to thank you
for that effort and that time. I’m going to conclude my
remarks by introducing the moderator for today,
Mahzarin Banaji our colleagues here in the
psychology department. She’s the Clark professor of
social ethics in the Department of Psychology, and
the senior advisor to the dean of the faculty of
Arts and Sciences on faculty development. She is also one of the
world’s leading experts on concepts of implicit bias
and implicit association. And when I give talks around
the country, around the world, on faculty development
and diversity, I often say– I’m going
to embarrass Mahzarin– that I have an unfair
advantage by having Mahzarin as a colleague and somebody
who I can count on to help work with me and with all of
you on faculty searches, and changing the
way we do things. I’m also going to
shamelessly promote her book. So this is a book that
came out last year. It’s called Blind Spot. It’s a great read, and she’d be
happy to autograph it for you. And it gives you some sense
about the implicit biases, and what she calls mind
bugs and blind spots, that we all carry with us. And one of the important
lessons is, I’ve got them, too. Mahzarin’s got them. It’s human nature. And I think, trying
to think about how we wrestle those demons, and
make good decisions in the face of inputs that might send
us in different directions, is part of what we’re
here to do today. So with that, let me
introduce Mahzarin Banaji. Thank you. Thank you, Judy. The only thing I’ll
add about the book is that its price is
falling precipitously on Amazon, even as we speak. So it’ll be worth
pennies if you wait two weeks before you buy it. Absolutely, among the most
exciting and important things we do is to select
people who will become a part of this community and
will eventually replace those of us who are in this room. For me, one of the
great surprises of my professional
life was that I made a discovery, maybe
30 years ago now, that was an unexpected effect
in an experiment that became so startling
to me that I decided to drop everything else I
was doing and focus on it. And it has been the work I’ve
done these last 30 years. But of even greater
surprise to me is the fact that anybody outside
of my 10 academic colleagues would care about it. But such is the state
of this knowledge, that people are using it. Hopefully they
will use it wisely. But everywhere you
go, people want to ask the question, how
can I make better decisions about the people I select? And until very recently,
there was no science of this. We assumed that, as people
who have technical expertise in a particular field
or work in– that that’s the expertise that will
drive the decisions that we will make. I have a certain set
of arguments to make for why is the right candidate. And then of course, the
science comes along, and over a 50 year period it
has produced many, many results that make us really rethink the
rationality of those choices. We won’t go into the
research results today. What we want to hear from is
almost the other side of it, real anecdotes,
things that people have experienced in the
trenches that we can learn from. And as Judy and I were
chatting before this session, I said to her that
even as a scientist, I’m often surprised at how much
the data don’t matter to me. And how much those anecdotes do. It’s a shameful thing to
say, but these stories about what people actually
encounter in the trenches remain with us. And they motivate us
to try to do better. Let me give you a quick
little example of something I experienced in
my own department as one of those
anecdotes and what we did about it with Judy’s
and Nina’s assistance. So we ran a search– I think
it was about a couple of years ago now– in which the field,
the base rate in the field is that 50% of the
applications, the junior search, 50% of the applications came
from men and 50% from women. And by the way, gender
is not by any means the only thing to be
worrying about here. This is a very broad
landscape, where we ought to think about every
aspect of a person’s category that might in some way
lead us to think that they were either the
perfect fit for Harvard or do not fit at Harvard. So 50/50 is the base rate. The committee does
due diligence, and they come up with
the five best people that they want to interview. And they come to
me because they’ve discovered that, after
their due diligence, what they have are five
outstanding male candidates. And they wanted
to know what they could do to understand
if this was, indeed, a decision that may have
incorporated some bias. I look at the
application pool myself, I come up with exactly the same
five people they would have. There was nothing I could
see wrong in the process, and that’s what it was. If that were one
year, we would say, anybody who understands
the law of small numbers knows that that
can easily happen. But we looked back
at our own searches, and we discovered that
that had been happening in pretty much every one
of the last six searches that we’ve had. And so we had to pause. And I’m going to
urge you that when you come upon data like that,
don’t just plod forward. Ask why that might
happen, even if there is no obvious reason as to
why that might be happening. So we did something,
the chair of the search. And I’m going to actually
sort of give a shout out to people who’ve been recently
tenured at Harvard, because I think that the baton
has been passed to you, and it is extremely
important that you play a role in raising issues
that your predecessors may not have, because you are younger. And as a result, you know
this evidence better. It was taught to you,
perhaps, in the class, which is not true for people my age. So the person who chaired
the search and who knew about this research
came to me and said, I can’t go forward,
he said, given what we’re seeing as the base
rates and then our selections. So he did something
very interesting. He wrote an email to
junior people in the field, including in our own
department, and said, we’re doing a search in X field. Who are the candidates
that come to your mind who you think ought to be people
we should think about? Six names came to us
repeatedly as the best people in the
field, and they all happened to be names of women. Not one of those six
women was in our pool. So why would that happen? Why would a junior
person, who clearly belongs in the top 5% of our
pool, not be in the pool? At that point, we
talked to Nina, to Judy, and asked for permission
to call these women and ask them, why did you not
apply to the job at Harvard? And to a T, the
answer was, I didn’t think I belonged at Harvard. Your advertisement read,
seeking exceptional candidates in blah, blah, blah field,
and I don’t think of myself as exceptional. Mind you, this from
a person who had applied to a similar position
at Princeton and at Stanford. So I’m here to
tell you that there are many ways for
us to think about, not just selection,
but even recruitment. I give you the example of this
kid, the 16-year-old Mongolian kid, that MIT brought
from Mongolia. This is a kid who lives
in a semi-nomadic group, manages to find access to the
internet for a few hours a day, and ended up scoring
so high in a MOOC that MIT teaches, anchoring sort
of the high end of the 150,000 people who take this
course, that eventually they sort of sent an administrator
or dean to Mongolia, they packed up his bags,
and they brought him to MIT. I would say, that’s recruiting. And I would like us to
do something similar. Just yesterday, we were
having a discussion when it struck us that somebody
had been discussed at Harvard, but we didn’t really
follow through. And now it turns out that
she is the leader in a field that we, maybe, should
have thought about. These are no longer little
offenses a mistakes. This is tragic, if we can’t
bring the very best people here. So today the job is to think
only about one area, selection, even though there are many
parts to this problem. And for us to begin with
three people, all of whom have shown amazing
commitment, in my opinion, to improving the integrity of
the process that we employ. I’ll begin by introducing our
first panelist, Iris Bohnet. Iris is a professor of
public policy at the Kennedy School of Government. She’s also the director of the
women in public policy program, and she serves as as Kennedy
school’s academic dean. Iris’ training is that of a
behavioral economist, which means that she doesn’t
just have models of how humans might behave. She actually does
some experiments and shows us how economic
theories may be right or wrong. She studies mostly things
like trust in decision making, but using gender
as one of her variables, and has been interested
in this method that has been labeled, “nudges”. The idea is to– so
psychologists like myself, we think about individual minds
and how can we change a mind to go from incorrect
decision making to correct decision making? But Iris doesn’t worry
about individuals. She thinks they’re a lost cause. And she, instead,
thinks about shifting the gear outside of the person. That is through
institutional changes, through policies, and so on. And what she has done
is written a book that she turned in only– Tuesday, Tuesday, titled “What
Works: Gender by Design”. Lovely title. Please come up and
tell us about it. Thank you very much. Thank you very much, Mahzarin
for that kind introduction, and Judy for putting
this together. So I might be
violating some rules. I’ll mainly tell you
anecdotes, but I’m also going to share some data. So also I should
correct, I actually stepped out of the
academic deanship. So I’m on sabbatical right
now. [INAUDIBLE] Start by reflecting on my
time as academic dean. So one time I came
to the office and had a group of students camp out
in front of my office door, saying that they needed
to see me immediately. And so we met. And they said, what I
interpreted as saying, that they were concerned
about the lack of women on our faculty. And so I talked
about faculty hiring, and at the Kennedy
School we only hire about five people every year. And I did the math
and told them that takes about 108 years
till we have quality, and things of that sort. But that’s actually that’s not
the main point of the story, because in my discussions, I
realized that they actually could care less
about the faculty. They cared about– I
don’t know whether that’s equally important for
every department– but they cared about people
who they see in the room. And there’s lots of fellows
at the Kennedy School. There’s post-doctoral fellows,
there’s doctoral students, there’s guests,
there’s celebrities. There’s leaders from
around the world who speak in our seminars, in
the forum, and other places. And they were really
concerned about the fact that they don’t
see enough women. A that opened the
question for me that I don’t think anyone
at the Kennedy School has ever paid attention to. And that is– We’re recording. Thank you. And that opened a question
at the Kennedy School to me that I think nobody
had ever looked at. And that is, who is
coming through school. I mean, what is the gender
diversity or generally diversity? But in this particular
instance, they were considered about
gender diversity? Who speaks at our school? So you have 12 different
research centers. They all have their
own seminar series. Nobody had ever counted. So some of the advantages
being academic dean is that I could ask the research
center directors to report every January to me
what the breakdown is of all the fellows,
and a the speakers, and any speaking engagement
that they have at their research institutes. And what we learned and they
learned was really eye opening. So you might imagine,
some of our centers had a breakdown of 90% male
speakers, 10% female speakers. The women in public policy
program that I direct had the opposite problem. So our breakdown, typically,
is 80% women speakers, 20% female speakers. But it is true, overall, we
had about 68% male speakers. I tell you the story,
because I do think that’s an important insight. That counting, measuring what’s
going on is really important. And I thought it was
shameful for me– not only what I kind of
latently working on the book that Mahzarin mentioned,
but I work with data a lot, but it had never occurred to
me to think about that question until the students came to me. So just lots of
learning involved, I think, this whole space,
for us collectively. When I reflect on
the selection process at the Kennedy School
for faculty selection, this is now not a
scientific segment at all, but I do think one of our
most important innovations that precedes me as
academic dean was to have an appointments committee. And I presume most of
you have that as well, but what they do is to
safeguard the rules. So first of all, they
had to design the rules. And then they fine
tune them all the time, kind of reviewing our
job ads, reviewing how we do things, how many
interviews has this person had. How long is the job talk? Who is the person meeting with? How we evaluated the person. What we reached out to. So now lots of things that
faculty have to submit. Faculty committees have to
submit where they advertised, who they reached out to,
what the applicant pool was, how the final selection list
compares to the applicant pool, lots of this. And I think it actually
is very helpful, because what it helps
us to do is also calibrate across searches. None of us is an
expert in this field. Neither am I, but it does help
to look at various searches comparatively. Look at the kind of
standards that we apply, where we look, how
many people we invite, how many letters of
recommendations we ask for. So I do think that was
very, very helpful, and really guided by Judy’s
office, kind of regularized some of the process. Now thirdly, I’m going to
talk a little bit maybe also about my
research, but one study that you probably are
familiar with that affected me and my [INAUDIBLE] the fact that
I turned from studying trust to studying gender, was Claudia
Goldin and Cecilia Rouse’s work with orchestras, with
the short summary that if you auditioned
behind a curtain as a woman, the likelihood that it will
advance to the next round is increased by about 50%. So blind auditions
was kind of my ideal when I became academic
dean, but that’s, of course, not how we normally do things. So I have three
slides, just kind of to make three points
here, maybe really more two. The first one is a
slide on interviews. And I’m sure my colleagues will
talk about many other things, but all of you probably have
job talks and interviews. And I’m going to suggest that
almost everything on this slide is wrong. So the first thing
that is wrong is, we shouldn’t do
panel interviews. I think they’re relatively
rare in academia, so I won’t spend a
lot of time on that, but there’s lots of evidence
suggesting that group think is just going to emerge when
we have panels interviewing people. The other myth that surprised
me as I was collecting data for the book
is that diversity on selection committees
doesn’t solve the problem. And that is because
Mahzarin and others have shown us, in much work,
that seeing is believing. If we don’t see male
kindergarten teachers or female mathematicians,
we don’t naturally associate those jobs with
men and women, respectively. And we often find
relatively small differences in the eyes of the observer. In fact, there’s really nice
randomized control trials, experiments in Spain,
where faculty selection committees are created
randomly, picked out of a hat from across the
faculties in Spain. And so what researchers
could do was literally analyze, what difference
does diversity make on that selection committee? And they found some
surprising results. So for example,
what they did find was an out group bias for
committees evaluating people to promotion to
associate professor. What does that mean? Women didn’t want to
promote other women. It was particularly
pronounced for women. Why is that? Because women had some
theory that there are– there’s gender
specific competition. That I’m competing with
other women for the 10 slots in Spain, and therefore,
I don’t necessarily want to have more like
me in my department, or even– because this is
Spain-wide in my discipline. It turned to an in-group
bias, so that means the more women on the
committee, the better for the evaluated women for
promotion to senior professors, so to full professors. At that point, women
were looking for friends. Now they’re looking for
people like themselves. So there’s actually this really
interesting and complicated stuff going on, in terms of
kind of diversity of committees. Now, don’t take that as me
saying, diversity on committees is not important. I think it’s important
for other reasons, because we have
different networks and reach out to
different people, but I have no evidence
to say that it is important in the
evaluation process. The third thing
I’ll say about this is– and I’m characterizing,
obviously, here a bit too much. What I mentioned before,
calibration across candidates is really, really important. Obviously, they won’t only look
at one candidate, I’m sure, but that’s something that turned
out to be very important for me when I became academic dean. And since then, I mean now as
a faculty member interviewing job candidates,
I’m really trying to adhere to that principle
that I compare candidates with one another. And I’ll show you in a
moment how I do that, and in a very, very
disciplined way. Fortunately, that’s my last
point I’m going to use. They might be using
what is called an unstructured interview. And I presume most of you use
an unstructured interview. I have always used
unstructured interviews. I have to tell you,
these are probably the worst predictors
of future performance that you could imagine. Now to– let me
actually very quickly give you one study, because it’s
so important and so relevant for us. So Texas had a– did run a
very interesting experiment, because the government of
Texas realized in May they didn’t have enough physicians. That was after all
the medical schools had chosen their students. So it turned into a
natural experiment because then the
government of Texas said, you all have to admit
50 more students. So now they have to
go back to the pool, go deep into the
pool, and select people who nobody else wanted. And it turns out– and
then what’s beautiful, then they could follow those
applicants over time and see, did the people who we initially
ranked number five, and who we admitted, do they do
better than the people who we initially rank number
1,000 and had to admit, just because Texas
state made us to do so? It turns out, as you might
guess, zero correlation. It doesn’t matter. And the beautiful thing is,
if you take out the interview score then the correlation
gets a bit more precise. So don’t overestimate
the interview. OK, but not that you have to
pay attention to this now, but so I did actually
spend some time to think about the
research on interviews and how we could structure
interviews better, because I don’t think we’ll give it up. That is kind of an
interesting human bias that many social
psychologists have actually been trying to work
against for 50 years and we haven’t been successful
in convincing people that the interview is useless. So I don’t think it’s
going to go away, so I do think we have
to just make it better. And there is evidence suggesting
that structured interviews do a much better job than
an unstructured one. And some key components
of a structured interview are, you ask the same five
questions, every candidate, in the same order. And ideally, afterwards,
you force yourself– and that’s exactly what
I did– to blind yourself as you possibly can to your
notes– I mean, to the name of the person. And you compare,
question by question, across the five
candidates, because what you’re trying to
do is you’re trying to protect yourself from what
is called the halo effect. You don’t want to be influenced
by whether you like the person, or whether the person
wore your favorite color, or whether the
person even answered the first question really well. You want to be objective across
all the different questions. So the very last
thing I’m going to say is this, and that’s become
a bit of a passion of mine. And that is, very small details
that you might not putting of can really matter in
this process, including what’s on your walls. So I’d invite all of you to
go back to your departments and have a look at
who’s on your walls. So the Kennedy School,
we noticed 10 years ago, that we had about 60 portraits
of leaders, typically public sector, private sector. Leaders, all of them male. All men. We had 50% female students,
we had zero role models for women on our walls. Now we’ve changed that. We’ve commissioned portraits
of Ellen Johnson Sirleaf, other– Edith Stokey, founding
mother of the Kennedy School, so we’re trying to change that. But go back to
your seminar rooms. I’ve been to some of
your seminar rooms. I’ve been to my seminar room. Some of our seminar rooms have
male portraits, exclusively. Imagine what it feels like for
a person applying for a job and getting a job interview
in such an environment. Thank you. So our next speaker, when I
said the baton has been passed to a new generation,
I mean Rowan Flad and the other
young people who’ve been tenured within the
institution recently. Rowan is John Hudson professor
of archeology in the faculty of Arts and Sciences. He is in the, I would
think, the stones and bones part of anthropology. Is that how you think of it? OK, all right. Rowan’s work focuses on the
development of complex society during the Neolithic period
and the Bronze Age in China. And what that does is it
gives him a great perch from which to look at whether
the decision making at Harvard University in the 21st
century looks a little bit like the Bronze Age in China. And then when he notices
that, he tells us about it, and we scurry around
trying to fix it. So one of the things
that he’s looked at are introduction of animals
in early Chinese society, their use and sacrifice. He has looked at social
change more generally. And there are, I think,
comments he’s about to make, that I find
especially compelling, in the way in which we identify
how our search ought to even be named. So with that, Rowan. Thank you very much, Mahzarin
and Judy for inviting me today. I feel a little out
of place, because I think I have much
shallower and more limited experience than the other
people in the panel, and probably most of
the people in the room. But I have been, I
guess, passionate enough about this issue to
get Judy to invite me here today in my limited time
in the tenured faculty here, partly because I think that
the process by which we go about selecting new faculty,
particularly junior faculty, is very important for
the entire pipeline into our senior faculty. And it is the way in which we
make this institution stronger. Partly, my interest has
come from an egregious lack of diversity in
our own department, particularly relative to a field
that is a very diverse field, both in terms of gender
and other categories. And so, that makes me take
pause when we’re doing searches. And I was fortunate enough
to lead a search a couple of years ago, fortunate
or unfortunate enough, to lead a search a
couple years ago during which we thought a lot
about this issue. And I’m going to mention
a few details related to that as I go on. But another reason why
I’ve been interested in this issue actually has
to do with issues related to implicit bias, in particular
because of work with my wife has been involved in
on motherhood bias in hiring practices,
and being exposed to a lot of literature
on implicit biases, and so forth, through that
kind of personal connection. So this is something that
I’ve been sensitive to and have wanted to bring that
sensitivity, to some extent, to the search processes
going on in our department. And so I’m going to have three
points, three main points. I don’t have any slides,
but I have three main points that I wanted to make
today, one of which probably deserved a little
bit of– maybe a slide. And they’re somewhat narrow,
in terms of their focus, probably because I
don’t have the command of the type of data that have
been talked about so far. Nor do I feel like I have
the depth of experience to speak to it in a much
bigger institutional sense. But perhaps some of the–
perhaps some of these points will be relevant to
other searches that are going to be going on. So the first point
relates to the kind of general idea of
insuring the rich talent pool for searches that go on. And I think one of the most
important components of this is the definition
of the job itself, including the job
ad, as it’s produced. But more, perhaps even
more fundamentally to that, thinking about what it is
that we should be searching for in our departments when
we are able to move forward with a new search, or even
when we’re requesting that. Because I think a
tendency– and certainly is the tendency
in our department, and I think in
many– is that when we have somebody who retires,
we want to replace that person. And we think that, first of all,
that’s a means by– a reason that the deanery is going
to permit us to do a search. And secondly, that if
somebody in our department has been a major player
in the field as a whole, through the lens of the work
that they do, and has impacted the field, that
perhaps that suggests that that element of the field
is particularly important. So the search I was responsible
for chairing a couple of years ago was an example
of this, where we had probably the most eminent
member– or at least certainly one of the most
eminent members– of our department retiring. He is somebody who
had been influential in the field in many
ways and had also sat in a position that had a long
legacy behind it of almost 100 years of people
within that field changing the way in which we think
about the origins of humanity. So clearly this is an
element of the field of archaeological
anthropology that has fundamental importance
to the way we think about our past and
the way we understand the development of humans. And it’s very easy, I think,
to think of just replacing that type of person, whether it
be at a junior or senior level. Unfortunately, the field
in which that person sat is the most non-diverse, or
one of the most non-diverse, sub fields within
our discipline. And so this presents,
I think, a challenge that is not unique,
by any means, to anthropology or archeology. And there are two
ways, I think, that we should be thinking about
at least evaluating that situation
when we are looking to replace people or
move our departments and our disciplines
in new directions. One of which is
whether or not there has been change
in our discipline, such that the fields that were
most important in the past are no longer the most
important, or at least can be folded within a
broader set of fields. And the second is
the way the job ad is written and
structured, in such a way– and I think that we should be
thinking about making, casting the net as broadly
as possible, and not having keywords and phrases
within our job descriptions that implicitly narrow the
fields that we’re searching in. So in the case that
I was involved in, this was a job that was
focused on the paleolithic and in the old world. So basically outside
of the Americas. And although the job itself
was structured in such a way that it really did
narrow to some extent the diversity of candidates
from which we’d applied, we were very conscious of
trying to both conduct outreach, and also consider
nontraditional applicants in terms of that position. So for example,
something that we did was do some outreach to
individuals in the field that people knew from
one way or another, or who might have ways to get
the word out about the job, that we’re not just
posting the same places. But in fact, I don’t think we
did a good enough job of that. Something that was
brought to my attention too late, or actually even
after this was finished, was one thing that we
probably should have done and that might be
relevant to many of you, is thinking about
posting job ads in places that are
explicitly targeted towards underrepresented groups. So whereas in an
anthropology one might post a job for the
American Anthropological Association or the Society
of American Archeology and other kind of
major venues, there are also small focus
groups that cater to underrepresented populations
within these societies. And I think that we
don’t often do enough, at least we didn’t, in terms
of specifically targeting such groups for
the advertisement of these sort of jobs. And that’s something that one
might want to think about. Secondly, charging
individual faculty within the department with,
not just those on the search committee, but everybody
in the department with actively
reaching out to people that they know, with an
intention of not just spreading the word, but
specifically spreading the word to populations and
individuals who may have interesting
path [INAUDIBLE], cutting edge work
that doesn’t obviously fit into the job
description, but nevertheless can be thought of
within that way, within the confines of the job. So that’s kind of one
major set of points, has to do with the– kind of
ensuring a rich talent pool. The second set of
points I wanted to make had to do with
evaluation of candidates. And this actually, the
second and third kind of go together to some extent. But I want to make a couple
of points here, some of which we adopted in the search
that I was involved in and some of which I kind
of reflected on later. I think that– and perhaps some
of these points are obvious, but I think they’re
worth making explicit. I think it’s very important
that searches do not rely on the individual
committee members reading files on their
own, ranking candidates, and culling the field down
with numerical rankings without any conversations
about the applicants to lead things off. And in fact, I think
that type of process allows for all sorts of implicit
biases and other factors to come into play that are
never explicitly made clear, and that have the real
potential for losing interesting candidates
who don’t obviously fit into the narrow way in
which a job was initially conceptualized. And so one thing
we did, and I think is good practice generally,
is to, at the very beginning, even though most of
the committee members had done some degree of ranking
of candidates on their own and thought about what
made certain candidates strong or not, we went
quickly in the first meeting through every single
person and talked about whether or not they– what
their strengths and weaknesses where. This takes more
time than it would if you just simply cut down
the candidate pool numerically. And in some fields, I
mean ours is not one where we have hundreds
of applicants, and so I would like to hear from
those who are in fields that have so many that
it would make it really impractical to do this. Nevertheless, I think that
this plays an important role in trying to diversify,
in terms of research interests, the pool of people
who look to be the most strong. Secondly, when ranking is
done as part of a process, I think it’s important to
have explicit conversations about the various aspects
of a candidate’s portfolio that can be ranked differently,
and also talk as a committee and as a department
about which of those are more valued in
a particular search. So for example, we broke down
in this particular search– although not, I must
admit, until the period when we were doing
the shortlist– the ranking of individuals into
a number of different aspects. The breadth of their
work, the depth of their work, the performance
that they had in the interview process, their research promise,
their teaching, and their fit were the six
categories that we had. And then within those
there were subcategories that I asked people
to think about, and particularly,
to think about how they ranked those subcategories
in relation to one another. So for example, in terms of
research promise, this kind of general category, I kind of
suggested a number of the sub components, promise as a
researcher, potential impact, demonstrated impact, scope
if vision, significance of research, trajectory,
relevance, and tenurability at Harvard, which I think is
actually a really important category that we think about
when we’re doing hiring, because we know that the
process of tenure at Harvard has its own quirks
and characteristics. And we really need
to be thinking about how it is we should
be hiring people who can be mentored in
such a way that they’re going to be promotable here. And so forth, and I can give you
similar things for the teaching component, which we had
many sub aspects of, and fit as well, if you’re
interested in hearing that. This is where I might have used
a slide, but I don’t have one. Having these kind
of subcategories focused our
discussion about what it was that was really
important to us as a committee and as the department,
such that I don’t think it’s
really possible when you get a number
of good candidates for a job to simply say
that this person is better than that person. And it almost never is
the case that that’s true in every regard. In fact, when we ranked our
six top candidates, all of whom we brought in for
interviews, there was no one of them that ranked
top in all of those categories, as I think is
predictable and it would be expected in almost any case. So another thing that I wanted
to mention in this respect, in terms of evaluation
of candidates, had to do with
sensitivity to the fact that when we have particularly
tenure track jobs, we’re going we have a
wide range of experience, in terms of the applicant pool. We have those who
are just finished, those who are a few years out,
those who are many years out. Oftentimes, people who are 10,
or 15, or even 20 years out who are applying for junior
positions for whatever reason. And it’s I think
quite a challenge to distinguish what makes a
strong candidate for a tenure track position here of somebody
who’s been out in the teaching world for 15 years, relative to
somebody who’s just finishing and maybe has one article,
or something, out. And so what we
did in this case– and I’m not sure
this is naturally the best practice– I’d
actually be interested hear. I shouldn’t say we. What I did is I separated out
these groups of applicants into these age grades, if you
will, and then thought ab– for myself, ranked them
individually, and then thought separately
about whether or not, what the strengths and
weaknesses were of somebody who had been in the
system for a long time, versus somebody who
was just finishing up. And finally in this
respect, in this– kind of concerning fair
evaluation– I’d mention the challenge of
international applicants, particularly for junior
and tenure track positions, for all the number of
reasons, both the– and this isn’t true with international
applicants from every context. But there is a
challenge of when we ask for letters
of recommendation, the nature of those letters
being written by scholars who address that type of request in
a very different way than what is expected of scholars
in North America. And also the
expectations that we have in terms of both teaching
experience and communication effectiveness, particularly
in the interview process where we’re
dealing with people and thinking about how
effective teachers they may be. And sometimes superficial
aspects of communication can outweigh other things. And finally, the
last set of points I wanted to make
actually relates a bit to what Iris was talking about
concerning the process by which we evaluate candidates. And ensuring– the
main point here is that I think that it’s
important to ensure that it’s a deliberative process. In fact, I was very
interested to hear about the fact of committee
meetings and panel interviews being a bad idea, which I think
is a really interesting take on a point for myself. I have, as an applicant
a long time ago, been sort of
simultaneously subjected to the American system and
then the British system, where you do get a panel
interview in academic context. And I wasn’t actually sure
which is more effective. And in many ways, I think
that– at least from kind of an outsider’s standpoint
as an applicant– I appreciated the fact that
I was asked a question once, and I could give
an answer, and then everybody kind of
heard the same thing. But on the other side of things,
the groupthink component of it, I think, is very
important to think about. And the last thing I’ll
say about this concerns those who we ask to take
part in our committees. And this relates to the point
about diversity on committees. And I think that actually
it’s quite important to ensure that those
who are involved in at least the deliberation
process about our applicant pools are not limited to a
small group of individuals from the same part a
department, with more less the same
perspective on what the job, on what a
good applicant is. And I think that
ways that we should go about ensuring
that doesn’t happen is by having at least one
member from an allied field who’s involved and
invested in our searches, even at the junior level. This always happens
at the senior level. But also involving graduate
students, in some fashion, in the evaluation process. And I know that
that opinion is not shared by everybody, at
least in my department. And it may not be shared
by everybody in this room. But in fact, in the search
that I was a chair for, the reflections by
the graduate students were extremely valuable. They were very effective. They made very
effective points that contributed to our overall
decision making process. And in fact, the order of
candidates that they as a group came up with ended up being
the order that was decided upon at the end of the process,
including the person who we ended up hiring. So those are kind of a number
of scattered thought of thoughts about this whole
set of questions, and I hope some
of that is useful. Thanks. [APPLAUSE] So Rowan’s comments
lead me to just mention that there are two kinds of
conversations we routinely have that you might want
to be mindful of. One is the kind of
conversation that happens at dinner with
the candidate, right? There are no rules here. You could talk about anything
you want to with the candidate. There is a preference
amongst us at Harvard to read The New Yorker the day
it arrives in our mailboxes. It doesn’t mean that all
candidates read The New Yorker, and that that is the most
important thing for us to be discussing with them. It is important to remember that
we’re not selecting a friend, that we’re selecting
a colleague who’s great for Harvard and
for the department. And those are explicitly
to be set aside, because left to
our own selves, we will veer towards conversations
that, you know, speak about the single malt scotch
that gets 98 or 99 out of 100, or The New Yorker. And if you do
that, just remember that there is no chance that it
can’t influence your decision. So we’ve thought for a
long time that we can then set those things aside. And what we’re discovering
is that the conscious mind is limited in that capability. And that it may not be
able to set that aside. So think a little bit before you
have your dinner conversations. What do we want to focus on? What kind of work do
we want to open up, broadly speaking,
with the candidate, so that we can hear
her or his comments on those kinds of things. And so don’t just
wander into them. The second conversation that we
have, I think, is more deadly. And that is, as soon as
you drop the candidate off at their hotel, or
whatever, the rest of the committee in the
car will usually say, so what did you think, right? And that’s just
something that we do. And I discovered
that there are often two junior people in the
back and two senior people in the front, end that the two
junior people are listening very intently to,
what did you think? The next day when you have
to vote, surprise, surprise, we all agree on
who we should hire. Isn’t that wonderful? So I kind of try to
stop my department from doing some of
this chit chatting about the candidate outside
of the for– And to my great surprise, in one of the
searches where we did this, we discovered that we
actually asked people to write down their
impressions of the candidate, take notes after every
meeting, et cetera, and then send them on to
the assistant to the chair. When those are printed out
and brought to the faculty, so you see for the first time
what your colleagues think about that candidate,
all of a sudden, there’s very little consensus. But at least– but
that’s the truth of what we are each thinking. So these are little
ways, again, for us to bypass the very biases
that we might have. The last speaker on this panel
is somebody I admire greatly, Avi Loeb, who is Frank Baird
junior professor of science in the Department of Astronomy. He’s the chair of
the department. He’s director of the Institute
for theory and computation within the Center for
Astrophysics, physics, and he studies little
things like the cosmic dawn, the first stars and the
galaxies that emerged, when those first stars
and black holes formed, and what effects they have
on our young universe. But what I love
about Avi is that he doesn’t shirk from thinking
hard about the diversity of his field in every way. He writes email and
pesters us about, think about this or that. He writes books
that make astronomy appealing to the general public. And I have a secret desire to
do a study with him, in which I want to bring people in to
think about the cosmic dawn and then show that, after having
done that thinking for even five minutes, that they will be
less prejudiced when they think about inter-ethnic issues. But that’s a study I’d
like to do with him. First, I would like to say a
few words about my background, because it’s not very typical. I grew up in a farm, and I
use to collect eggs as a kid. And often when I’m frustrated
in my current work, I think it might have been
more relaxing to go back to those days. Avi, I think it’s on. Is it on? Yeah. And since I started with very
unusual initial conditions, and I got to where
I am right now, it offers me
perspective about what is the right approach
to selecting people. And my personal history, which I
regard as quite an unusual path from interest in philosophy
at a young age to being a scientist right
now, and astronomer, teaches me independence in
the selection of my research topics, but also diversity in
the selection of colleagues. And that’s what I’d like to
discuss today, how to collect matches this will catch fire. So you can think
about a matchbox, and it’s very difficult to
tell in advance whether, if you pick a match and rub it
against the walls of the box, whether it would light up. And that’s a difficult decision
in any selection process. And so I would like
to advocate diversity for a different reason
than commonly argued for, and that is that it leads
to diversity of ideas and to innovation, because
different types of people with different backgrounds,
different genders, think differently about problems
and approach them differently in ways that could allow
breakthroughs and innovation. And once there is diversity
in the community of people thinking about a problem,
there is more likelihood that not all the
matches will be duds. We are all familiar
with the phenomenon of seeing young people, that,
let’s say 20 years ago, where regarded very highly, that
were thought of as geniuses in the field that they work in. And I will not mention names. I can think of very specific
examples in my field that 20 years later, really
did not produce much, and did not fulfill
the expectations, and became sort of
dead wood, in a way. And the question is, how
come other people that went to lesser institutions
succeeded so much in advancing the same field? And what was wrong with
the selection process? Now of course, there is
a random aspect to it, that it’s very
difficult to forecast the success of individuals. But there are also
some systematic trends. And often, people try
to reproduce themselves. They look at the mirror
and they like what they see, most of the time. And they, therefore,
like to see themselves replicated in other people. And there is a
sense of longevity of the line of research that
they promoted for many years, if they find a person that looks
like them doing the same thing. The problem is that
that blocks innovation. And there are many examples in
the history of astronomy were the entire astronomical
community fought about known truths that cannot be
refuted that must be right. For example, the sun must
be made of the same material as the Earth, because
the Earth after all was made of the debris
left over from the sun. So you would think
that the sun is made of the same stuff as the earth. That was very natural to assume. But Cecilia Payne-Gasposchkin,
when she did her Ph.D.– almost 90 years ago–
she realized that, by examining the spectrum
of the sun, that it’s made– at least the
surface of the sun– is made mostly of hydrogen. And
she wrote that in her thesis. And she was discouraged to
include that in the thesis by Henry Norris Russell, the
director of the Princeton University Observatory
who told her that we all know that the
composition of the Earth and sun are the same. And she was so frightened by
this single person telling her what the right thing is that
she took it out of her thesis. And a few years down the line,
he realized that she was right and admitted that. Eventually, she became
the department chair here at Harvard, of the
astronomy department. She had done the first Ph.D. in
astronomy at Harvard Radcliffe. But that’s just a
good example to show that the mainstream view is not
necessarily the correct one. And diversity helps us innovate. It also reduces the trends
that I mentioned before, about self-replication of
both research and policy. So if we have a diverse
group of colleagues, we’re less likely to repeat
the mistakes of the past, irrespective if whether
these mistakes are in terms of the
research we’re doing, the truths, the scientific ideas
that we believe in, or policy matters. So I think it’s very much to our
advantage to promote diversity for these reasons. The mistake that is
often made, and I’ve seen it quite a bit
among my colleagues, is that when they
see a young person, they immediately develop an
opinion about this person after hearing, let’s
say, a talk, or speaking with that person. And they maintain a static
image of that person. So if, at a very young age,
they thought the person is not very promising, when this
person grows and becomes much more prominent, they
still maintain their view. They have a static
view of that person. And that is convenient,
because you don’t need a lot of information. You just need to develop the
initial image and keep it. The problem with that is
that the initial image that you have of a person is
often shaped by circumstances. If that person went to school
in a place that is not, doesn’t have
distinguished scholars, then that person will not
get the same education, will not be brought up
to the same standards as the rest of the people. And what one should
pay attention to, instead, is to the
growth of the person. So looking at the
initial conditions, where this person came
from, and looking at the derivative of how fast
that person is progressing and developing. That’s much more important,
but it’s more time consuming, because it requires
you to monitor how a person develops with time. You can’t just assign labels
that are static to a person. And so, one lesson
that I’ve learned is that even if you
had a modest impression of a student or a
post doc, you need to keep monitoring
what they’re doing. And later on in
their career, when they are applying to
faculty positions, it’s your duty to update
your image accordingly. Many people prefer to
maintain the image, even when presented
with new facts, because they feel embarrassed
admitting that they were wrong. That’s a very common
tendency of people serving on committees, even
grant allocation committees. They would prefer to give
the funds to those people that they believed in
early on in their career, and block those people
that they thought are not very promising, even if
the evidence shows otherwise, just so that they
would not have to admit that those other
people are successful. So it’s very important
to allow images of people to evolve with time and
reward growth, rather than academic ancestry. And by ancestry, I mean
initial conditions. Who this person worked
with for their Ph.D. We pay a lot of attention
to the letter writers, and often, Ph.D. Advisors
that are well recognized get much more attention in
terms of recommendations. More on the day-to-day side. My experiences as
department chair and as director of the Institute
for Theory and Computation is that it’s very important to
stay practical, to try and find the practical solutions,
rather than being confrontational with people that
don’t share your set of ideas, because by again getting into
fights over particular issues, by getting confrontational,
a lot of energy gets wasted. And very often, those people
are blocking other moves towards the right direction. And so, what I found
to be a strategy that’s works very well is trying
to bring other people, to explain to them why it makes
sense to promote diversity. And to my surprise, I found
the honest, straightforward, non-political approach
to be very effective. And I haven’t encountered any
resistance from faculty members in my department
when I adopted this. It may well be that
they are afraid, they don’t want to express
their voice loudly. But I don’t care what the
reason is in practice. They did cooperate, and I didn’t
encounter much resistance. And the results are that
among our students– and I should thank
also Dave Charbonneu, he’s sitting here from
my department, who served on many committees
that helped promote that– among our
graduate students, 46% are women right now. And from the incoming class
this year, 30%, about 1/3, are minorities. Underrepresented minorities. Underrepresented minorities. At the Institute for Theory
and Computation, about 40% of the post-docs are women. And five out of six of my
graduate students are women. I didn’t do anything
special to encourage that, it just happened. So let me move from
this slide, which is more general,
more philosophical, to some specific rules
which I think are helpful in promoting diversity. And these are my personal
rules, guiding principles. The most important
guiding principle is that one should maintain
a high quality, high level, in the hires. And there are many
reasons for that. Obviously, we all want Harvard
to be the most prestigious institution, most successful. But it’s also important to
realize that if you compromise, that leads to
reinforcement of prejudice. So if one tries to just
improve the statistics, without maintaining a high
level, a high threshold, in terms of the quality of the
candidates that are recruited, then it goes against
the goal, because it says that on the books, perhaps,
the numbers are promising. The statistics look good. But obviously, that candidate
will encounter difficulties later on. So that’s not a good idea. And the other reason to
maintain high quality threshold is to generate role
models for other searches. So obviously, there are
faculty members that are from a diverse background. That helps in inspiring young
people to enter that field and follow in their footsteps. The other thing that– the
other mistake that is often made is to think that
organically, things will get organized in the right way. And that doesn’t seem
to be what happens. Leadership does matter. And in that sense,
it’s very important to select the search
committee to include members that do appreciate
the value of diversity. It’s important to, as
was discussed before me, to encourage promising
women and minority candidates to apply, obviously. And to search for target
of opportunity hires. Very often, once you
establish a strong reputation for the department or for the
group that you’re working with, there are potential hires that
are lurking out there that might be interested in coming. And most important,
it is to create a constructive atmosphere
within the department that is supportive of this process. Of course, the
candidates that apply do so based on the
impression they get from the
community at large as to how comfortable it is
to be within the department at Harvard. And as we heard
before from Iris, some candidates don’t apply
because they feel that they’re not good enough, or
perhaps their environment is not nurturing enough. And so in that sense,
it’s very important to follow up on hires by
nurturing academic growth, addressing problems when
they are still small, and nominating, for example,
junior faculty, or post-docs, or students to prizes
and fellowships. It’s very important to show
that we care about the less senior people around us. And that includes also attention
to personal needs, which are particularly important. Every individual has
her or his needs, and one should pay attention
to their special needs. That includes creating a family
friendly environment that supports families, and
avoiding overburdening women colleagues or minorities
with too many committees. And another important
element of all of this is to create a broader impact
than just here at Harvard. And in the astronomy
department in particular, I’m very proud,
again to acknowledge work done by John Johnson
and Dave Charbonneau. For example, we do have a
summer internship program this is promoting diversity. There is a post
doc program, thanks to Judy that was established
to promote diversity. There are mentorship
opportunities for either post docs or
faculty to promote diversity. And one can organize workshops. And all of this is
important because it creates an image of Harvard
as a place that accepts people from different backgrounds. A lot of the problems
have to do with the fact that people don’t
expect themselves to be accepted to Harvard,
simply because Harvard has an image which is very
selective and very specific. And finally, the ultimate
goal of everything I do is to increase diversity beyond
a certain threshold, such that the environment
that I’m embedded in makes decisions in a stable,
self-sustained manner. So wouldn’t matter who
the department chair is, and who is pushing in
one direction or another. There would be a large enough
number of people, colleagues, that care about it, such
that the process will be self-sustained. And I think, that’s the
healthy, stable, state that we all aspire to be in
so that the issues that we bring up and discuss today
will not have to be discussed in a special forum. It will become self obvious
that this is the reality. That’s the way things are. And that’s my dream
for the future. [APPLAUSE] Thank you very much, Avi. So I think the
rest of our time is to be spent in discussion
with the panel. But also, Nina Zipser is here
from Mike Schmidt’s office, the FAS, and Judy’s,
here and I’m here. So we can consider
all six of us as able to answer different questions
that you might have for us. Please mention any
struggle that you confront. We have this possible
situation, what might we do? Or other thoughts that
you have about what is not making sense about
what it is that we’re saying. Any and all of these, we’re
open to receiving from you. So let’s just get started. I see a hand slowly rising. Yes, please. I just wanted to know
whether one of you could say a little
more about the problems with panel interviews. I [INAUDIBLE] a major revision
of the procedure in philosophy to get rid of those, but. Is that what you do
right now in philosophy? That’s what you say? Well, if I’m understanding
what a panel interview is, that is to say several
members of the faculty– Yes Interview the candidate, that’s
one of the things that we do, yes. I see, OK. I wasn’t aware that, but OK. Iris can say something. I’m happy to answer
that question. So the evidence
suggests that if you have several people interviewing
one candidate at the same time, it’s very difficult for
the three or five panelists to form an independent opinion. So that’s the crux of groupthink
is that sometimes groups are worse than if I collected
people’s ratings individually and then just averaged the five
different ratings, for example. So that’s the reason why
group interviews generally are discouraged. I might add one more thing. If you want to pick
a baseball player, would you throw a
ball to him once, or would you throw
it 10 times to get a sampling of how good a
baseball player that person is, a batter? I think panels don’t
allow multiple shots, so that you can. So even though I
think that there is one very positive thing
about panel interviews, and that is that we’ve
all heard the same answer. And now, if we disagree about
the quality of the answer, we can actually tussle about it. We can say, I think
what she meant was x, and you think what
she meant was y. So there is a small benefit. But I think the disadvantages
outweigh the merits. Yes. And then. I was just wondering, in each
of your faculty meetings, how many of you have
been in a meeting where there was an explicit
discussion about implicit bias? OK. Because for us, that was
revolutionary, because before then, whenever we
talked about diversity, I think our faculty
naturally viewed it as, we’re going to
compromise on quality for a socially desirable good. And what the research, and
what that powerful stuff about implicit bias does, it
just breaks that narrative and makes us realize,
oh, we’re not hiring the very best people. So if there’s someone who’s
exci– if you excited, I would encourage you to
get that discussion going in your faculty. It gives you a
completely different way to talk about this goal. Can I just add to
that that it just makes me cringe when any
question about quality ever emerges when I show the
research I do, because that is so far from what we’re
actually trying to say, that all I’ll say is
it makes me cringe. The reason I want to
just add one more thing is, you don’t have to know
everything about implicit bias, but you have to know one
overwhelming fact, that there are now, not 50, not 500,
but 5,000 studies that in one way or another take
exactly the same resume, the same product
of work, and just change the name to vary
in ethnicity, sexuality, nationality, gender,
dominant language, whatever, social
class, all of these. And what we know, over
and over, and over again, is that the same product,
the same the work, is not viewed the same
way because of that name. That, to me, is the single
thing that we need to know, because we think that we are
indeed doing what our job is. And that is, to
attend to competence. To attend to the
merit of the work. And what this
research is teaching us is that we’re
not able to do that. We need to find other ways
to be able to find out why we’re not able to do it. And it all boils
down, it turns out, to the decision
about competence. So the most relevant
study to us might be a study done by
a biologist at Yale who published in PNAS,
maybe a couple of years ago, in which she actually
selected people like us, people who are in the
life sciences, who run research labs. And she had people be selected
for the job of lab manager. As you all know, lab manager
positions are very important. It places the
person in that role to getting to a much
better graduate program than they might have if they’d
gone straight from college. And what she finds
in that study, amongst people
like ourselves, is that for exactly the same
resume, we, both men and women, over select men
compared to women. We do it because we
actually see competence in the male candidate
over the female one. And again, I don’t
mean to harp on gender. It’s just the one
that’s been studied. But this could be something
like looking at two people, one of whom has a
military background, and the other who doesn’t. And we may have some
stereotypes about what people from the military
might do to our environment. Those are the kinds
of things that we can’t allow getting in the way.
$4,000 more in starting salary given to the same equally
qualified male over female. And to me, this is
even more stunning, reporting on spending
three times as many hours mentoring the candidate if that
person is male over female. So once we know this,
then the question becomes exactly the one
that David is raising. And it’s all about, how do we
get to the most accurate sense of merit? Yes. I had one follow up question
and then one separate one. The follow up was with
the panel interviews. Does the research suggest that
the panel interview itself is problematic, or just
that, in comparison with five other individual
interviews, it’s worse? If the panel
interview’s accompanied with the five other
individual interviews, is that problematic? What is the extent of the actual
empirical research on this? It’s a very good question. I don’t have any answers. I only have an
answer to your first. So the typical
study has been done with five different
separate interviews compared to the five
people on a panel. I don’t know about
sequencing, whether sequencing would be a good thing. So for example– I
mean, this is just a hunch– I could imagine
that having the five separate interviews first,
which often actually happens. And then for the five panelists
in some way coming together and maybe reinterviewing. I don’t know whether that’s
going to add value or not. I really know, I don’t
know any evidence. Yeah. The problem– when you ask, is a
panel interview decent but just not as good as the
individual, the problem is that interviews suck. That is the problem. So almost any– so I don’t
know about doubling up on them to kind of see if
we can do more. In a sense, the talk
is the panel interview. There is a person
presenting the heart of what we want to learn
about, and we all listen to exactly
the same thing. We don’t come away
with the same things, and we don’t all listen
equally, and so on. But at least there
is that moment, where the most important
piece, the work, is being viewed by
all of us jointly. So I still, based
on what I’ve read– and I don’t believe
that we have the time to do both the
individual interviews and then gather again as a
group to do a panel interview. So, so far, individual
interviews– and if you can get your
faculty– I mean, look, it’s like herding
squirrels, not just cats. But to get faculty to do
anything systematically is not going to be
plausible, really. To say, could you ask
the following questions? Could you stick to the work? One of my graduate students
just had this happen. She is a concert level pianist. She went off to graduate school,
got a Ph.D. Unfortunately, she had had a little line on her
CV that talked about her music competence, and
everywhere that she went, faculty asked her, how do I get
my kid into Juilliard like you did. Explain to me what happens. And so the entire hour was
spent talking about how. And at the end of
the interview, people felt they didn’t know anything
about her work, which was true. But that was the selection
of the questions you ask. Hugely important. Just to add to
that, that’s why you have to have a list of the
five questions you want to ask. Or 10 questions, how many
questions you want to ask. And stick to those. I mean, I couldn’t just agree
more with what Mahzarin said. It’s so easy to
go out on a hunch and kind of talk about
wherever they went on vacation. The other thing I
thought was interesting is Google actually experimented
with the optimal number of interviewers. That’s another thing
that we’re encouraging. Companies or even
departments actually measure what you’re
doing yourself and kind of see what works. Now, many of our samples
aren’t big enough, but Google had the advantage
of having a very big sample. It turns out for them,
four different interviewers is the optimal number. There’s nothing magic
in the number four, it might be different for
different organizations. But what is magic is
that you can use data, and you can look at which
point are the scores starting to converge. And then you can save the
company lots of money, and you can increase your
accuracy at the same time. I had one other
independent question, which was one of the
points that came up was that it’s best to make
searches as broad as possible, rather than sort of specialized
narrow specification of the candidate. In my own department,
epidemiology, every search, as far back as I
can remember, has been targeted to
a particular sub discipline within
epidemiology, often to meet what are perceived as
the needs of the department. Do you have any further
comments on trying to balance the desire
for a broad search to obtain the best possible
faculty member across all the areas of a
discipline with meeting, say, the departmental
teaching needs, or the need for having diverse
representation across the sub disciplines within a department? I should say that
in our department, we have a tradition of
having open searches, not even defining the
area, not to speak about defining a narrow area. And it has worked extremely
well, because, for example, I was hired in a search
that was conceived as a search for an observer. And I’m a theorist. And vice versa,
we have observers that were hired when
the department really need a theorist. The advantage of having it
more open is that– well, first of all, obviously you are
open to a more diverse pool of candidates. But also, the level of
excellence that you can recruit is far better, because
it’s not at all guaranteed that in a given year,
the applicants will be of the top quality in
the particular narrow area. So we’re not just talking about
the number of people working in that area, but also in this
particular area, whether there is an exceptional candidate. And so I think it
serves a department well to be as open as
possible to candidates in a wide range of fields,
such that the quality threshold would be very high. Rowan, do you have
something to say about that? I couldn’t agree more
with what Avi just said, but this is based on my
generalized perception of the way that broad
fields are improved. I think that if you– I mean, if
the real need in a narrow field eventually trumps everything
else, most likely you’re going to get the best
applicants in that field anyway, if the broader
definition incorporates that that narrow one. But you will also
have the opportunity to look at creative applicants
or unusual applicants that you never would
have seen, had it not been for a broader deposition. So our view of great
men and great women is so strong that when
one of them leaves, we’re willing to give up
searches in our own area to fill that position
with somebody like them. The more admired they are, the
more that is likely to happen. And I think Rowan gave
a great example of that. So I do like the
idea of saying, this is the kind of person
we think we need. Now, let’s go to one
level above that, to a slightly more
prototypic level, rather than the subordinate level of
that field, and to specify. Yes, it’s a little
more work, because you will get more applications. But the biggest
reason to do it is because our minds
about our own fields changes slower
than the actuality of those fields changing. And so what you get is
an infusion of something you never even imagined, or
that you did not know about. And so I like
going into a search not being prepared for
what I’m going to see, and leaving explicitly
some open space for being able to shift gears. And this is where
the chair of research can play a very important role. And there’s no reason to
say, we’re looking for x, or we must hire x, if y
ends up being somebody who was non
canonical, but ends up being perhaps the kind of person
who could change the future. And because we’re a school that
has much smaller departments than many other
schools do, for us every one of those people being
able to do more than one thing tends to be appreciated. Can I add one other thing? I think that there’s been an
interesting procedural change here, as I understand
it, in the last year, such that now there’s
an extra step in terms of the approval of
search committees. And as I understand it,
then job descriptions are being asked for before a
committee is actually formed, which means that this
conversation, if that’s true– if I’m understanding this
accurately– then that means that this conversation about
the nature of a job request needs to happen before the
job definition takes place, but rather when a request
to the deans is being made. So I think that we should
all take that to heart, that when we’re in the
academic plans, and so forth, and when we’re requesting job
searches for the [? coming ?] academic year, I think we should
be thinking about how to define what it is that we’re requesting
as broadly as possible, to incorporate the
perceived needs and yet permit this kind
of broader definition to be used in the job advert. Yes? Just yesterday, we had
a spirited discussion between the heads of two
different search committees, one in molecular and
cellular biology, and one in FAS systems
biology, over the utility of using Skype panel interviews. So the panel part
we’ve talked about. But the Skype aspect
is, you get to a point where you have 25 people
on your long list, and you’re trying to
get down to six or eight people on your interview list. And whether or not to use a
brief Skype interview, about 30 minutes, with
three people to try to narrow that list down and get
a little bit more information. And the two points of
view were basically, on the one side
when we bring people in for a two-day
interview, people feel that they often know
within the first hour whether this person is
going to be a good match. And so why waste the two days? And the other point of
view, which is probably obvious in this room, is
that a short Skype interview seems to be rife for problematic
judgments about somebody’s photogenicness more than
the substance of their work. And maybe just from the
way I said that, you’ll know what side I was on. But I’m interested
in people’s views on this, because we
do have a problem. MCB, for example, is
one of these areas where we get 200 or more
applications, and we have to winnow those down to a
manageable number to interview. And whether or not to use
these brief Skype-like things to winnow it down. There must be data
from phone interviews, maybe not from Skype interviews. And I’d like to know if
that data’s out there. So yes, there is some data. And one of the books that
I recommend you read, which I think is the best book
that has ever been written in human resource management, is
written by not a human resource manager, by Mr.
Bok, who is chairing HR department in Google. And again, they’ve experimented,
measured everything. And sadly, it comes down to
what Mahzarin has said before, but generally the added
value of any interview is very, very small. So for them, whether it
was Skype or in person, actually didn’t matter. Whether it’s 10 minutes or
half an hour, it didn’t master. Structured interviews
added validity. But I think the really
important message is we’re just generally
bad in judging people. And we should replace the
value that we attribute to whether a Skype interview
or personal interview with as objective
measurements of performance as we possibly can. Now, I’d like to add
just a footnote here. And I’m sure you have been
in those situations too. Sometimes interviews don’t
even serve the purpose to predict future performance,
but actually to sell Harvard. And I think that’s a very
different perspective. That sometimes you can be also
part of the discussion that I often have. So I have my
structured interview, and the last 10
minutes are for me to talk about the
Kennedy School. And I think that’s
an opportunity for us to kind of also sell our place. So interviews do have a place,
maybe in particular I think, in that realm. So interviews do give
us unique information that is not in the CV,
no question about that. They even give us very
useful information. But they also give
us a lot of crud. The question is how to– and
that’s why, overall, they are less good than only
making decisions from CVs– as many meta analyses
have now shown– because it’s that other stuff. So if we can figure out
better ways to interview. So to answer your
question about Skype, I would say that human beings
will use any shred of evidence that they can find
to make a decision. So in the old days, we used to
have people attach photographs with their applications. But then things would happen,
like at Yale in the ’40s, where somebody said, reason for
rejecting the candidate, he has a Mediterranean nose. That was sufficient to
indicate to a colleague that this application should
go into a different pile than the good pile, or whatever. We got rid of
photographs, and now of course because of Facebook,
and Skype, and all of that, they’re back. And if we had time, I could
show you, independent of gender, race, anything, it
turns out that we have fundamentally
wrong views about what certain facial features
tell us about the psychology of the person, about
their competence. It turns out if somebody has
eyes that are a little closer to each other than
average, you will think they’re dumber than they
actually are, or certainly compared to the other person
where the eyes are a little bit wide-set. So I’d be very skeptical of
facial data, which is something that makes interviews go wrong. And so what’s the solution? In my mind, we can’t
get rid of them. So my solution is a little bit
of harder work on our part. And that is, read,
read, read the work before you bring
the candidate in. We’re having discussion
in psychology. Many of my colleagues
would like us to bring a whole series
of people through and our brown bags, and
just think about their work. And I’m arguing to Max
who chairs that committee, no, let’s just read the work. And when we know how we
feel about the work, let’s, for the first time,
see the person. And then maybe our
conscious minds will be able to actually
do some good work. To say no, OK, that
person is obese, but that doesn’t mean that
we don’t want them here. That’s the kind of work
we can do, if left to us. A few more questions, yes? Other thoughts? Yes? So this I think
picks up a little bit on a previous
question, and trying to understand any of
the particular risk that arrives when you’re conducting
open [INAUDIBLE] searches. Because from one
perspective, that’s a great way of maximizing
the potential diversity of the candidate pool. But it introduces some
obvious challenges, especially if you think that, say, gender
diversity or racial diversity in the candidate pool
has improved over time. And so you might have a
relatively racially diverse tenure track pool,
but the tenured peopled you’re comparing
them to are not so diverse. But they’re also, by
definition, more accomplished. And so how do you work
through and mitigate some of the biases that might
influence the way you approach something like that. Nina and Judy, can you guys
say something about how [INAUDIBLE]. So we tracked often,
as you know, people ask us to authorize open searches. And we budget them as senior,
because they almost always come back as senior searches. So we just have become
careful to do that so that our budget doesn’t
kind of go out of whack. So my suggestion is actually
to have them run a tenure track search, and then really
canvas the field. And if that’s not
working for a while, or if there is someone that–
if then in canvassing the field they find there is a diverse
pool of senior candidates and they’re very strong,
then you might consider it. But I find open– like we just
chuckle when we put open down, and we say, OK, we’re
authorizing it in open search because we know it’s
going to come back senior. So that’s really the answer,
according to the data. But look, we are
supposedly now a university that will be
tenuring from within. If that is the
case, then I think it’s very important for us not
to think a default search can ever be a senior search,
but that instead our job is to conduct a junior search. Now, this obviously doesn’t
mean that if there’s some demigoddess out there, that
we shouldn’t go and get her, even though she’s senior. That’s not what we’re saying. But I think that
if we really want to make the kinds
of changes we do, and to really have the
greatest opportunity to promote from within, that
I just don’t see us making a case for senior
searches very easily. Other questions? Other thoughts? Judy, anything you’d
like to say about? You want to close? Or you want to– I’ll close. OK. Why don’t I just close this
out by thanking our panelists, thanking Mahzarin,
and most importantly, actually thanking all
of you for being here. I think that your
presence here signals a lot about the potential
future of Harvard. Some of the questions
that are coming up about panel interviews, about
Skype interviews, about reading the work. That’s actually one of
my favorite questions to ask at an ad hoc tenure
review, have you read the work? And I’m amazed at
the number of people who actually admit that
they have not read the work. So I think that
that kind of message is an important one to get out. The other is that this
is not the last time. We have lots of resources
that can help you. If you’re in the faculty
of Arts and Sciences, Nina Zipser, who’s sitting
here in the gray jacket, and Mahzarin Banaji are more
than happy to talk with you. If you’re in the faculty
of Arts and Sciences, or anywhere else at the
University, I and my colleague Elizabeth Ancarana, in
the back of the room there, are more than
happy to talk to you. We held this event
early in September, even though we know this
is an incredibly busy time, because we’d rather talk to
you before you get underway, than– we’re happy to talk
to you along the way as well. So please feel free to shoot
an email or make a phone call, but we’re happy
to brainstorm with you. What you’re hearing
here is, in some areas there’s a lot of
research we can point to. And in some cases, it’s
a lot of craft knowledge. And the craft knowledge
can often be as important as the research, in
terms of experiences in difficult situations. So I hope you all feel free
to reach out to any of us, I daresay our panelists. But I think talking
to colleagues who are in the same positions as
you can be very valuable, which is why we asked Iris, Rowan, and
Avi to be the panelists here. So we’re running a
little bit over, so let me just say thank you, all. And I hope, honestly, to hear
from each and every one of you, because I think that would be
the best outcome of this event. So I wish you all best of
luck on your committees, and with the rest of
the academic year. Thanks a lot.

Leave a Reply

Your email address will not be published. Required fields are marked *