Blind audition study: Truth or myth? | FACTUAL FEMINIST


The blind audition study is one of the
most celebrated social science papers of all time. It’s famous for showing that
when orchestras auditioned musicians blindly behind a screen, women’s success
rates soared. But it turns out, the study showed no such thing. What’s going on? That’s coming up
next on the Factual Feminst. During the 1970s and ’80s, the nation’s orchestras became more open and
democratic, and to ensure impartiality, several introduced blind auditions. Two
economists, Claudia Goldin of Harvard and Cecilia Rouse of Princeton, noticed that
women’s success rates increased along with the adoption of screens. Was that a
coincidence, or the result of the screens? That’s the question they tried to answer
in their 2000 paper “Orchestrating Impartiality: The Impact of Blind
Auditions on Female Musicians.” They collected four decades of audition
records and rosters from eight major orchestras and crunched the numbers. Now,
their paper includes multiple warnings about small sample sizes and
contradictory results — but few readers seem to have noticed. What caught
everyone’s attention were some strong claims in the final paragraph: “We find
the screen increases by 50% the probability that a woman will be
advanced from certain preliminary rounds.” According to Google, the study has
received 1,500 citations in academic articles and thousands of media mentions,
and showcased in so many diversity workshops that one recent attendee
begged never to hear about it again. Now, the study’s appeal is clear: two
prominent economists, in a top journal, wielding state-of-the-art econometrics,
captured and quantified gender bias — and they also documented a solution. Or so it
seemed. The research went uncriticized for nearly two decades. That changed recently
when a few scholars and data scientists went back and read the whole study. They found
a tangle of small, ambiguous, contradictory findings. For example, the
screen seemed to help women in preliminary audition rounds, but in the
semi-final rounds, they didn’t. And none of the findings were strong
enough to draw a broad conclusions one way or another. And the authors say as
much, albeit ambiguously throughout the paper. So where did Goldin and Rouse get their totemic conclusion that blind auditions
dramatically improved the success of women candidates?
Well, after warning that their findings were not statistically significant,
they simply declared them to be economically significant. But what does
that mean in this context? “That doesn’t mean anything at all,” says Columbia
University data scientist Andrew Gelman. In a recent commentary on the study, he
said that they’re just “fine words” that really mean that our data is “too noisy
to form any strong conclusions.” Now my guess is that the authors thought they
detected something with real-world relevance somewhere in all that
noise — but that’s a reason to call for more research, not to declare the
transformative power of screens in women’s quest for equality. Still some
may think it seems obvious that the screens contributed to equal hiring, but
it’s not. The screens may have been a reflection of changing attitudes — and it
was those attitudes, not the screens, that helped women. After all, women didn’t need
blind auditions to move ahead in law or business, medicine, or even in the
Cleveland Orchestra, which had not resorted to blind auditions, according to
the study. Now, Gelman and other critics — they don’t
deny the reality of gender bias and they don’t question the potential merits of
blind auditions as a means of achieving impartiality. But
Goldin and Rouse verified nothing about the special benefits for women, and nor
has anyone else. The subsequent research on blind
recruitment is just a morass of baseless claims or retracted statements,
contradictory findings… So how did such an equivocal study achieve iconic status?
Well, a lot of the credit goes to confused but influential fans, such as
the writer Malcom Gladwell and the Harvard psychologist Mahzarin Banaji.
They saw the study as powerful, indisputable proof of the ubiquity of
gender bias, as well as a way to counter it. According to Gladwell, “Orchestras in
the 1980s started putting up screens in audition rooms and immediately —
immediately — orchestras started hiring women left and
right.” And here’s professor Banaji in a 2017 TEDx talk: “In the late 1970s,
American orchestras were almost entirely all male . . . Once the curtain dropped, the
case study shows that the number of women who were selected doubled — they
went up 50%.” But it didn’t show that. Truth matters. Overhyped claims create
confusion and undermine public trust, and they don’t solve problems. Sex
discrimination in the workplace is a serious matter,
but improvements require solid data, replicable research, and careful
evaluations of causation. As the scholar Alice Dreger says, “Carpe datum . . . Evidence
is an ethical issue.” If you found value in this video, please show your support by subscribing to the series, and follow me on Twitter, and listen to my podcast,
“The Femsplainers.” Thank you for watching the Factual Feminist. Wave goodbye, Izzie.

100 thoughts on “Blind audition study: Truth or myth? | FACTUAL FEMINIST”

  1. Given that this idiot has lied numerous times and spews garbage from a think tank to peddle one ideology, I'll take actual science than an activist. whenever men have it rough in some area it's a social cancer from society on men yet when women do then it's just "just how society should naturally run"

    Lol garbage

  2. Linear regression modeling in the social sciences is a fraud. The social sciences are not a quantitative science and the value of statistics is minimal. Social 'scientists' also misuse it so often that we risk discrediting the methodology by continuing to allow them to use it.

    I was happy to see a Factual Feminist video today after such a long time.

  3. I'd like to see a study on the validity of academia.
    We have fake news,
    We have fake academics,
    with no accountability.

  4. “Doubled” or “went up 50%” ? Probably both in the advanced maths book for SJWs. Or make it up as you go.

  5. Many of these garbage, authoritative, agenda driven, regressive social science studies fall apart under basic scrutiny and deeper examinations. These types of studies, books, "scientists", and such alike, were only accepted to further a false societel narrative. Basic human nature in totality usually contradicts them. Yet, the sad part is this stuff is inculcated throughout society to our dismay. We've had three new studies just this year putting to bed lies ranging from the "gay gene" to transitioning sexually being a good idea in the long run, to even incarceration rates of minorities for whatever reason along with the decriminalization agenda that's behind the lie. Let's not even forget about the ubiquitous, and quite annoying, climate change alarmist lies that only are for government subjugation.
    This stuff only goes so far because we continue to let it. The public is largely too stupid, distracted, and/or cowardly to push back against this crap.

  6. Orchestral musicians tend to be a nervous bunch to begin with. I wouldn't be surprised if blind auditions help BOTH genders to perform better, because they can pretend no one is watching them. (This is an issue for people who aren't soloists, but are fine playing in front of others if they are part of a group. I'm not a pro, but I did try out for regional honor band and state honor band in high school, and I know that the screen in between us made the audition a lot easier for me. I talked to some of the other clarinetists, and they felt the same.) Long story short: you might get better quality musicians (regardless of gender) using the screens.

  7. Why should anyone trust someone paid by the AEI, one of the most neoconservative propagandist institutions out there? Ms. Sommers typically misleading analysis of this study (modest in its conclusions by its own admission) is just an excuse to point at one small-scoped faulty study and blame the entirety of the progressive analysis. In other words, as her well-funded employers desire, to keep the status quo going. Women have it so good today right? Forget about the attacks on reproductive rights, blatant office sexism and harassment, sex crimes, persistent exclusion from managerial positions, shrinking healthcare rights, infant mortality rates, proletarization of single mothers… Saying it’s better than the 1950s is a smug insult to the suffering of tens of millions of women, and conveniently omits the fact that what progress has been made results from the very same feminists and activists this channel reviles. The ideology of the AEI has given us the ultimate monstrosity in Trump, let’s keep that firmly in mind.
    Ms. Sommers has proven time and time again she refuses to engage meaningfully with serious leftist critique beside nitpicking cherry-picked studies. If she were intellectually serious, perhaps she would read or analyze a little Noam Chomsky? Or Naomi Klein, Glenn Greenwald, Keeanga-Yamahtta Taylor? Instead of regurgitating Steven Pinker’s fabricated statistics, perhaps she would learn about Jason Hickel? For someone who purports to be a scholar, I find Ms. Sommers’ knowledge bafflingly limited, and far from factual.

  8. On a secondary point, she should look into the current Me Too scandals sweeping through the classical music and opera world, instead of picking over the corpse of an outdated study. One may find the current situation invalidates her entire narrative.

  9. This was also done recently by the West Australian govt. for hiring public servants. When they discovered that there was a decrease in females being hired they just dropped it and never mentioned it again

  10. Screens sound like a good way to also block bias against older, white and black musicians. Asians might be assumed more skilled.

  11. The screens may relax the person playing, and help concentration. However, this would give a false impression of the musicians ability to preform in front of people passing judgement.

  12. The lesson I learned is that musicians are disposable commodities whose talents are meant to be used to compete with each other. Gender doesn't matter in the pay to play Art World. Money does.

  13. Why is this SJW horseshit popping up in my feed. Fuck you Google and fuck the whole women's movement bullshit. Get a job and shut up like everybody else.

  14. They tried hiring people in Australia using "blind" methods. This was apparently going to fix all the "sexism". It backfired! More men than before were being hired, so of course they had to put an end to it because,…vagina.

  15. There's another factor that may not have been taken in. Violinists and other string players tend to be female in lesser orchestras so there might be a difference between permanent hires and freelancers. Someone is always needed to fill pregnancy openings. The rise of the Pill in the 1970's-1990's may have actually helped younger women who were no longer assumed to leave shortly after being hired and getting pregnant.

  16. ''diversity and inclusion'' means

    white men are bad

    white women must rule everything

    let's virtue signal towards people of color while we mostly benefit white women

  17. Factual feminism is an oxymoron. Like equality and feminism. All that is left is misandry nowadays. Do not call yourself a feminist, the term is now associated with pure misandry. You know that…

  18. I love blind application processes because every single one has shown that sex discrimination in the workforce simply doesn't exist, and the feminist led organisations that implement them are very quick to unimplement them again as a result.

  19. All this proved was that the women performed better when they felt no one was watching. I fine feeling for someone you want in front of an audience 😉

  20. Counter point: If "screens" make it "less sexist" how the heck do these authors assume that women can't be biased against by men… other than visually? Maybe men can pick up the signs of a "female" violin player the way practically any man can notice a female writer's prose in fiction or journalism. Maybe the female musician plays notes, play slides, stops and starts at different times than a man does. Maybe the SOUND of high heels reveals her evil genitalia! Maybe she literally brings in her pheromones and she's discriminated against by SCENT–even the smell of her clothes betray her! It's a stretch, yeah. Except that's the thing scientists are supposed to examine and rule out. You can't say "men discriminate against women BUT ONLY FROM VISUAL CLUES" and use that assumption to prove the visual clue. The fact is, if men DO discriminate against women, they're NOT just discriminating against them "visually"–they're discriminating using any possible senses (because they hate women, remember? They don't hate VISION they hate WOMEN).

  21. 3:00 Speaking of finding something promising in noisy data… This reminded me of the satirical article "The Noise Miners" https://extranewsfeed.com/the-noise-miners-cffe6c14b626

  22. The Australian Public Service (governmental bureaucracy) proposed the same thing and even put it to the test with coded CVs rather than named. They proposed that both groups of females and of POC would benefit from blind submissions. Well, it turned out that the program DISADVANTAGED the proposed oppressed groups and it was dropped like a hot rock. Yes, it basically proved that the proficiency and performance of the PC favoured groups had been lacking and the employers were biasing in their favour already. So, it proved that the only oppressed group in the study was …you guessed it: the white male.

  23. I am so sick of hearing "womans quest for equality"
    Its actually time for MENS quest to catch up….its men that now need equality.
    Just sayin 😈

  24. The Australian public service trialed blind screening for job applications, so no age, sex, or race. Male employment went up so it was scrapped.

  25. Anonymised recruitment was tried by the civil service in the UK and Australia where the results ended up with even more men being recruited than through the “Biased panels” with the result of anonymised recruitment being dropped almost immediately as it did not meet expectations of certain parties.

  26. I would wager that in 2019, the blind screen auditions might result in a reduction of female orchestra members. Wouldn't that be a hilarious result?

  27. Doubling, tripling , quadrupling? Sounds impressive, but if 1 in 50 moves to 2 in 50 it's not very impressive. Orchestras have an easily definable number of members (small population size) where real numbers of males and females over time can be easily counted. This study is using using statistical mumbo jumbo to obscure an easily tallied fact. If auditions are "blind" then presumably the best musician for the job is being hired, end of story – If women are still not getting fair representation on the orchestra then perhaps applicants need to practice a bit more.

  28. All part of the replication crisis in the soft sciences, where confirmation bias was evident in the selection of inquiry, the standard of evidence, the interpretation of statistics and the willingness to publish with no criticism.
    Because the results leaned to a narrative they wished to propagate.
    And then they wonder why laypeople discount their science.
    They should discount their science if they are intellectually honest and do not wish to perpetuate a fraud.

  29. The trick is to see that the limitation in the article is as to the PRELIMINARY rounds. The goal is to get the very best of the best. Apparently these higher levels of competitive power are not even confronted by the study itself. It's like saying that we can help women compete in the Men's 100 yard dash by forcing the inclusion of women at the first cut. It's not the first cut which will really determine much, or even anything. It's the last cut, and the next to last cut, which really separates the best from the almost best. If the article won't confront this, it really has only confronted the middling nowhere.

  30. As a neuroscientist, I am often shocked how low a standard of evidence is often accepted in social sciences. I realize this comment is going to offend a lot of people, but it's true.

  31. Maybe the screen makes the person auditioning feel more at ease and women benefited more from that.
    Basically, how can you know that the effect is due to the listeners and not the performers? This seems like they are confirming their biases.

    Are orchestras going to be able to play concerts behind a screen? I think playing in front of an audience might be a job requirement.

  32. Let's try an experiment where women pick a man out to date but instead of putting them behind a screen they aren't allowed to ask questions about career or income.

  33. Don't concede that discrimination or sexual assault in the workplace are "serious issues". Deny it all! Radically deconstruct it! Force feminist researchers to come as supplicants and present real statistical data properly presented, with analysis properly applied. Not emotions. Not feelings. Not slogans to build female solidarity like "one in five". No crybullying.

  34. You did the same sort of strap-hanging onto the transgender community as it helped your cause. This is no different. Clean your own room first.

  35. The super rich have imported war into the west because the western people HATE the rich and can see the rich are only rich because the fed printed off money and handed it out like candy to themselves. 4TH wave feminism is a smokescreen like SJW and BLM and the leaders only give traction to those groups because it gives traction to the causes (like more control over the population) of draconian laws and more jails.

    We are in the middle of fascism and my life was destroyed because of it.

  36. "…Once the curtain dropped, the case study shows that the number of women who were selected doubled – they went up 50 percent."

    It's hard to trust anyone who thinks a 50% increase doubles the number. A 50% increase in quantity 1 is 1.5; in quantity 10 is 15; and in quantity 100 is 150. On the other hand, a doubling of quantity 1 is 2; of quantity 10 is 20; and of quantity 100 is 200.

    There IS a percentage increase the equivalent of doubling and that is 100%. A 100% increase in quantity 1 is 2; in quantity 10 is 20; and in quantity 100 is 200.

    If the person who said that the quantity doubled – increase 50%, then they are deficient in middle school arithmetic and can hardly be trusted to comment on, let alone verify, a thesis that depends on statistics.

  37. Maybe they could have had to panels of judges for each audition 1 with and without screens? Then comparing the results would give you a really good idea of the effect

  38. Could it be possible that more qualified women applied for those positions, as a proportion of all applications? If ten people apply for a job, including only one woman, assuming they are all qualified and suitable, there is a 10% chance of the woman getting it. However, if four women had applied, there is a 40% chance that a woman will get it, all things being equal.

  39. Who needs screens when yo have quotas? 50% of all orchestras should be female, black, Asian, fat, bearded people, LGBTQ, and tone-deaf. The sound might suffer but think of the virtue!

  40. Worth watching, as usual. I would subscribe, but that gets me all AEI's videos, which I don't want. Please consider giving TFF its own channel.

  41. Australian civil service found the exact opposite, hiring by gender neutral resumes removing and names and sex indicators so it would be on merit alone. The feminist politicians who had it adopted a a cure to sexist hireing emediatle backtracked when I did the opposite. The new system was hiring even more than normal males. In the process it uncovered a sexual bias the used to practise to hire more women than men. Funny think. That. Had it proved the conclusion feminists wanted to be true there would have been huge hype, laws, marches, scorn and probable a day called gender hiring equity day. The reality was that they reversed the practise and swept it under the rug. What really amazed me is that they really did uncover unfair hiring practices and somehow it’s ok to go back to it. They are deliberately reinstated a sexist practice that favours women. EQUALITY My ass

  42. I teach violin. For the first 15 years that I taught, I always had approximately half girl students and half boy students and for many years, it leaned towards more male students. I stopped teaching for a couple of years to live abroad with my husband. Three years ago I moved back to restart my studio. For every 4 or 5 girls I get in the studio, I only get 1 boy. Granted, my sample size is incredibly small and I'm also dependent upon Google algorithms in part for exposure to new students, but it has been consistent and feels off-kilter. There is even an attitude among the girls that girls "rock" and they are quite vocal about being "better" and superior to boys. My pet theory is that if the little girls have picked this up, then so, too, have the little boys and perhaps even their parents, who may no longer be willing to invest in their sons who may also be feeling inferior and not displaying what parents might look for in order to seek out lessons. I'm wondering if anyone else has had this gender attitude and/or imbalance in their practice.

  43. I appreciate your effort, but expecting that socialists embrace the truth from data science facts is quite too much asking.

  44. Blind auditions or blind judging is still a good idea where there is some subjective evaluation going on. Good examples are cooking, art, music, creative writing, etc. In each of these the prospective judges could be influenced to like a person more because they resemble themselves in some way or because they have some type of notoriety. Blind judging is helpful to use in these situations where possible.

  45. My study of this video indicates that dog owners look like their dogs. I have a sample size of one but i can make a definitive statement.

  46. I always found the popular conclusions of this study to be interesting, and it sounded plausible enough that I never really checked.

  47. 04:30 Why is professor Banaji wearing a portable tent?

    You can trust me to bring you the most cunning and incisive commentary.

  48. There was a recent study blinding resumes in STEM – female sounding names got a 6% benefit.

    after the results showed women benefit asymmetrically the authors said blinding should NOT be used to achieve fairness. (from memory – exact numbers and wording probably differed)

  49. One of my favorite new terms coming out of the Intellectual dark web — I think it was coined by Bret Weinstein — is the notion of "idea laundering". Start with the desired conclusion, write a paper that manipulates data to arrive at said conclusion, get it published in a journal and then cite it as far and wide as possible until it becomes accepted as axiomatic truth.

  50. Interesting. I was hoodwinked by this "study" as well then. Sounds like the study didn't produce the results that were expected, so they injected virtue-signaling pseudo-conclusions to the study to make it more PC. I'll file this alongside the BS we all hear about the "gender wage gap".

  51. Blind auditions? Hell yeah, let's use that to hire firefighters, meaning, whomever can carry a grown person down a ladder gets the job.

  52. Anyone who claims that a 50% increase in hiring doubles the hires shouldn’t be a scientist in the first place due to mathematical ignorance. To double (2x) a population, it needs to increase by 100%. A 50% increase obviously increases the population by 50%, meaning it grows to 1.5x its original size.

  53. Men are faster, stronger, harder working, more focused and, at the high end, smarter than women. Women are NOT equal to men and never will be.

Leave a Reply

Your email address will not be published. Required fields are marked *