Evaluating Sources – Critical Thinking



I’m Sarah Morehouse from the Empire State
College Online Library. This is the fourth unit in our series of videos
about evaluating information sources for research and everyday life. In this unit, we talk about misconceptions
people have about the trustworthiness and even infallibility of information sources
that come from people and institutions that we respect. We also talk about how we can trust information
sources at all, if trusting the people and organizations that created them is not enough. To give you an idea of the kind of thing we’re
talking about, here is a clip from a talk by Gayle Bush, who is a professor in the National
College of Education. While you’re watching it, please think about
two questions. First, what does Dr. Bush mean by “authority”? And second, if we’re supposed to question
the experts, how can we know what evidence is valid? “In the twentieth century, we taught that
you need to answer the question. And in the twenty-first century, we teach
that you need to question the answer. Now the students coming up are very uhh facile
in finding information, but not very much so in evaluating it. And the teachers we have came up predominantly
in that old style of learning: cognitive authority, meaning that we could trust – that there was
authority to the information. And now we know that we need to – that we
cannot trust every information source that we see, and we need to know how to think critically,
how to maintain that critical stance, how to employ it, and then how to stay open to
new ideas and information, and to be open to change your mind if you find evidence that
is valid…” There’s no such thing as a perfect, all-knowing,
infallible human. And besides, part of becoming educated is
gaining your own expertise, and a basis for your own informed opinions. So instead of just trusting an information
source because it’s in print, or comes from a big name, or was given to you by someone
you respect, you always have to be on the lookout for signs that it might not be trustworthy,
and we will go over many of those signs in this series of videos. If the information source seems like it might
be untrustworthy, you need to investigate. Then you can choose for yourself whether or
not you are willing to use it. Within every information source, you will
need to examine the information present for its validity, sort out correct information
from incorrect, and separate fact from opinion. I read a blog called Regret The Error. It’s the blog of a journalist who reports
on mistakes and misleading statements made in the news. As you will see, sometimes even the most well-regarded
news sources fall short of their ideals. This article is about an NPR radio show that
did an expose on working conditions in the Chinese factories that produce iPhones and
other Apple products. It turned out that the person they were interviewing
was making a lot of things up. The article comes to the conclusion that the
radio show’s fact-checking wasn’t nearly rigorous enough. But even good fact-checking can miss things,
which is why it’s important to maintain an open mind, but also a skeptical one. Here’s another blog post from Regret the
Error. This one is about how CNN and Fox News both
made a huge mistake when the Supreme Court first ruled on the constitutionality of the
Affordable Care Act. They were in such a hurry, that they read
only the first part of the opinion and thought that the Supreme Court had struck down the
Affordable Care Act as unconstitutional. Within minutes of releasing that story, they
realized they had it backwards and retracted it with apologies. But sometimes even a quick retraction doesn’t
control the damage. Not everyone checks back, so they may not
find out that the retraction even happened. What’s worse is first impressions matter
even more than people realize. Human beings anchor on the first thing they
see or hear, and form their emotional reactions based on that. The correction doesn’t make as strong an
emotional impression. So their judgment is still based on the incorrect
information that they heard first, and not on the correction that they heard later. So if even trusted news sources make these
huge mistakes, what can we do to make sure we’re not taken in by them? First, cross-check with other sources. If there’s general agreement, then you can
feel more confident in the accuracy of the information, but if all the stories are different,
then you shouldn’t have much confidence in any of them. And if most of them agree but one or a few
disagree substantially, you will have to look deeper into the matter. The second thing you can do is come back later
and make sure that the information wasn’t later retracted, amended, or corrected. And finally, use your common sense. Use what you already know, and your powers
of reasoning. The same kind of mistakes can even make it
through the peer review process of scholarly articles and monographs. Here is a post from another blog I follow,
called Retraction Watch. It’s reporting on a scholarly article that
went through peer review and got into print even though the conclusions were based on
mathematical errors. Many scholarly retractions are based on this
kind of thing because most fields of study require the use of statistical analysis, but
not everyone is a statistical expert. Unless you’re a statistical expert, there’s
not too much you can do to identify when mathematical issues are present, except pay attention to
the retractions. Journals publish retractions in every issue,
so if you are concerned, you can either check the Retraction Watch blog, or check the following
issues of the journal in which the questionable article was published. Scholarly articles are also retracted because
of scholarly misconduct. Types of academic misconduct include plagiarizing
other researchers, not disclosing financial conflict of interest, falsifying your data,
massaging your data to say something other than what it actually says, publishing things
you know aren’t true, and trying to squeeze multiple publication credits out of one piece
of research. Here is another blog post from Retraction
Watch. This article was initially retracted because
it was being published in two journals at once, which is considered cheating because
you’re trying to get more publication credits out of the same research. But it turns out that there were more serious
reasons behind the retraction. The author, whether he was trying to deceive
everyone or whether he was deceiving himself, had written a bunch of pseudoscience. In this case it was so obvious that an economist
was able to recognize the problems in the article even though it was a biology article. In other cases, it’s more subtle and may
be out in circulation for years before someone finally realizes that the results of the study
can’t be replicated. It’s much harder to retract a scholarly
monograph. It’s very unlikely that the books will be
pulled from the shelves. But scholarly journals typically contain reviews
of scholarly books, and if a book is deeply flawed, you will find that information in
the scholarly book reviews. One of the reasons we have the peer review
system for scholarly publication is to catch those problems. But peer review isn’t infallible. There are usually only two or three peer reviewers
for an article, and they have their own blindspots. They have limited time and they don’t usually
have the option of checking every bit of data and going over all the math. Once the article or book is published, it’s
much more likely that somebody will spot the problem because there are more eyes on it. But just like it is for retractions in the
news, by the time you retract it, it’s too late because people have already anchored
on their first impression. An important thing to remember is that for
every retraction that happens, there are still more mistakes and misconduct that don’t
get retracted. Reputable news sources and scholarly sources
are reputable because they usually get it right. They are usually authoritative and reliable
sources of high quality information. But when they do slip up, it’s very bad
because people are trusting them, so they tend to accept the mistakes and lies without
questioning. We can have the same attitude toward people
we respect. If a doctor hands you a pamphlet, you usually
trust what’s in the pamphlet, but the truth is the doctor may have been too busy to read
it herself, and she was just given it by a pharmaceutical company rep. When your professor sends you to a web site,
you might think that web site is up to academic standards, but professors are busy people
and their expertise outside their own subject area may be limited. When a book is in a college library you expect
it to be college-level, but libraries buy content in big bundles and not every single
item gets scrutinized. The solution is to always have an attitude
of questioning. In other words, have a critical attitude. In this case, critical doesn’t mean negative
or disapproving. Critical thinking means a way of thinking
empirically, meaning based on facts and evidence from the real world, and a way of thinking
logically, which means being sure of the relationships between your premises and your conclusions. Critical thinking is a way to decide if something
is true, false, partly true, or sometimes true. Critical thinking identifies the right questions
to ask and the problems that need to be solved. It helps you to examine your own assumptions
and the assumptions you find in the information sources you use. It help you to recognize the attitudes and
values that underlie what you believe is right and correct, and what is presented as right
and correct in your information sources. It helps you to evaluate the validity of evidence
and the reliability of conclusions. Critical thinking isn’t so much a skill
as a whole set of skills, and it’s something you can get better at with practice. The next units of this Evaluating Sources
video series will talk about different critical thinking skills that you can bring to bear
on your research materials and on information sources that you use in your everyday life. Please take some time to visit the Supplemental
Resources link that is listed on the video information page. In Supplemental Resources, you’ll find links
to the Retraction Watch blog and the Regret the Error blog. Look through the blogs and note the different
kinds of errors and misconduct that are being reported. Ask yourself what are some of the possible
consequences of some of these mistakes and ethics violations. The Supplemental Resources page also has links
to other web sites and videos on this topic if you want to study this topic in more detail. In the Supplemental Resources page you’ll
also find a link to a very short self-quiz. When you complete the quiz, it grades itself
and it will tell you what you got right and wrong. It will also explain the answers to any questions
you got wrong. When the quiz is completed, you will have
the option of downloading or printing a certificate, which you can use as proof that you went through
this video tutorial, if you need that for one of your courses.

Leave a Reply

Your email address will not be published. Required fields are marked *