Getting Ready for the 2018-19 State Assessments

If you have questions please type them
in the question and answer box we’ll get to them as we can. Here’s a list of the contacts – this is
possibly one of the most important slides in the presentation. Email
addresses for that people you might need to get in touch with – Bryan Truesdale is
our chief for the assessment group. Jen’s about access the test access
there’s an RA account ra – LEP email, that RA account with your access
questions if you have accommodations questions for
that english language english learners assessment contact Rebecca Baum- Lehmann.
Lisa Hampe is a special education advisor dealing with assessment. Drew
Shuckman works with Nate he’s not here today, Lisa is not here today, as well
however if you have questions I’ll get back to – email them and I’ll get back to
you quickly. The people who are here are Rebecca, Diane, Charlie, and Craig and
Linda – who is also here whose contact information isn’t up there yet, but will
be here’s what we’ll talk about today. We’ll do some general information, and
we’ll talk about an overview of all the assessments for the state there were
some resources tools the PSSA and then the Keystone exams. First thing we’ll
talk about is some updates if you have accountability questions, we don’t answer
accountability questions and assessment we’re not allowed to answer
accountability questions and assessment. Please email the RA account if you have
accountability questions [email protected] There’s a link for the school
performance profile again with an RA account for questions we have our PVAS
the Pennsylvania value-added system. There’s a link for information about
that with an email we have our educator effectiveness system with an email and
if you have test security concerns, but I’ll give you a heads up we’re working
on coming up with a new number and email address for test security that won’t
happen this year if that happens it will happen for the following year, but we’re
working on a new email account and a new phone number. If you have test security
questions and if you do have test security concerns at all the preferred
method of contact is the email account. Please email that RA -ed- PSSA –
[email protected] and use the subject line. Use as much detail in the subject line
as you can so we know how to respond to your question. We
can triage it better if you absolutely have to you can call the number of the
email account is the preferred way to report security concerns.
So here are two things that are new for this year gonna start with the second
one first second one is the translators have to complete the piece that you only
have to complete the piece that for the TA’s not all of them for before the TA’s
they have to do the PSTAT any translators or interpreters because
they’re dealing pretty closely with the test they’re actually reading test
questions so they need to know what they can and can’t do based on what’s in the
PSTAT, so please have all the translators and interpreters complete
the PSTAT this year and new for this year for all of our assessments the
primary test administrators initials need to be bubbled in on the back of the
student answer booklet. That’s what it looks like on the back of the student
answer booklet. All we want to know is the first middle
and last initial and if there’s more than one test administrator in the room.
Sometimes what schools do is they’ll have one test administrator start to
test session now in the period it ends that test administrator leaves and
another one comes in so there’s multiple test administrators in the room. We need
to know if that happens with this test administrator or with this testing
session or this testing room we’re doing this not as much to keep track of test
administrators, but we’re doing it so we can sort students we know what group all
the students are in so we can classify them we can arrange them we can sort
them by all the students that had XYZ test administrator – we can that’s that
saves the district the hassle of us contacting the district and saying who
are the test administrators for all these kids so in the long run it saves a
lot of districts a lot of work. I’m not going to follow all of these
links but there are links here these are hot links for all of these issues access
for ELLs I guess now it’s access for ELs not ELLS, and the same RA account that we showed
on the last one, but that’s a hot link if you want more information about that
click on that link. Same thing with WIDA the PBA for NAEP
and PASA and I mention Liinda is also here.
That’s her email address at the bottom [email protected] For English learners all English
learners are required to take the access for Els the English language proficiency
test if the English Learner meets the criteria to take the pasa instead of a
PSSA then they should participate in the PASA, I’m sorry participate in the
alternate access free ELs not the PASA, for the PSSA, if the English learners are
in their first 12 months in the United States schools not Pennsylvania schools,
but in the United States schools they don’t have to take the reading or ELA or
literature State assessments. They do have to participate in the math and a
science, however their scores don’t count they participate just for participation,
but not for their scores. If they’re eligible to take the PASA
then they follow the same requirements as the PASA for non English learners for
more information another hot link for the 2019
accommodations guidelines for EL’s. Here’s the PSSA testing window for the
upcoming year. Notice that the math and science and the
make ups is a much smaller window than we’ve had in the past – keep that in mind
when you’re doing scheduling for the Keystone exams we have the
winter spring and summer the winter has two waves December in January. You can’t
take the same test in winter wave one and winter wave two you can however take
one test for example algebra in winter wave 1 and then biology and winter wave
2. If you’re not proficient in winter wave
one or two you can retake the same test in a spring so you can retake the test
in the next administration. I’m gonna hand this over to Linda to
talk about the PASA. Thank You Charlie. Good morning everyone again my name is
Linda, I work for the Pennsylvania training and technical assistance
network as the statewide special education assessment coordinator I also
work in conjunction with Bureau of special education advisor Lisa Hampe. I
believe in prior slides you did see each email address both for Lisa and myself. In a subsequent slide you will see our contact information again we
appreciate the opportunity that is afforded to us today to share some basic
updates regarding the alternate assessment. In short, this slide and the
subsequent slide talks with each of you about the participation requirements for
the PASA to remind everyone students with the most significant cognitive
disabilities who meet the six eligibility criteria may participate in
the PASA and determination is always based upon an IEP team decision. It is
important to note however that with the changes in ESSA that in prior years a
focus used to be upon reallocation of proficient and advanced scores, but with
the changes in ESSA regulations the focus is upon participation and we it
stipulates that no more than 1% of the EL’s s total tested population should
participate in the PASA. The Bureau of special education provides frequent
updates on this particular policy, and if you have additional questions on it Lisa
or I are happy to assist in answering those specific questions. To go a little
bit more deeply something that is new this year is that the sixth eligibility
criteria for the PASA has been updated as a result of statewide stakeholder
input. There was an emphasis on enhancing the EL’s ability criteria to provide greater clarity to teams as a result of this
focus upon one percent of the total tested population participating in the
PASA as a result a publication produced by the pennsylvania training and
technical assistance network. In conjunction with the bureau of special
education has been released the title of that document is called a
decision-making companion tool for IEP teams. That publication is available
through the bureau of special education website. It is also available on the PASA
digital site and is available through publications through the PATAN system. I apologize there’s the slide that
focuses upon the details I just shared. This particular slide focuses on the
2018-19 PASA testing cycle as you will see we are presently in the enrollment
window and the enrollment concludes on November 16. The bureau of special
education has provided details in a subsequent training referenced and known
as the PASA getting ready training webinar. During that particular webinar,
we give guidance on the fact that when the enrollment closes this year on
November 16th and any other additional enrollments should be focused upon
students who are new to the LEA or service provider – such as a student who
is newly in the agency because of a move in or perhaps a student who is newly
identified as either IDEA eligible or PASA eligible. In those instances the
PASA vendor and the enrollment may occur for those particular students. Any other
situations outside of newly new enrollments would be a requirement that
a contact be made to PASA and to the bureau of special education. To enroll
those additional students as you will see there is test administration
training there is a window for test administration and with the PASA, the
school is able to determine a schedule that it would like to follow in which to
administer all the content areas. That means during that window of February
5 – April 12, the LEA or service provider determines in what order sequence and schedule it
would prefer to administer the assessments for students who are
eligible for it the PASA. It’s important to note the material due date is just
that it is for materials that means that no additional testing or uploading of
student performances can occur during that time period. All test administration
must be completed during that window above which is February 25th to April
12th and then score reports are provided in August for service providers and
local education agencies for their individual students assessed on the PASA. Just as a reminder, the PASA is delivered
digitally that is all student assessment materials – the actual test itself is
delivered digitally to the particular inbox of the assessor, but student tests
materials are mailed separately. If you have questions regarding test materials,
that is the assessment website that is the public-facing website email address
that you see there or web. I’m sorry not the email address, but the website
address as a reminder all performances that are recorded for the student taking
the PASA are uploaded digitally through the PASA digital site. If any LEA or
service provider is in need of webcams the possible does continue to have some
webcams available that are free of charge. This is new this year for the PASA,
students eligible will be taking a PASA ELAand math assessment
consisting of two tiers instead of read levels. Any individual who’s listening to
this webinar and is familiar with the PASA recognizes that over the years the
PASA has been comprised of three levels of cognitive complexity. More recently
the science PASA assessment has been reconfigured into two tiers. Beginning
this year, the passing language arts and math assessment will be streamlined and
will be systematized to follow the same leveling system as science so all
students will now be eligible to take one either tier one or tier two of all
PASA assessment. As a reminder the PASA rating questions are incorporated
into the assessment this year known as the PASA ELA and it will be
operational with scores being reported back to schools next year in August. As we mentioned previously test training
specific to enrollment test administration and security are
available at the following web site. The PASA assessment org that is the public
facing website for PASA. Those individuals school personnel who do more
in-depth work with the PASA – if you secure a login information then
you can be able to you are eligible then to access the training guides training
enrollment guides are currently up and available for school personnel who are
in the process of enrolling students. A previously sponsored BSE and PATTAN
webinar entitled PASA getting ready 2019 updates was conducted on September 25th
2018. The webinar has been recorded and is available on the PATTAN website
beginning this week the PowerPoint presentation is already posted to the
bureau of special education website. Beginning this week, webinar and
PowerPoint presentation will be a complete package and available on both
locations BSE and PATTAM website. It’s critical that all PASA assessment
coordinators and special education administrators be familiar with this
training so that they understand the process and procedures associated with
enrollment test administration returning and materials and resources and tools
that are available to assist in this process. There is one additional slide I know the
Charlie has shared some contact slides previously, this is specific to the PASA.
At the bottom, you will see that there are two email addresses unique for
the PASA if you have a general inquiry about the test or just about a procedure
[email protected] as the web excuse me the email address that you will want
to use if you are in the midst of the testing window and require technical
assistance, or you have questions pertaining to technology the past
digital email address is available there for you its [email protected] And then our contact information for both Lisa and myself is available at the top. With that I believe I’m going to turn it
back to Charlie. He will speak further with you about resources and tools. Thank
you Linda, I guess I should have introduced myself in a beginning I’m
Charlie Wayne. My primary responsibilities at PDE is math
assessment. I’ll talk this one slide about resources and tools and we’ll just
go over that real quickly and then we’ll get into some math specific stuff again.
All of these are hot links to direct our testing calendars for PSSA and for
Keystone. There’s a link for the performance level descriptors and the
cut scores the performance level descriptors are abbreviated PLDs and
it’s a highly underutilized resource. Tt’s a really good resource so what the
PLDs do is they describe very generally what a student would look like if that
student is performing at the four different performance levels: below basic,
basic, proficient, and advanced. So if you want to know is my student generally
at the proficient level this these performance level descriptors could give
you an idea about that. Here there’s some things that students should know and be
able to do at each of the each of the levels and they’re very general. Again,
paints a broad picture, but they’re very useful. The technical reports are also
posted up there on our website the technical reports are very long and they
can be very tedious. However, they contain a lot of great
information. For example, if you want to know what the maximum score was on the
test for a given year it’s in a technical report if you want to know
what scaled score 35 raw points corresponds to that’s in a technical
report. So there’s a lot of really good information in a technical report if you
want information about accommodations. There’s a link there for that there’s a
link there for SAS and a link for the CDT. Starting off with math, there’s a bunch
of links hot links as well most of these we’re gonna go over the
PSSA. This just a link to our general PSSA page. The math test design – we’ll see –
there’s a link for the item in scoring samplers. There’s new samplers put up
there every year – scoring guidelines. It’s a general description of scoring
guidelines are available. Our calculator policy hasn’t changed for a long time,
but that’s available on our website. Essentially we don’t want kids using
calculators that do algebra for them that make noise. We went all the
information cleared in the calculator before and after an assessment so
nothing is taking taken into the assessment that would give a student an
advantage and nothing is taken out that would compromise the integrity of the
test. There’s a link to our anchors and our glossary. Our glossary for math is
another highly underutilized resource. Many math terms are in the glossary and
it not only tells you the definition as to how Pennsylvania looks at that
definition, but it also tells you the grade level when that term is first
introduced. The grade level part is useful, but also the definition for
example is an it is is an isosceles triangle – a triangle with exactly two
equal sides or could it have three equal sides . In other words, is an equilateral
triangle. Also an isosceles triangle you read the glossary you’ll find out how
Pennsylvania views that. There are five reporting categories at
every grade level three through eight. In math, the reporting categories vary based
on the grade level this shows you the reporting categories at each grade level.
So base 10 is at grade 3 to 5, but it’s not in grade 6 to 8. There are five of
them again for each grade level. The test design remember this year in
2018 the test was shortened so we have two sections we have 20 multiple choice
questions in each section that count toward the students score. We have eight
multiple-choice questions that are field test items. We also have two questions
and again these are split up so there’s four field test questions in each
section and then there’s two multiple choice one multiple choice question in
each section that is a core linking item. That doesn’t count for student scores,
but it’s not a field test item and the core linking items help with equating.
Now, that helps determine that how the difficulty of the test changes from year
to year. There are a total of four open-ended items. One of them is a field
test two one each section one section has one field test item. The other two
the other section has two core open-ended items on it and then there’s
an estimated time and at a 40 multiple choice questions. Each one is worth one
point the three open-ended questions are worth four points – each for a total of 52
points for a possible score. In each reporting category there are roughly 7
to 9 points in each reporting category and that varies based on where the
open-ended question falls, and no I’m sorry, but we don’t release where the
open-ended question will fall ahead of time so you don’t you don’t know that
going into it. As I mentioned, there’s four field test
questions in each section. So it’s an embedded field test based on the
Pennsylvania core standards. All the open-ended items are scored on a zero to
four-point scale. We have a general description of the scoring guidelines,
but we don’t have a general rubric. Every grade 4 through 8 – it’s a formula sheet.
The formula sheets are available on our website. You can have this posted in your
classroom throughout the year during the test students can have this on their
desk with them. When you hand it out it has to be a clean copy. If it’s posted on
the wall during the assessment which is allowed it has to be a clean copy you
can’t have anything else written on there. You can’t have other formulas, you
can’t have things highlighted or underlined – which has to be a clean copy,
but you can have that on your wall. During the year and during assessments
all the multiple-choice items are based on the assessment anchors and the
anchors are aligned to the Pennsylvania core standards. The eligible content is
now considered assessment limits – that is we can’t ask questions that go beyond
the eligible content, but we can ask questions that go between the eligible
content. All items can now cross including
multiple choice items open-ended items, as well can cross eligible content
descriptors anchors and reporting categories. So now we’re not asking
questions based just on the eligible content for multiple-choice questions, we
can ask questions that are more math, more math-related, not discrete-skill related.
My favorite example of that type of question is here are the dimensions of a
room – here’s the cost of carpeting. How much does it cost the carpet the room? If
the items can’t cross eligible contents etc., then we couldn’t ask that question
because that comes under two different reporting categories, but it’s a very
good real-life very common math question. But now we can ask questions that are
like that and ultimately all items aligned to the anchor when the items
crossed anchors are eligible content or reporting categories. We align it to the
anchor that we think is the meat of the item. The meat of the item or the hardest part of the item or sometimes if we think
it’s equal, sometimes the distractors are written so that they show common
mistakes in one of the anchors, not both of them – but in one of
them so then we choose that anchor to align that item to and again use the
glossary the glossary is available lots of people don’t even know it’s there, let
alone use it. So use the glossary. Look through there again. Another example is a
number sentence. In Pennsylvania what is a number sentence? So for our open-ended items, you know the
open-ended items now count for a higher percentage of the total score on the
test, because we cut back on multiple-choice questions. So the the raw
point value is the same, but the percentage is higher so knowing a little
bit more about open-ended items is probably a good idea.
Here’s how some open-ended items might be phrased. This isn’t an exhaustive list –
this is just just an example a few types of phrasing that we use show or explain
all your work that’s probably the most common one you’ll see and we don’t want
to say just show your work if we say show your work, and a student tells us in
words what they did and they don’t get credit, because they didn’t show their
work. They explain their work. We don’t want that to happen. So we say, “show or
explain all your work.” You can do one or the other. You don’t have to do both and
you can mix them. Sometimes it can be showing, sometimes it can be explaining.
That’s fine as long as it’s complete. We still might see show all your work and
explain why you did each step so then you have to do both. You have to show
something and explain something, and by the way – when you see the word explain in
order to get full credit there has to be some words in the answer. You can’t get
full credit for and explain without using words.
Same with describe, you have to have some words in your response. We might ask
explain why for example – explain why not all rectangles are squares.
We might ask you describe describe how to do this describe the characteristics
of this. They still do a lot more work than is
required on their open-ended items and of course there are a lot of students
that don’t do enough work – as well have students read the items very carefully.
Especially the open-ended and answer exactly what’s being asked. That’s hard
for adults to do. Adults still skip directions, they don’t read everything,
they answer what they think is being asked not what’s really being
asked. Have kids answer exactly what’s being asked, that way they’re answering
everything they need to and they’re not doing more work which would contribute
to test fatigue. So here’s just one example of an
open-ended item. It’s not a refined item. It’s not a good item, nothing like that.
It’s just here for illustration purposes. Don’t read too much into it. Roy is running for class president,
he pulls 50 students. 60% of them say they’ll vote for him. There are 350
students in Roy’s class. Based on this survey, how many students should he
expect to vote for him? Show or explain all your work. So we’re looking for an
answer and work. Part B – Roy will sell fruit drinks to raise money for his
campaign. The fruit drink costs 10 cents and he sells it for 50 cents. How many
fruit drinks does Roy need to sell to earn $800 or $80.
Show all your work and explain why you did each step. So we need an answer, work,
and an explanation. Two things required for part A; three things required for
Part B. Part B should be worth more points
and in this rubric and it’s just for this example item only the answer in
both parts is worth a half point the explanation. I’m sorry the work in both
parts is worth the full point and the explanation in Part B is worth the full
point for a total of four points. Notice that in Part A and in Part B for the
work and the explanation you can get a half point. If you do something right, but
it’s not complete so if something has to be correct and your work and explanation
to get that half point – so what we’re trying to do is we’re trying to reward
students for for doing something that’s more than nothing – if we wouldn’t give a
half point and you didn’t get everything correct – you’d get a zero for that. So
even if you’ve got the correct answer for A and B and you did something right –
in the rest of it, but it wasn’t complete. If we didn’t get half points, you’d get a
1. By giving half points, you can raise your score to a 2. Here’s how we do the
scoring then – if you get everything right you get 4 points. If you get 3 to
3 and a half points, you get a 3. That’s because you didn’t earn a 4 –
2 to 2-and-a-half you get a 2. You didn’t earn a 3. The half points again or
to help build up your score. It’s possible that someone with a 2 and a
half got 2 half points somewhere and would have only gotten a 1 – had they
knocked out in those half points. If you get a half point, we give you a full
point because then you did something right. We want to give you credit for
doing something right and of course if you don’t get any points, you get a zero. So some notes about open-ended items
every year – we say this and when I talk to people in in schools, or at
intermediate units individually lots of people aren’t aware of this but it’s on
this PowerPoint every year it’s been there for for a lot of years. Ever since
we’ve been doing this yes and check is a valid method. Students can use guess and
check that’s fine, but they have to show at least two incorrect guesses to get
full credit for their work. That’s because we need to know we need evidence
that the student knows how to do this item. If they just happen to get lucky
and guess right on the first or second try – they’re not showing to us they know
how to do the item. They have to be able to provide to us information that shows
more than just , “okay I got a lucky guess.” If they get a lucky guess, but they
explained to us why they arrived at that guess in the first place, and that’s more
than guess and check – then they’re providing some rationale for their
solution and they might not need to show two incorrect guesses. But, if it’s
strictly guess and check at least two incorrect guesses need to be shown.
Not all open-ended items require a Y at one point they did, but they don’t
anymore. So not all of them require a Y and at least half the score points have
to come from one anchor. The other two score points can come from anywhere else
in any of the reporting categories, but half of them have to come
from one anchor. Open-ended items don’t have to be perfect to get a 4. They
can have a misstatement. The misstatement can’t be something that’s wrong in their
work, or explanation, but it can be saying something like, “I’m calling this
rectangle a square when it’s really not a square.” That might be incidental to
solving the problem, and they just happen to use the wrong word. So sometimes good papers have blemishes and blemishes don’t lose credit. If it’s strictly just
a blemish – if it shows a real misunderstanding about what’s being
assessed, that’s different. A missing dollar sign may or may not be
a blemish. It depends on the grade and what’s being asked for. With thinking about a missing dollar
sign brings us to the point of units a paper that has missing or incorrect
units can’t get a 4. So we’ll go through the paper and we’ll score it if
it gets a 4 and units are missing you’re incorrect.
It’s knocked down to a 3 if we go through and score it and it gets a 3
to a 1 and units are missing. You’re incorrect – it stays out a 3 or a 2,
or a 1 – the score doesn’t change, you just can’t get a 4 if the units aren’t
right. If there is an answer box, we don’t have to use the answer box. Sometimes
students transfer answers from their paper to the answer box and they make a
mistake. They write down the wrong thing whatever is in the answer box overrides
everything else. The student is telling us that’s my answer.
If that’s wrong does it matter what’s above it? They lose credit for the answer.
However, if they don’t use the answer box and they have the answer somewhere on
the page, and they let us know that this is my answer. They’ll get credit for
that answer if it’s right. We have to know it’s their answer –
generally that’s not a problem, but sometimes it is so that’s why we have an
answer box. Sometimes also the answer box has the units after it. So if we’re
looking for the area of a rectangle and we want it in square feet – the answer
blank or the answer box might say blank square feet. So it gives us the units, but
the answer box can be useful for things like that. We try to eliminate carry
through errors, but they still happen with lots of items. It’s still possible
to make a mistake in one part, but then use that mistake – use that wrong answer
that you get from that mistake and another part correctly. If that happens,
you’re only penalized once. You’re only penalized the first time you make a
mistake. If you use that incorrect answer in a correct way – later in the problem –
you’ll get full credit for that because your procedures right. Your input wasn’t
right, but your procedure was right and you used it in the right way. So a carry
through error still gets full credit. Students in grade 3 will get rulers.
Students in grade 4 will get protractors. Rulers are scaled to the 8th inch and
the millimeter, but students have to measure one unit higher so they’ll
measure to the quarter inch and centimeter. A measure one unit higher
than a scale – they have to use the rulers and the protractors that are provided
with the assessment. However, when the assessments is done you can keep those
rulers and protractors and use them then throughout the year, but again next year
then for the assessment you’ll have to use the new rulers and protractors again.
The ones that come with the assessment as always grade 3 can’t use
calculators and their test booklets are consumable. That means they mark their
answers right in the test booklet they don’t have to transfer their answers to
an answer booklet. Grade 4 and up they do have a test booklet and an answer
booklet, and you can use a calculator on all but the first few items and the
number of items change from year to year. It’s three four or five depending on the
year, but on the rest of the test you can use a calculator as long as the
calculator meets the guidelines in the calculator policy. I’m gonna turn it over to Diana Becca
now for the ELA part. Thank you Charlie. This is Diane Sam Aska.
I am the ELA Keystone literature assessment lead at an accommodations
lead at PDE and I work with Dr. Rebecca Baum-Lehman who also works in
assessment, but she is also the curriculum lead for ELA and Keystone Lit.
on this side. On this slide you’ll see that we have eight resources for the
2019 ELA PSSA test. Each resource is a live link that will take you to the
appropriate page on the PDE website. If the link does not open for you when you
click on it – please switch browsers. This often will correct the problem. You may
also go directly to the Pennsylvania Department of Education website and use
these bread crumbs from the PDE landing page. Click on data and reporting – then
click on assessment and accountability and then click on PSSA. The slide that you see now is the
landing page for the PSSA and it displays a brief description of the
purpose of the annual statewide test, as well as links to resources regarding
subject specific materials, reporting documents, testing accommodations,
administration materials, assessment anchors, individual student report guides
the performance level descriptors as Charlie described them earlier, cut
scores, and also the application process for educator assessment committees, as
well as updates where the data interaction site just a plug for the
educator assessment committees. We encourage all educators who are
interested in the assessment to click on that link and to apply to be a member of
one of our assessment committees. Pretty much guarantee that they will enjoy the
experience and ask to come again. A PSSA English language arts test plan
shown in this table is organized by grade and broken down between multiple
choice (MC) evidence-based selected response (EBS) are those are the two
two-part multiple-choice questions. Short answer essay – which is grade three only
and text-dependent analysis (TDA) which only appears in grades 4 through
8. Core items which are the items which count for a student’s score are
also distinguished from items that serve the role of psychometric use – which is
equating and field test (FT). The tables following that particular view summarize
the ELA test design and you should notice that the design perform is
from 2018 onward. The English Language Arts test design did not change for 2019.
Therefore, the title of the document continues to read PCS tests on in 2017-
18. Know that this is the same design for 2019, and a new document will not be
posted. The blueprints are organized around three reporting clusters – that is
reading conventions of standard English which is writing, and text dependent
analysis, which includes reading and writing based on the expressed emphasis
contained within the pennsylvania core standards. As stated in the released PDE
assessment anchor and eligible content documents, the reporting categories are: A
which indicates literature text – B which indicates informational text, D which
indicates the conventions of standard English –
which is part of writing, and E which is the text-dependent analysis – which
includes both reading and writing. In addition, the first two reporting
categories literature texts and informational texts are understood to be
the genre of reporting categories. The genre of reporting categories A and B
for ELA are mapped as part of a dual alignment into the core competencies
reporting categories. Typically, there are three themes that
are prevalent throughout the PCs ELA standards and these themes appear in
both the literature and the informational text that will appear on
the core based PSSA ELA test. There is a table that also shows how the results of
the specific PCS based assessment anchors and eligible content are mapped.
Please remember that the ELA test design was revised in 2018. The revised
assessment anchor reporting categories are as follows: one reporting category C –
which is writing, has shifted from the statewide accountability tests to the
local level and to the English language arts standard one point for writing
continues to be assessed by the ELA PSSA tests in the reporting categories of D –
the conventions of standard English as well as an e-text dependent analysis. The
actual test itself contains three sections and due to the different roles
of short-answer and text dependent analysis items the design for grade 3 –
differs from the design of grades 4 through 8. So you will see a separate
design for grade 3. Remember short answer items are used at grade three only and
TDA prompts are only used in grades four through eight. It’s also important to
remember that each grade level consists of nine different forms. The test items
are scrambled and field test items as well as equating items are embedded
within each form. Neither students nor test administrators may be made aware of
which test items are used for equating or field test purposes. It is critical to
ensure that the forms are statistically equal amongst administration years and
test forms which is the purpose of the psychometric items and the field test
items will also help to ensure that. Six core and field test conventions of
standard English items appear at the beginning of section one and at the
beginning of section three. That’s a frequent question that we will
get at PDE where particularly when the SAC is inquiring about the read aloud
accommodation. So again, there are only 12 questions that deal with conventions. Six
of those appear in section one – six of them appear in section three, and these
items are clearly identified in the student test booklet – as well as in the
student answer booklet. So there is no problem with the read aloud for those 12
items and there is no opportunity for the test administrator to violate any of
the security issues surrounding those items. There are seven passages, including
both fiction and nonfiction and they’re divided among the three sections. One
of the text-dependent analysis questions is a field test TDA
and one is the core TDA and the length of the text will vary across the
sections during an official test administration students are given as
much additional time as is necessary to complete the test questions. The item and scoring sampler is a useful
tool for Pennsylvania educators in preparing local instructional programs.
By providing samples of test item types and scored student responses the item
sampler is not designed to be used as a pretest, a curriculum, or other benchmark
for operational testing. The items and the samplers may be used as examples for
creating assessment items at the classroom- level. Classroom teachers may
find it beneficial to have students respond to the short answer question or
the TDA question in the samplers. Teachers can then use the sampler as a
guide to score responses either independently or together with
colleagues within a school or a district. Each question is accompanied by a chart
that contains the assessment anchor and eligible content coding,the answer key,
the depth of knowledge, and testing data. Each question is also followed by a
brief analysis or a rationale. The short answer question is displayed with an
item specific scoring guideline, or in the case of the writing prompts a mode
specific scoring guideline and examples of student responses with annotations at
each scoring level. The Pennsylvania core base PSSA may be administered in paper
and pencil format or online. As a result, the samplers include items of responses
in both formats. The text-dependent analysis prompt is displayed with an
item specific scoring guideline and examples of student responses with
scores and importantly – with annotations at each scoring level. If the response
was an online response that is indicated with a computer symbol.
Remember the writing prompts in the item and scoring samplers are provided only
as an instructional resource. Again, the 2019 ELA PSSA test will not include a
mode specific writing prompt, but it turned it over to my colleague Becca. Good morning, my name again Rebecca Baum-Lehman and I
work both in the division of curriculum and also assessment. In front of you – you
will see a picture of the scoring guidelines. The TDA prompt is a text
dependent analysis prompt based on a passage or passage set that each student
has read during the test event. There are three response pages in the paper and
pencil format and up to 5,000 characters. In the online format, both literature and
informational texts are addressed through this item type. Students must use
explicit and implicit evidence to make inferences leading to a conclusion or
generalization in response to the task. Stated in the prompt, students construct
a well-written analytical essay to communicate inferences and connection to
the evidence using grade appropriate writing skills. The TDA response is
scored using a holistic scoring guideline on a 1 to four-point scale. The
scoring guide laid guidelines can be located at the link that you’ll see in
the PowerPoint. The guidelines for the text-dependent analysis prompt remain
the same as in previous years. As Diane mentioned earlier, additional
instructional resources for text-dependent analysis work in the
classroom are located in the TDA toolkit. On the curriculum page on the PDE website
you can follow the breadcrumbs. PDE, teachers, and administrators, curriculum,
and then English language arts. The assessment anchors and glossary are
located on the same page because the glossary directly references the
assessment anchors. Notice for English language arts, we have compact assessment
anchor documents. The compact assessment anchors are in essence – a shorthand of
the assessment anchors and eligible content the documents are convenient
references to use when scoring or creating your own assessment items. In
the screenshot, you will notice the reference to the PSSA assessment anchors. The ELA tests no longer measures the
mode specific rating prompt in grades 3 to 8. The mode specific rating prompt
included the standalone narrative informative or opinion argumentative
prompt the assessment of the rating standards for the mode specific rating
has shifted to the local level. The text-dependent analysis essay continues
to demand strong rating skills. The average total test difficulty did not
change from previous years. The equating procedure used for the PSSA adjust
for the difference to the test blueprint making sure score meaning does not
change status of the text-dependent analysis TDA confirms Pennsylvania’s
commitment to the importance of writing in preparation for college and career
readiness. It is important to remember that the ELA test continues to assess
the writing and language standards. Many other writing standards are assessed in
the conventions of standard English multiple-choice items and the TDA prompt.
Now, I would like to introduce Craig Weller who will speak to you about
science. Thank you Becca and good morning
everybody. Let’s take a look here so we have the
PSSA science – this particular slide here has links, direct links to the science
test design, which I’ll go over with you in a few moments. The science items
and scoring samplers the items and scoring samplers are there from 2015
through 2018 and we have the scoring guidelines listed as a link there and
also the assessment anchors and glossary. Keep in mind that the glossary is really
not designed to be all-inclusive, it was simply to demonstrate the types of
terminology that is that are on the assessments and if we were to include
all of the things we would have a thousand pages of a glossary and we
don’t want that. So let’s move on here and look at the test design. So for grade
four we have two sections, of course science we have grade 4 23
multiple-choice with the field test items included. So you’ll notice on the
right hand side we have 38 estimated minutes for completion of that and this
is the second year we’re going to use the reduced test components that were
put in place last year – and that seemed like I went very well. So we have 38
minutes for section 138 for two and total 76 for your planning purposes and
the next slide shows grade 8. We also have the time on the right hand side of
45 minutes – 45 for a total of 90 minutes, and it seems like the time is that it
was estimated is pretty much right on for testing. The next slide is a
breakdown of the different types of items and of course, you see multiple
choice for a core psychometric use which aren’t scored field test items are not
scored and when you get over to the end we have 48 points for both the grade 4
and grade 8. If you’re looking at something I though I wanted to share
with you here if you’re breaking this down when you
on the PSSA – a test design link that I had shown you previously –
it’s breaking it down for grade for 38 and 38, 45 and 45 on the one page and if
you go to page one of three on the PSSA test
design sheet – it shows that there are 38 core points for multiple choice. Two
equating block six embedded field test for grade four, open-ended has five core
items which are scored and one field test embedded which is not scored. So a
total of 38 multiple-choice items and five open-ended items for grade four and
that’s where the 48 points are derived. Now, when you get to grade eight you’re
looking at 34 core multiple-choice items and you have four scenario-based items
and each one are worth one point a piece and we do have to equate a lot – and four
plus four scenario-based for embedded field tests which are not scored. So when
you look at it’s a little confusing just because you have the scenarios for grade
eight, but it comes out to 48 total points for that as well. I
wanted to remind you the breakdown we have the nature of science makes up 50%
of the assessments for grade four and eight category A the other are of course
we have biological sciences physical science and Earth and space science –
making up the others. So if you were to break it down points wise, we’re looking
at 24 points for nature of science for grade four and then eight points each
for each of the other three content areas of bio physical science and Earth
and space for the total of 48 and that’s a same across the grade four and grade
eight as well. So that has not changed and a reminder that PC is listed on
here because that’s the based on the Pennsylvania standards it’s not the PCs
as was previously posted on this this document this document has been revised
and re-posted to you website as a PAS because that’s what it is since 2002 and
for the next slide I guess we’re going back to Charlie. I can try to do this because they’re
just hot links anyway so for the Keystone exams what we’re looking at we
have the test designs up there for algebra, one bio and literature, and those
haven’t changed as far as from from last year. So those links the assessment acres
eligible content scoring samplers scoring guide lines and the SAS. This is Becca again, Diane and I wanted
to spend a few minutes on the literature. The Keystone literature we’ve heard in
the past that we talk a lot about PSSA and we sort of don’t talk about Keystone.
So we wanted to spend a few minutes speaking to the Keystone literature this
slide contains nine resources for the 2019 ELA Keystone exam. Each resource is
a live link that will take you to the appropriate page on the PDE website. If
the link does not open for you when you click on it, please switch browsers. You
may also go directly to the PA Department of Education website and use
the following breadcrumbs PDE click on data and reporting, click on assessment
and accountability, and click on Keystone exams. This will take you to the landing page
for Keystone exams it displays a brief description of the purpose of the annual
statewide test as well as links to resources regarding subject specific
materials reporting documents, testing accommodations, administration materials,
assessment anchors individual student report guides, performance level
descriptors, cut scores and the application process for educator
assessment committees. The literature Keystone exam consists of
two modules module 1 fiction literature module 2 nonfiction literature. The
literature Keystone exam measures the literature assessment anchors as defined
by the eligible content and is aligned to the concepts and competencies module
1. Fiction literature and module 2 nonfiction literature each consists of
17 core multiple choice items, three core, three point constructive response items,
for a total of 26 points with a combined total of 52 points. Approximately 65%
multiple-choice and 35% constructive response. This is Diane, the item and
scoring samplers contain released operational multiple choice and
constructed response items that have appeared on previously administered
Keystone exams. These items will not appear on any future Keystone exams.
Released items provide an idea of the types of items that have appeared on
operational exams and that will appear on future operational Keystone exams.
Each item has been through a rigorous review process to ensure alignment with
the assessment anchors and eligible content. The sampler includes items that
measure a variety of assessment anchor are eligible content statement
statements, but it does not include sample items for all of the assessment
anchor or eligible content statements. When we take a look at the
guidelines we see that there are the constructed response items are scored on
a zero through three point scale. This is an example of the general description.
Remember that every constructed response has an item specific zero through three
point scoring guideline associated with it. It is important to remind educators
that these students are scored on the content of the question. There is a
difference between the literature constructed response question and the
text dependent analysis question. Both of those prompts are asking the student to
analyze in the TDA. The student is scored on writing eligible content, but for the
literature keystone literature exam the student is only scored on content. In
other words, writing skills at this time are not measured on the Keystone
literature constructed response item. You may find the Keystone assessment
anchors and eligible content documents on the Keystone page located on the PDE
website. The assessment anchors an eligible content are designed to enable
educators to determine when they feel students are prepared to be successful.
In the Keystone exams, an evaluation of current course offerings through the
lens of what is assessed on those particular Keystone exams may provide an
opportunity for an alignment to ensure student preparedness. We have recently
added compact assessment anchors for the Keystone literature exam. The compact
assessment anchors are a shorthand of the assessment anchors an eligible
content and these documents are convenient references to use when
scoring or creating your own assessment items. At this time I’ll turn it back
over to Charlie. Thanks Diane. That concludes the webinar for today, however
I want to go back to the beginning for those of you who might have come in late
just so we can put up the what’s new part. The updates though what’s new. Remember the the primary test administrators initials have to be bubbled in on the
back of the student answer booklet and there’s a bubble if there’s multiple
test administrators in one room. Remember a test administrator and a proctor are
different. Proctor is there to be an extra set of eyes for the test
administrator, but if there are more than one adult in the testing room – one of
them is a test administrator – one of them’s a proctor, but we still get have
multiple test administrators administering a test session. So we need
the initials of the primary test administrator bubbled in and translators
and interpreters must complete the PSTAT stat for the TA’s. That’s what the
bubble part of the answer document looks like. If you have any questions you can
type them into the Q&A and we’ll answer them, but we’ll stop our recording now
with apologize again for the glitches we had getting started.
Thanks that the rest of you can make it. I appreciate that and any questions
again we’ll try to answer them in the Q&A. We’ll get this posted to our website as soon as we can. And hopefully the PowerPoint will be up
there as well, and thanks for coming.

Leave a Reply

Your email address will not be published. Required fields are marked *