Future Ready PA Index – January 2016 Webinar


Now starting all attendees are in listen only
mode. Ok Good Afternoon I’m Mathew Stem, Deputy Secretary of Elementary and Secondary Education
at the department of education and I’m pleased pleased to welcome you and thank you for joining us
for today’s webinar most of you on this webinar webinar have heard about the departments efforts to
establish the future Ready PA Index and our goal today is to share some additional insights
on our progress to date were gonna begin with talking about the purpose of the Future Ready
PA Index then we’ll discuss context and background including somewhat of a detailed timeline
that we’ll walk through at the beginning then we’ll be talking about the building blocks
of the Future Ready PA Index with some highlights some highlights of what were endeavoring to
accomplish and then really what we will be spending the majority of our time is discussing
some of the proposed indicators in detail and what are the definitions and what are
we considering in that process and finally we’ll highlight our next steps moving forward
so as you hear more about on the upcoming slides this work really began with a guiding
vision and that was to establish a system of school performance measures that moves
beyond point in time achievement and values schools efforts to help all students learn,
grow, and succeed in the classroom and beyond. A focus on multiple pathways to college and
career readiness has been an underlined theme in all of our conversations and so from a
process standpoint looking at where were at today the development of indicators was really
the second step to follow the establishment of our guiding vision and please be aware
that the work around methodology is ongoing as we explore the most effective calculations
and reporting systems to be a part of the future Ready PA Index that methodology will
be a part of future communications as we look to the weeks and months ahead before we dive
into the indicators were that were considering as part of the purposed future Ready PA Index
we want to take a few minutes to step back and consider so historical context for this
work. Pennsylvania’s current tool for reporting school performance the PA school performance
profile or SPP was launched in 2013 to fulfill requirements established under act 82 of 2012
the indicators currently used in the SPP are established in statue regulation and they
reflect most heavily on singular point in time measures while the SPP was developed
to fulfill Act 82’s requirements to calculate building level school or as to factor in educator
evaluation the SPP also provides information to parents, families and other community members
about school quality and has become the most visible source of information about how schools
are performing when governor Wolf took office in 2015 he directs Secretary Rivera and the
department of education with re examining our reporting system and exploring more valid
comprehensive and meaningful measures of school performance the department began by holding
more than 30 review sessions during the 2015-16 school year on possible revisions to the current
SPP in the process we gathered input from more than a thousand diverse stakeholders
across Pennsylvania regarding the proposed indicators formulas and waiting several revisions
occurred throughout the process based on feedback that we received and throughout these discussions
with stakeholders there really was a consensus that emerged and there was a strong desire
for more holistic indicators to determine the ethnicity of schools across the commonwealth
while the department was at work defining the proposed measures and seeking feedback
from diverse stakeholders congress passed the every student succeeds act or ESSA which
replaced the no child left behind as a nations prevailing education law ESSA’s emphasis on
broader accountability measures and more tailored evidence based school improvement strategies
aligned with the types of measures that the department was contemplating in the revisions
to the existing SPP the department launched phase 1 of it’s ESSA stakeholder and engagement
efforts in April of 2016 more than 250 stakeholders convened for a general information and feedback
session and 4 stakeholder workgroups were charged with identifying opportunities and
recommendations in 4 key areas accountability which ties into our conversations today as
well as assessment educator preparation and educator evaluation in August of 2016 the
work groups concluded their work identifying framework recommendations that were documented
and contextualized in a report compiled by a 3rd party research partner at the American
institute of research this fall PDE kicked off phase 2 of ESSA stakeholder engagement
hosting 60 events throughout the commonwealth with more than 500 in attendance to date the
feedback ideas concerns and insights shared with the department over the past 18 months
inform the work and in December the Secretary announced this new proposed tool the Future
Ready PA Index at the 2016 SAS institute the new name reflects a commitment to ensure the
success of our next generation of citizens in Pennsylvania
so moving from the over arching goals to the strategy level there is several things were
endeavoring to accomplish in this new tool first we are looking to increase the emphasis
on student growth measures which should incentivize the focus on all learners and is less sensitive
to demographic variables so we’ve heard a lot from stakeholders in our travels that
growth allows for all learners to have their achievement over the coarse of a year better
communicated to the public also were looking to measure English language acquisition among
ELL students, not simply performance on a test of grade level ELA standards 3rd were
looking at incentivize career awareness instruction beginning at the elementary level were also
looking at addressing the issue of unequal weighing of content areas in the current SPP
so in the current SPP English language arts is weighted twice what Math and Science are
weighted and that’s one of the issues that and one of our goals in this is to address
that unequal weighting of content area were looking to provide indicators to students
success after graduation so we’ve heard loud and clear from parents from business owners
from community member’s from elected officials that a schools job is not done on the day
the student walks across the stage with a diploma but the true reflection of a schools
success in many ways is what a student is doing and accomplishing after they graduate
high school and were looking to increase the emphasis along those lines on student access
to course offerings that prepare them for that post secondary success and that includes
AP, IB, College credit baring courses such as new enrollment and also really emphasize
an CTE programs of study as a significant and important pathway for our students were
also looking to provide additional snapshots of students success academically beyond the
PSSA’s so were looking to include locally selected reading assessments and locally selected
Math assessments as additional snapshots of student progress and we’ll certainly talk
about that shortly we want to incentivize schools to offer a career pathways that
culminate with high value industry recognized credentials so many of our high schools not
your CTE’s but traditional high schools are offering industry recognized credentials and
things such as in coding Microsoft certifications job of programming’s c++ alike among others
and that’s work that we believe should be recognized and we would like to see those
sorts of activities increase among our high schools and additionally we want to recognize
schools for decreasing the percent of students scoring at the below basic level and we know
that we have schools that are moving students out of that bottom core tile and it’s important
to communicate that out to the public so that their aware of the progress being made by
all students there are 3 sections in the proposed design of the Future Ready PA Index the first
is a section is state assessment measure 2nd is on-Track Measures 3rd is college and career
readiness measures we’ll take a closer look at the proposed indicators beginning with
state assessment measures they are percent proficient and advanced on PSSA/Keystone Exams
the percentage of students meeting annual academic growth expectations (PVAAS) and a
look at students with disabilities meeting annual academic growth expectations for each
of these indicators were going to address definitions and for some we’ll discuss additional
considerations particularity signaling those indicators that are going to require some
new calculations so beginning with percent proficient and advanced on PSSA and Keystone
exams the inclusion of this indicator is not a change and reflects the importance of
demonstrating grade level proficiency so were still looking at all students that are scoring
proficient or advanced on each of the PSSA and Keystone exams we would still be including
all students who take the PSSA or Keystone exams or the PASA and were enrolled for a
full academic year the PSSA would still apply to students in grades 3-8 and Keystone exam
scores would continue to reflect students best score to date for all 11th grade students
enrolled in the school for the full academic year and as a reminder going back just one
bullet the PSSA exam in grades 3-8 this is now going to be the 3rd year of the new PSSA
exam which we put in place in 2015 and is certainly more rigorous than the prior PSSA
as it’s in line to our PA core standards Meeting Annual Academic Growth Expectations
(PVAAS) measure represents the academic growth of the group of students taking the state
assessments as measured by the change in their overall achievement during the reporting year
and as a reminder an academic progress measure is required as part of ESSA accountability
rules and this would be a measure that could satisfy that academic progress requirement
additionally significant stakeholder feedback as I mentioned before has reflected that this
is a high priority for emphasizing student growth in our system and the majority of our
stakeholders from all sectors have reported that this is a more meaningful measure than
just purely the percentage of students who are proficient or advanced engaging the success
of their schools and moving student learning that and a couple of other slides here to
answer some questions that have come up specifically around the new PSSA exam that we were just
discussing and one of the questions that we’ve received many times is around the correlation
of poverty and PVAAS scores and particularly as the correlation relates to the new PSSA
so what you have on your screen in front of you is a scatter plot of the 2016 math scores
across all grade levels and each of those dots represents the percentage of students
that were economically disadvantaged in the schools so the x axis represents poverty levels
from 0-100 and then the y axis represents the growth index scores and what you’ll see
here is a relatively flat distribution of scores in math relative to the percentage
of students who are economically disadvantaged in the chart that you have in front of you
now is English, language arts PSSA and again this is all grade 3-8 and you’ll see that
this distribution is much the same as it was on the other as we had seen in math so the
distribution is relatively flat regardless of poverty levels at the school level as a
side note while you are looking at the grades 3-8 English language arts we are aware that
the distributions on PSSA science as well as on the keystone exams is not as evenly
distributed as they are on PSSA in English Language Arts an Math though these correlations
now over a number of years have consistently flattened overtime
the students with disabilities is meeting annual academic growth expectations is a new indicator
and this represents the academic growth of the group of students taking the state assessments
as measured by the change in their over all achievement during the reporting year and
this has been revised overtime and so there’s always been a focus on how do we ensure that students with disabilities
are making appropriate progress in school and this is probably the 3rd iteration
of what that indicator could look like and so as we move forward this ensures that schools
are focusing on all learners and that our students with IEP’s are making progress commencer
with their peers the next section of the Future Ready Index
is our on track measures and they include a grade 3 reading indicator of success grade
7 mathematics indicators of success English language proficiency attendance and closing the
achievement gap the percentage of students meeting the indicators of success on a locally
selected assessment in Grade 3 reading allows allows for schools to report a snapshot of academic
progress in addition to the PSSA score and from the departments level we would not anticipate
that this would be a new assessment for schools in fact it shouldn’t be a new assessment for
schools but will recognizing the fact that almost universally our elementary schools
are engaging in other assessments of reading progress and that those assessments are providing
meaningful information to teachers and schools regarding student progress and we believe
that those are indicators that it would be very valuable to also share those with the
public in our system so this would represent the percentage of students who demonstrate
on track performance measure locally selected assessment providing additional data point
as we mentioned includes all students who take the locally selected assessment
and were enrolled in a full academic year and this would be discretionary at the LEA
level to determine whether or not to report this measure so an operationalize indicator we
know that PDE must define criteria by which LEA’s can ensure that the locally selected
assessments measures students pathway on the track to success we also must facilitate trainings
to the field to ensure understanding of the criteria and the appropriate ways to evaluate
a locally selected assessment for this purpose along the same lines were proposing that their
be a measure of students success on another locally selected assessment in grades 7 math and grade 7 math is one of those key transition points for students where this really is a
time to ensure that they have the conceptual math skills as well as the computational math
skills to engage in the higher level math courses that will follow in high school and
that explains the targeting of grade 7 and all the other descriptors are the same as
we described with the grade 3 English Language Arts meaning that it would be a locally selected
assessment and LEA’s would determine whether or not to report this measure now we do know
that this is an area that is not as universally operational as grade 3 English language arts
another words there are a number of schools that don’t have assessments that their currently
giving for additional snapshots on math for whether it the 7th grade level or just the
middle years in general but we do believe that this is a practice that results in increased
precision around how were setting up an instructional programs for students and if a school is engaging
in assessing student progress at the 7th grade level using an alternate assessment this is
another area that we think should be communicated to stakeholders around those snapshots of
achievement the English language proficiency indicator is another indicator that is required
under ESSA its one of the 5 required indicators under ESSA is a measure of English Language
acquisition and this is another area in which we in Pennsylvania are ahead of the curve
in exploring our teams and stakeholder groups have been looking into this measure many,
many months before ESSA had materialized and what were looking at here is really the percentage
of students who are enrolled in a full academic year and are on track to English language
proficiency and we would be using the ACCESS test measures which are currently available
for all identified English Language Learners (ELLs) and again were not talking about a
new assessment and in fact one of the things we didn’t mention are one of the beliefs that
under guards our work is the future ready index is not a place where were introducing
new statewide assessment that’s not the purpose and it’s not a belief that drives our development
and so the access in the assessment already been taken by the English language learners
and while specific calculations are still being explored for how we would translate
access scores to a measure this indicator really recognizes the importance of not just
holding English language learners to grade level expectations and standards but really
more importantly to development of their English Language Skills based on when they entered
a school or when they entered the country given the body of evidence on school attendance
and it’s correlation with achievement attendance remains a a critically important indicator
and there are two definitions were reviewing that are going to impact our eventual methodology
and calculations the first is very similar or really identical to the way we currently
measure attendance on the school performance profile and that is essentially average daily
membership so the value represented for the reported years the number of aggregate
days of student attendance divided by the aggregate days school was in session for those
students and from a reporting stand point this is always trailing indicator based on
the availability of data so it’s always would be reporting the prior years data the alternate
definition under consideration is chronic absenteeism and this is a definition and different
way of looking at attendance that better satisfy federal accountability requirements so in
chronic absenteeism it’s typically measured as the percentage of students missing a certain
number of school days regardless of the reason and can be calculated as the percentage of
students missing a defined percentage of school days or a defined number of school days all
that would be determined while we are considering both of these definitions and in particular
the second we know that there would have to be controls for things like extended illness’s
and other excused absences that extend over a period of time so were capturing the accurate
data for students at risk for the appropriate reporting reasons
closing the achievement gap is actually 2 indicators that we have reflected on one slide
here both we will be using the same formula in some capacity one indicator represents
closing the achievement gap for all students and then the second indicator represents the
historically underperforming student group which includes not all English language learners,
economically disadvantaged students and students with disabilities but also racial groups as
part of that sub group disaggregation and closing the achievement gap in simple terms
will be looking at growth in overall proficiency over an established number of years now this
is another one of those indicators that although at face value is vary similar or would appear
to be the same as the close in the achievement gap measure in the current school performance
profile one of the things that we know needs to be considered in the methodology around
this indicator is how we control for the sensitivity of small targets particularly among high proficiency
school in other words right now in our current SPP if a school has 95 percent of students
that are proficient or advanced in a given year the way were currently calculating closing the achievement gap makes it very difficult for those schools to get any points or it becomes
sort of an all or nothing indicator because the gap is so small and that’s weather that
gap is for the all group or for the historically underperforming student group so we would we are contemplating ways that this can be calculated that control for that and do not and does
not penalize schools that have proficiencies that are already well above 90 percent on
a given content area the 3rd category on the Future Ready PA Index is college and career
measures and include standards benchmark industry standards based competency assessments or
industry recognized credentials AP/IB College Courses/career pathways and graduation rate
and an indicator around postsecondary transition to school, military or work the career standards
benchmark is a new indicator and this is one if there was any indicator that we received
an overwhelming and consistent amount of positive feedback it’s been around a career standards
benchmark and Pennsylvania is a state that does have an established set of standards
know as the career education and work standards and these are skills such as collaboration
communication teamwork time management, problem solving and also specific activity such as
creating resumes writing letters to employers so we have the standards in place in Pennsylvania
but nowhere have we ever measured or reported on them in any kind of public way and as such
it can be a disincentive for schools to invest there resources in these critically important
standards which particularly our business and industry partners have communicated to
us are lacking in many of our current high school graduates and so this indicator would
reflect the schools efforts in these areas and many of our schools are doing great work
in this space and many of our schools still have a great deal of work to do to build capacity
in these areas but at a surface level we would be looking at the percentage of grade 5 students
who demonstrate the acquisition of skills related to the career education and work standards
through engagement in career exploration and preparation activities then at grade 8 we
would be looking at students who create individualized career plans and finally by grade 11 students
who complete career portfolios during that reporting year. However we operationalize
this we know that this is going to require an efficient process for demonstrating evidence
at each of these three levels remember the statement I’ve shared before that the department
as a core belief does not propose that we introduce new assessments to measure these skills we
could certainly do that and there are assessments out there that can measure some of this work
but really what we want to do is to be able to operationalize a process and efficient
process for schools to demonstrate the evidence that can be reported out and we also want
to ensure that anything that we do relative to this indicator should be aligned to any
potential changes in graduation requirements that could be forth coming in the months ahead.
The indicator of advance placement and IB college course offerings and career pathways
really is an indicator that acknowledges the importance of course offerings aligned to
college in career success and in ensuring that students are matriculating into those
programs now the methodology around this indicator will likely be more complex than some of the
other indicators if not most of the other indicators there’s several different factors
that we would want to consider in this measure so the first being just an input measure of
course offerings and a percentage of AP/IB or College course offerings college credit
bearing school offers so that’s one input the others are reflections of either the percent
of students enrolled in those courses or even the percentage increase in students enrolled
compared to the prior years so we want to have a system that’s sensitive to there maybe
a district that doesn’t add new AP courses but is able to significantly increase the
number of students that are taking the courses that are being offered and so in methodology
we’ve got to account for that and then finally and vary important looking at the percentage
of students enrolled in a career and technical education program areas of concentration who
complete all of there secondary-level competencies identified as part of approved CTE programs
we acknowledge and recognize that we do not have enough students in Pennsylvania that
are matriculating into our CTE programs which have very high levels of either job placement
or post secondary pathways and in reality there coming from our CTC programs and this
is a place to recognize schools and districts that are ensuring that they are having the
right conversations with students to direct them into the pathways that best meet their
interests and aptitude the Graduation Rate calculation or the Graduation Rate indicator
is much the same as in the current school performance profile representing the percentage
of students who graduate here were looking at 6 or fewer years as opposed to the current
formulas which look at the 4 year cohort graduation and the reason for looking at it this way
is that we have a number of students in our schools who are extending beyond 4 years in
there high school program and many of whom for example are students without IEP’s who
are entitled these would be there IEP to attend school until the age of 21 and those students
are currently not captured in our graduation reporting and we believe that every student
matters and that every student and school should be recognized when a student graduates
whether it’s 4 years 5 years or 6 years so that’s one thing of which to be aware here
and then obviously the graduation rate is only as meaningful as the graduation requirements
that we have in place and so PDE really looks forward to collaborating with the general
assembly on that good work and we hope to have those opportunities again in the weeks
and months ahead and finally the last indicator is the post secondary transition to school,
military or work and this is another indicator that’s partially reflected in ESSA reporting
requirements as well as in recent changes in IDEA regulations but more importantly this
reflects belief systems amongst not only the department and our partners here in Harrisburg
but among parents, business and industry members in our communities that really believe that
our work as a school is not to simply get students to graduate but to ensure that their
skills are reflected in what it is they do after they graduate and so we’d be looking
in this one indicator of numbers and percentages of students that are enrolled into an institute
of higher education within16 months of receiving a diploma or students whom enlisted in the
military within 16 months of receiving a high school diploma or entering the workforce in
Pennsylvania within 16 months of receiving their diploma and this is another area that
would require access to various data sets and that’s work that is ongoing for us some
of these data sets are easier than others to be able to utilize with a reliable correlation
so that work is on going as we explore that and obviously this is an indicator that may
not be applicable for all purposes such as whether it be act 82 or federal accountability
purposes so as we look at next steps there are several items to signal one being and
we want to make sure that we appropriately highlight this on November 28th the used promulgated
the final ESSA accountability regulations and there were some substantive changes in
the final regulations the draft regulations that have been communicated for many many
months one of the most significant is around a methodology and many states are now moving
toward a dashboard approach to reporting at instead of a single summative score now to
date our work around these indicators has been with the notion of a summative score
continuing to be reported but we will continue to do our due diligence around exploring a
dashboard approach as an option giving national trends and aligning some of those with our
core values and beliefs the other the other item of note one of the other items of note
is possible revisions to proposed indicators and so as we continue to engage our stakeholders
and collaborate with our partners we certainly won’t rule out any revisions to the indicators
themselves so the indicators that we’ve discussed there still could be further revisions as
we continue our stakeholder input and collaboration but we were comfortable being this far down
the road in at least signaling where the indicators stand at this time and 3rd this notion around
sub group calculations you know one of the strongest requirements of ESSA is that every
indicator that’s used for federal accountability purposes needs to be disaggregated by sub
group and so right now we are contemplating in our methodology how we would account for
sub group disaggregation for each of the indicators that we just discussed an what implications
that would have on our calculations and reporting methodology we also are continuing our stakeholder
engagement as mentioned before were in phase II of getting around the state we’ve our team
has been in Pittsburgh, Erie, Lock Haven, Philadelphia, up in northern Bucks county
and I think were still on our way to Scranton and also we have other target groups that
were still continuing to work with including parents, students and others and certainly
continuing to collaborate very closely with the general assembly and so that engagements
going to continue we will be submitting our ESSA plan in September and our goal is to
launch the future ready PA Index in the fall of 2018 using the 2017-18 data sets so we
believe that when were successful in this endeavor we will have established a system
that incentivizes students enter practices that fully prepare them for success after
graduation and were thrilled at this opportunity that we have and look forward to continuing
to partner as we move forward on this last slide there is an email address to which any
questions or suggestions can be sent we carved out a new address specifically for the future
ready PA Index so I know things can get a little complicated because of the overlap
with ESSA and we have a separate email for ESSA but this email address can be shared
with any stakeholders who have specific questions and suggestions or thoughts regarding the
Future Ready Index PA and how we’re moving forward I thank everyone for making the time
in busy schedules to be a part of todays webinar and we look forward to staying in
close communication in the weeks ahead as we get more precise around landing on file
indicators and the associated calculations and methodology. so thank you for your time
and have a wonderful afternoon.

Leave a Reply

Your email address will not be published. Required fields are marked *