Visible Learning for Literacy Practices That Best Accelerate Student Learning



Frank hello everybody thank you all for being here so we're going to spend some time talking about our newest work I'm talking on Nancy's computer right now and Nancy's chatting on mine and as we trade back and forth will trade laptops so we can always keep the chat going as you react and think and the discussion going as I talk in the Nancy duck this is the opening sentence for our new book every student deserves a great teacher not by chance but by design and we're wondering if some of you want to put in a phrase or one of the words from this sentence that really jumps out to you is it every student is it great teacher is it chance the design is it deserves what line what phrase jumped out to you and maybe a little bit about why so we'll get some chat going about how you think about this sentence that John Hattie and Nancy and I open the book with what part of all buns all we're seeing lots of chat very nice keep those comments coming you see so from this sentence we have to think about lots of things and you're all commenting on things like every student and what it means to be great that a lot of education is left to chance that if you're lucky enough to go to this third-grade classroom and not that third-grade classroom then you get a great education if you go to that school but not that school you get a great education and we're trying to make it so that kids are guaranteed an amazing school experience not because they're lucky and they got a great location but because we have designs that are intentional that we really have a really really good procedures in place to make learning happen and one of the things that we've been playing around with thinking about and talking with John Hattie about is his work on visible learning I'm going to do a really rapid review of how John has summarized massive amounts of data really big data at this point after about 15 years of collecting meta analyses he has about 300 million kids in his database so you're going to ask him a question about what works and he can tell you from this day that it's pretty amazing and the way he explains it is through this barometer of effect so he's trying to show visually what we know about some of the influences that teachers and communities and parents have on their children's learning for example some of the things are negative they have a negative impact on students of treatment that that um it's harmful some things are just because you got a year older some things are because there's some teacher of fact it probably was you know some some good done but maybe it's just individual variation of a teacher and where we want to work is in the zone of desired effects like this really has a evidence base behind it that it impacts learning in significant ways and what he found was it if you get the effect size at for and above then you get a year of growth or year of school and I had it vex eyes when I was in graduate school I learned how to calculate FX ice but I never knew how high the effect had to be to say it was worth the effort so we had to do two different things and one had an effect size of 0.6 and one is effect size of 0.3 and they cost you about the same in terms of resources I'm going to choose the one that has a greater effect but then we started think what got the year of impact that's where one of John's great contributions is that that year of growth four year of school is fantastic so we're looking a couple of these great level retention has a negative effect on student achievement how many staff meetings and conversations that we had about gray level retention we're spending a lot of time on something that really doesn't work ability grouping tracking doesn't really work we spend hours talking about so we have a low tracks with a high track what should we do it's not working to change the treatment and I've been part of school meeting faculty meetings where we'll spend the entire time on having conversations about regrouping by grade level and getting all the kids of a similar perceived ability in the same group and the summaries of evidence that's not really where you should be spending your time homework overall doesn't have a great effect Elementary homework less so high school homework more so but in general that's not where I'm going to spend all of my time it's thinking about how to do better homework what I know is that small group learning is better we get a year of growth for your school when you get kids together in small groups and you provide some instruction to them teaching kids how to study not how to take tests but how to study has a great impact on their learning getting kids to read things over again as a significant impact we're well over a year of growth now that that we can get kids growing and thinking thinking because we got them read it again they are working on a project right now that work tentatively titled who killed repeated reading and we've talked to lots of parents who and many of you have this experience you sent a child to school who loved to reread the same books over and over again and all of us know those four-year-olds and five-year-olds who love to reread text over and over again and then something happens where by about third and fourth grade they don't like to reread again so something we did in school made kids not want to reread we have to figure out what that is because we see repeated reading has a great impact on students achievements having a great relationship between students and teachers has a very positive impact on students learning how do we learn we learn in part from people that we like set another way students don't learn from grownups they don't like and we can get a lot of effect just by developing those positive pro-social relationships classroom discussion well over point four point eight to getting kids into classroom discussions where they have academic language and vocabulary they're having these amazing powerful conversations with each other with their teacher changes their experience we've been saying this for a while you don't get good at something you don't do and so if you're not using academic language on a regular basis you're not going to get very good at it that's where we want kids to work they want we want them in collaborative productive conversation and the last one I put on here there are 150 of them I just put a few to show you how we think about this teacher clarity knowing what you want kids to learn communicating what you want them to learn knowing what success looks like communicating success criteria it really impacts learning it's not just that people want to figure out ways to harass us and like oh I'm going to make you have an objective it's really powerful when students know what it is they're supposed to be learning why they're learning that and how they will know if they have learned it so what am i learning today and what's the success criteria that teacher clarity is very very powerful when it comes to impacting students learning so let's talk about what's new what how this contributes to John's amazing body of research that he's done visible learning for literacy is really about taking a slice of that amazing research that John has put together and looking more closely at the literacy aspect of that you know that literacy piece really transcends every part of the learning day and we felt that it was really an essential place for us to be able to start the conversation about how it is that visible learning gets applied in so many different classrooms in so many different ways literacy as all of you know is really at the at the heart of any kind of learning that happens even in other content areas because when we're talking about literacy we're really talking about reading writing speaking listening and viewing and all of those elements contribute to students learning when you think about what it is that happens in your classroom or grade level or school children and adolescents are always engaged in one or more of those domains reading writing speaking listening and viewing it really forms the heart of how learning occurs and so what we've done is we've taken the amazing work that John has done and have expanded it to really address issues around surface learning and deep learning and transfer of learning so I want to hang on this idea for just a moment because there there have been times we've talked about surface learning and people shy away from that they think that surface learning means superficial learning and that's not the case whenever any of us are learning anything new we're engaged in surface learning is how it is that you get an initial lay of the land so to speak you can begin to understand what some of the boundaries are what some of the important markers are as we gain some initial information some surface knowledge about what what a particular concept or idea or theme is then we can go deeper into it we can begin to consolidate that learning we can begin to understand how it is that that learning that knowledge might apply in different situations and here's where transfer really comes into play especially this idea of far transfer when we talk about smart transfer we're really talking about those takeaways there's a really important skills and concepts and ideas that we want students to be able to use and I had a math teacher one time tell me here's how I know when transfer of learning occurs when they do math outside of the classroom and I'm not there standing there saying do math it's that transfer of learning your ability to be able to utilize that knowledge in novel situations to recognize similarities and differences that's really what we're teaching for and let's not fool ourselves not everything that we teach in class is strictly for transfer of learning what we do have to build a foundation of the kind of knowledge that students need if they're ever going to go deeper and then finally to be able to transfer some of that information now the idea of surface learning and deep learning is certainly not a new one at all but this is one of the classic definitions from some of the early educational psychology researchers our found surface learning and deep learning and I think it's classic enough that it bears repeating here when when any of us are engaged in surface level learning what we're learning how to do is to reproduce information to recall to recognize to reproduce information or as they say that we begin to learn the signs of that knowledge for example and I'll use a I'll use a really simple example of this when children are learning how to read one of the things that they learn is the alphabet they learn what those signs are but as they go deeper they begin to understand what those signs actually signify for those of you that have owned your own personal children you know that very often young children children who are who are not yet anywhere close to being readers can begin to recite the alphabet they'll sing the alphabet song to you in other words they begin to recognize the signs of the alphabet very early on at the age of three or four or so on but it's only typically after they've entered formal schooling that they begin to understand that those signs really signify something that there are sounds that go along with it that you put those sounds together in different ways and that those different sounds and letters together form a word that's a simple example of surface learning versus deep learning and I think that none of you would would disagree that we don't need to spend time with young children teaching them the alphabet for example but we wouldn't stop the learning there we need to go deeper all the time and if you consider any of the content that you teach you begin and introduce it to them by making sure that they understand at the surface level of learning what the signs are what the features are of that knowledge and then you begin to go deep are into them so we spend and we've organized in the book chapters that are specifically about surface learning as well as chapters that are specifically about deep learning surface learning is absolutely important we can't state strongly enough that surface learning is not superficial learning it's your entry point it's the place in which you begin to know some things about that unit of instruction and there are ways in which we can facilitate surface learning now this is a major feature of the book that that John and I have and Doug have done and that is in understanding that when we have our students engaged in surface learning that we match up the kinds of approaches that really contribute to surface learning so using that effect size knowledge and what are those things that that exceed the hinge point they include things at the surface level of learning like really being strategic about leveraging what your students prior knowledge is engage in any variety of vocab three techniques remember that surface learning in part is understanding the signs the language the markers of that unit of instruction that you're doing at the surface level of learning this is where you all see reading comprehension skills and approaches that are being taught in context when children are learning how to use that reading comprehension we make sure that we present it to them in context in other words not away from text but actually within text we also make sure that at the surface level of learning that we have students engaged in lots of wide reading of the topic under study there's been a tremendous amount of research on on this you know that someone's knowledge of a topic is going to predict their ability to be able to comprehend written text about that topic and again think about what it is that you do if you're supporting students in a classroom and they're learning about turtles or they're learning about Edgar Allen Poe or any other topic that you can think of you have them engaged in lots of different pieces of text that are going to build their background knowledge about that and then as well making sure that they have lots of opportunities to summarize to summarize what it is that they know to restate that whether in through oral language or through written language these are ways in which we can facilitate that surface learning and engage in that initial acquisition of knowledge will give you just a couple of examples of ways in which surface level knowledge can be facilitated for instance many of you recognize the use of prayer cards word cards using a Frayer model that include the word itself that is understanding this isn't an example when the students in in English did the word is sonnet over on the top left hand side you ask students to write a definition in their own word so it's not copying the definition but rather doing it in their own words on the bottom right hand corner in that quadrant we ask them to think about an opposite or a not not every word not every term has an actual opposite to it but a sonnet is not an open verse sonnets have 14 lines there's a rhyme scheme that goes along with it and so in this case the student used her knowledge of what you knew about poems to end poetic structures to be able to talk about what is isn't and then importantly she drew an image that would help her to recall all of that that's a form of a mnemonic there are lots of different kinds of mnemonics Peg mnemonics and so on and an image mnemonics are numb or number sorry are another way of being able to accomplish that remember that the surface level of learning we're asking students were helping them to recall to recognize to reproduce information as students move through that surface level of knowledge we move them closer and closer to consolidation of knowledge so initially its acquisition and then we move them to consolidation I'll show you another example of that this is called vocabulary stepping-stone done and I've used this many times in our classes and it's useful we find especially whenever we're going to have students watching for example a film clip or they're going to be watching a demonstration especially when there's that visual element and what we ask them to do is to listen for vocabulary key vocabulary that's associated with that content and put it in order as they hear it after for example they've watched a film clip and they've put those in order usually with a partner then what they do is they retell that information using that vocabulary in order as a reminder for them about what it is that they need to talk about these can also be done or a written summary as well well what we're asking them to do is to really begin to a consolidator thinking consolidates that knowledge it's a great formative evaluation for us to a formative assessment because we can listen in on what students are talking about or we can ask them to write it as an exit ticket so simple way of doing this if you if you're doing a film clip for example is to just play that several times write down the vocabulary that you consider to be the key vocabulary and by the way this is a perfect time to not just pay attention to the those Tier three words those those big glamorous terms but to also be paying attention perhaps to some of the more unusual unusual adjectives or verbs that are being used so that you can expand their repertoire in their language that they use as well listen to it a few times we type them up on paper cut them up put them in an envelope and as they listen to and watch that video they as I said they put them in order and they keep going deep learning is also super important getting kids into that deep learning is value as Nancy said earlier let's not limit it to surface learning that surface learning is important let's not skip it deep learning is important that we have a whole bunch of ways we put four of them up there as examples there are ways to really push people's students deeper thinking and their learning and we've gotten some pushback on concept mapping and people said oh you can use content maps at the beginning and for surface or for other things but when we really look at it if you know some stuff but your content maps are really strong and what we worry about is when content maps are given prematurely students end up doing a lot of copying of the teachers content maps you have to know something to have a discussion about it you have to been thinking about it or pondering or reading you move to metacognition you're thinking about your thinking and we know reciprocal teaching super-powerful very high effect sizes it's about helping kids learn about information by talking to each other as they go deeper and deeper and deeper and deeper so much of what it is that students are doing as they're deepening their learning really has to do with the kinds of discussions questioning that they're engaging in you're asking as a teacher you're asking them questions that at this phase of learning is meant to really prompt to their thinking and that's where the issue of funneling questions versus focusing questions comes into play and again there's a there's a good body of research on the comparisons and contrasts between funneling questions which can limit thinking and focusing questions which can help to expand their thinking so we put just two examples up here to compare and contrast the two it's easy enough to ask a student what was the setting of the story and you can imagine the limited reply that you're going to get it's a funnel in question you're funneling them into what you believe is going to be the correct answer on the other hand a focusing question shows them where to look but doesn't tell them what to think how did the setting influence the story asks students to really expand their thinking as opposed to simply recalling when worth a deepened deepening level of learning we want to make sure that we're actually deepening their thinking and their learning another example what's the meaning of the word confusing is a funneling kind of question on the other hand why do you believe the author chose the word confusing in this passage opens up their thinking it expands their thinking deep learning approaches don't work any better at developing surface level learning than surface learning strategies work to develop deep learning and that was an aha moment as we were working on this project is wow we have a we have believed or thought that all literacy strategies work all the time for everything for all kids we have to get to this to the point of what works and when does it work that's what we've been pushing on in this new project that leads us to the idea of transfer that neglected cept around getting kids to take what they've been learning and move it and use it apply it in other situations that might be two weeks later it might be next year it might be you learn something in language arts and you're applying it in your social studies we really want to get to transfer we're not the first talk about transfer grant and Jay had it in their 2011 book that they had some ideas around transfer it's hard to get learning to transfer we recognize that there there is some some work we have to do to get kids to take ownership and have responsibility for that learning I'm show you two examples from our school we have had school-wide conversations about transfer goals what do we want kids to accomplish from the learning we provide them across all of their classes as an example this first one is from speaking and listening I'll give you a second just to read that and when you love to be part of a community where people who kids who graduate knew how to do these things this would be a great outcome and these are we we research this week we investigated and our staff our colleagues went through this and said here's what all ninth grade teachers commit to that doesn't matter if you're in science or history or math and English and art it's all about that that belief of what's going to happen yes they learn the content of science yes they learn the content of history but we're going to make sure they do this and here's someone writing take a minute look at that see there's good things when teachers get together and start saying what do we want kids to transfer what's our goal for transfer and then how do we facilitate that transfer what are all the ways instructionally that we can impact learning and maybe we've spent too much time talking about the teaching side and we need to spend more time on the learning side here are ways that we can transfer learning across time across space across classes across years and these are good things it takes a lot to know how to read across document and organize something conceptually dance is going to tell you more about that these more formal discussions like formal debates and Socratic seminars take a lot of application problem-solving teachers teaching works really good for transfer doesn't work very well for surface if John were on this call right now he'd say problem solving teaching problem-based learning is it completely ineffective at surface level learning but works well to get kids to transfer so here's an example we've done work with students here at our school around the Sherman Alexie book and this is a commonly used book in high school the absolutely true diary of a part-time Indian a great book if you haven't read it so far but what we want to do is we want to deepen and then really facilitate that transfer of learning so it really became about how can we construct opportunities for students to engage in multiple pieces of text and to read across those texts and read against those texts as well as with them so after we read the book the part-time Indian book and had conversation about that then we looked at in read and discussed one of his short stories that he had written about 20 years earlier called every little hurricane for those of you that haven't read it it is a nearly autobiographical account of an incident that happened in his childhood so gave our students a different lens in being able to understand the book that they had just finished we also viewed a video interview with the author called I can only run a marathon sober where he talked about his struggles with substance abuse and so once again after we talked about that piece of text video text now we began to understand at a different level both the book and the short story the final piece of text that we read is a pump called elegy for a forgotten oldsmobile and the author identified this as being the poem that changed his view of who he could be I'll let you read the paragraph from an article in the Atlantic that we read together we then read the poem and we paid special attention to that very first line of the poem that the author had identified as being so transformative for himself that in turn spurred new conversation different conversation about what we had taken away from the book from the short story and from the video that ability to be able to read across documents really help students to transfer information outside of the classroom to begin to apply in lots of different venues what we're really focused on is in figuring out the right approach at the right time for the right type of learning and that means that we need to know lots about our students and be able to recognize what phase of learning they're in surface learning deep learning or transfer and the hard part about all of this is you might have students in the same class who are ready for different things so again it's the design of learning experience and what impacts we have on students that's going to make the difference but it took us a while we've heard of surface and deep for the last couple decades but we had to really get better as saying what works to develop surface what works to develop deep and what do we do for transfer there's a big conversation about rigor Anansie they've been talking a lot about rigor because everyone uses the word rigor but no one can give us a definition of rigor so we want to propose that rigor is a combination or a balance between difficulty and complexity how much work how much effort how much time and how much thinking how much knowledge you need it takes both and that's where I think we've made some mistakes as a profession most of the time rigor simply means more effort more time or more work we want to make sure that we're also thinking about complexity here's a sample item released item from a big test us to measure common core and I would argue this item has virtually no complexity some students will find it difficult depending on how old they are and what their language proficiency is but it never got more difficult so it only got more difficult it never got more complex there's another item also released and I would argue this one has more complexity but isn't very difficult if you know how to estimate and you know proportion you're going to get this right and we were thinking about this and we were wondering if our society is super enamored when people do hard thing look at this data from 2007 she memorized the exact order of 59 decks of cards over 3,000 cards in order that's impressive hard I'm not sure why you'd want to do that but society gets really enamored with hard tasks and we have to teach our our parents and our communities to also be impressed with complex tasks then working on this grid how we think about it some tasks that we give students are low difficulty low complexity there it's not that they are bad they're just not difficult and not complex in fact we think all four of these quadrants are super important and they help us think about surface deep and transfer in the low low quadrant we develop students fluency their procedural fluency their conceptual fluency in the high low quadrant we develop stamina and there's a lot of kids who don't yet have a lot of stamina we need lessons tasks assignments that build their stamina we also want to build strategic thinking how do we get kids to know when to deploy which kinds of things can also be metacognitive in nature but really being strategic and lastly we want to develop their expertise students need to be in that quadrant of expertise but if they don't have a lot of surf and acknowledged it is hard it is nearly impossible to engage in highly difficult highly complex tasks so all four of these are important and they help us plan lessons for kids to learn more for example copying notes note-taking is not unimportant it's just a low difficult low complexity task as having a time to reading tap build your stamina wide reading build your stamina studying those notes you took from a book or from a lecture is more strategic and then analyzing argument across text is more around expertise and we can go on there's all kinds of different tasks that would fit in each of these the idea is that we help teachers plan lessons where they know which times they want kids in the fluency quadrant or the expertise quadrant or the strategic thinking quadrant or the stamina quadrant it's important in considering how it is that we support students with differing ability levels through this and and what we found over the years and what we've recognized in our in our own practice is that we misunderstood differentiation as being something that was related to complexity and what we've learned is that when we differentiate for students we want to make sure that we're differentiating according to the difficulty but not to the complexity and this can be a real issue for students especially soon too who come with paperwork of one kind or another that states how it is that they need to be supported and not losing the proper level of complexity that you have noted in that tap are you going for building fluency or is it stamina building or are you building their strategic thinking or are you building their expertise should be differentiated according to the difficulty but not losing the complexity along the way the last version of the book is about looking at impact and for those of you that have followed John's work for years you know that when he talks about visible learning he's talking about making learning visible to teachers and making learning visible to students we have to understand what our impact is on student learning and it isn't sufficient to simply say by looking at a summative test an end of unit test well here's what it is that I taught students because a certain number of them scored at the a level and the B level and so on we really have to think more closely about what our impact is by pre assessing and then post assessing what it is that they've done now there's a statistical formula that goes along with that does started out with talking about effect size with 300 million students but you can figure out what effect size is for your own classroom or even for an individual using very similar mathematical formula now I'm going to show you what the formula is and here's what I want you to understand you can perform this on a Excel spreadsheet simply by plugging in what the scores are pre assessment and post assessment it will do all the calculation for you when you're looking at determining your effect size for a group of students you take their post assessment average and subtract their pre assessment average and you divide by the standard deviation now again a spreadsheet will do all of this for you but I just want you to see what the math is that goes behind it in terms of what it is that you're actually doing you're setting up a spreadsheet and you're putting in two columns that have the pre and the post scores that are in there and it will do this for you you can also do this for an individual and by the way this is a great way to be able to monitor in terms of RTI in terms of MT SS making sure that you're seeing progress and that it's not due to lots of other variables we're looking at our impact very similar only now you're looking at individual scores post and pre assessment rather than a group score as well so don't be afraid to be able to measure your impact because you can do it what we want to do though is we want to be able to see what our impact is on their learning and not simply to assume that they learn something because they were in our presence we have to get we have to get to a level where we can note what it is that they're doing so give you an example from here at the school where where Doug and I are there were a group of teachers they were working on their writing for argument unit they they figured out that the standard deviation for the pre assessment what they did at the beginning of the argument unit and did a calculation they averaged it they came up with an effect size and it was above the threshold of point four zero so what the teachers were able to understand was that they had a positive effect on what students were learning now there are other times where you don't have a positive effect where it is below that point four zero hinge point that Doug talked about early on in the presentation and when that's the case that's when you adjust your teaching and you go back and you do some reteaching something different either for the individual or for the group I'll show you an example of what this looked like a list of students and the pretest and the post-test and they were able to generate that sighs what they found in their particular case and this is for I want to be real clear consistent this isn't that meta-analysis research for this group of teachers what they found was that their effect size on this unit of instruction with this group of students was at the point seven seven level which is for them definitely noted that they were above that point four zero level now give you another example when it didn't work two teachers two colleagues of ours were working on students public speaking skills they had some of data had some great lessons but they did a pre pre assessment as well as a post assessment and in their particular case they found the effect size was only point three zero and that was when the teacher said when we were talking to them they don't seem to be getting much better at this they weren't actually seeing as much improvement as they fought perhaps they were witnessing so what they did and that was but that was below that hinge point so what they did is they adjusted the unit isn't this what all good teachers do they adjusted the unit and in their particular case they taught students about how to be able to perform those peer critiques and they were watching short videos of one another that had been posted to be able to craft those critiques as well as a number of other adjustments that they made to the unit as well they met again a few weeks later talked about that effect size again and this time what they saw was it had jumped up to 0.75 except for a couple of students again meeting that that hinge-point test of exceeding the point four zero level and I don't want to forget about those three students but being able to pin and be more precise about who those students are so that you can do the kind of further development that you need to do with them is absolutely essential the need for a pretest is something that we talked about in in teacher school I work in teacher school I always teach my students about pre assessment but in practice it's not something that is done as commonly and I think what happens sometimes is that people think oh pre assessment it has to be this big elaborate kind of thing or they're worried that their students are going to get kind of freaked out about the whole thing so a few a few points about that you can do a pre assessment that only includes a handful of items that you're going to use on that post assessment it does not have to be a full-blown hour-long pre assessment but what you're comparing or how it is that they performed on those items before instruction and then after instruction another thing to is make sure that your students know why it is that you're doing that pre assessment students understand when you say to them this is what helps me as a teacher this is not about a grade I need to understand some things about what it is that you already know as well as things that you don't know that makes me a better teacher for you we also want to make sure that in having those discussions about impact that there are lots of opportunities for teachers to collaborate with one another and again whether it's at the grade level or it's at the department level these are great ways for teachers to be able to come around and say here's what it is that my students know right now here's what it is that I want them to know here's how it was that they did and now let's analyze what worked and what didn't work we also want to look for increased opportunities for true differentiate especially to be able to talk about how it is that we can differentiate for students based on difficulty and not on the complexity what we saw among teachers as they collaborated and learned more about impact was that we saw an increase in the amount of small group instruction that was going on because they got a lot more precise about figuring out who were the kids that were making progress and who the kids that weren't and they got much better at being able to use their time wisely in order to be able to leverage instruction with the students who needed more support they were also more willing to abandon lessons when students had already demonstrated mastery of it one of the things that pre assessments can do is they can really help you figure out how not to waste time we always hear teachers all of us talking about how we wish we had more time to be able to instruct well one of the ways we can recapture some of that is by doing some pre assessment and knowing which lessons perhaps we don't need to do or which lessons that we only need to do with a smaller group of students as opposed to the entire class now I am going to apologize because Doug stepped out he's finally had enough of a coughing fit that he's he's just out there coughing right now so it's just me on the line right now I don't know whether he'll be back or not but what I wanted to do is to turn this back over to you I cannot see the chat on my desktop but I'm wondering if perhaps David or Margaret if you post any questions to me verbally I can respond to those as well so if there are themes or questions that are showing up in the in the chat box maybe I can help with that I imagine one of the questions is how do you do that calculation on Excel it's easier to go ahead and google how to do that there there are countless numbers of videos as well as instructions in there that show you how to put that formula in there so I I won't consume time with that but David or Margaret if you have questions say that you want to kind of relate to me I'm happy to respond to those I'm double checking right now me and see if anyone has any questions they can ask in the Q&A and then we'll go ahead and ask them from there's a lot of people so from Scott Williamson will limbs on does the effect size chart of characteristics exists in the book yes absolutely yeah so those are those are in there Michelle Jung asked understanding that yet is calculated on standardized tests what tests were used to measure public speaking skills what the teachers did was use the rubric that they had developed for the public speaking skills remember that what a rubric always ends up translating into is a quantitative information and so they were able to plug those scores in there there was teacher developed their public speaking rubric that they use for 9th and 10th grade gene Fowler asked can you explain more about the difference in complexity and difficulties yes absolutely i when we talk about difficulty and this is this really comes from the assessment world when we talk about difficulty we're talking about the amount of effort that is needed in order to be able to complete a particular task and what what assessment folks are always looking for is making sure that they don't confound the amount of effort that's involved with the amount of complexity that's needed for that in terms of measuring effort on a test what they look at and many of you are familiar with you know piloting items on large scale tests what it is that they're looking for is to figure out how many people are actually able to complete that accurately and that gives them a measure of the difficulty of it difficulty is not the same as complexity in terms of complexity and I'll draw in Norman Webb's work here with depth of knowledge it's really about the number of steps cognitively that you need to engage in in order to be able to complete that task so if you're familiar with Webb's work with V okay you know that it goes from level one to love four and all of those have to do with the number of cognitive steps that are needed in order to complete that tap norman is also the first person to tell you that level one tasks are not inferior to level four tasks which is why we built that that visual of complexity and difficulty you need all four of those you don't need just level four and I get concerned sometimes when I go to schools and I hear administrators say while all the tasks that the kids should be doing should be level three and level four tasks think about in that grid that we showed you level 3 and level 4 would be equivalent to strategic thinking as well as expertise you can't stay just in those two quadrants and expect that students are going to learn any new information you have to also spend time in those lower quadrants level 1 level 2 if you will building fluency building stamina that's how it is that we end up acquiring new information and then deepening that knowledge as well so I hope that helps a bit great great answer we have time for a few more questions Joanna asks how many items are needed on a pre and post-test to determine a valid effect size well I die that's a great question I actually don't know how many are needed in terms of you know John would answer very differently of course because what he's looking at are very large scale studies where we're talking about meeting those statistical tool requirements that are needed when we're talking about being right here in our classroom paying attention to what it is that students who are right in front of us are doing we're typically only looking at a handful of items four or five or six items great Cheryl wants to know if your book focuses on elementary and secondary equally if it is we did write it as a k-12 so we use lots of different scenarios from all over all over the mouth at the middle school in high school level is English specifically that we're looking at in elementary all of those scenarios and the videos that go along with it and by the way there there will be videos we're in the process of developing those right now those videos come either from elementary classrooms or they come from middle and high school English classrooms awesome we have time for I think three more questions and we'll try to get them as many as possible katie is asking kim students play a role in helping the teacher determine what kind of learning they need how can we gradually release some of this responsibility deep surface transfer as you know I thank you so much I love that question you know when when when John talks about visible learning he talks about it not just in terms of being visible to teachers in other words understanding your impact but he also talks about him we didn't really address it in this webinar he talks about students understanding what the learning intentions are what the success criteria is and then setting their own goals and if you're familiar with the work that he's done by far far and away at the absolute highest level is that impact of students being able to set their own goals and then work toward those goals it makes a huge difference in terms of student learning about whether they are simply doing something because of compliance they're doing it because they like you or are they doing it because they understand that this contributes to their learning and so engaging in lots of conversations with students around reflective thinking causing them to engage in reflection causing them to set goals and then to measure their goals for example setting out the purpose for a lesson at the beginning of a lesson here's what you're going to learn about today here's what you're going to do with it and then at the end of the lesson let's revisit that your goal today was around these particular goals how to do how did you do how do you self reflect how do you evaluate when you get the learner in on those conversations you can move forward so much more quickly because now they understand what their goal is it's not just the goal that you have for them it's the goal that they have for themselves great I think we have two more questions and from Lori one is do you address students with learning disabilities in your book I imagine that they're in there if it's hard for me to kind of separate that out in the sense that for all of our classrooms for all of us we have students with paperwork and without paperwork we do have an entire chapter in the book that's around RTI and MTFs and what that looks like for students who identify in one way or another as struggling with their learning whether they have paperwork or whether they don't have paperwork awesome I'm trying to find a good question to end on do you use the exact same assessment or address the same skills with different passages to get accurate results Oh for the pre assessment in the post assessment so so whenever whenever we advise people to do that it's really more of a unit level it's it's not something that we're doing every day by by any stretch of the imagination this is really more looking at the unit level of instruction but yes so the items would be the same and as you know it's an essential teacher practice that you know going into a unit what it is that you want to get out I am NOT down with the the teacher who says oh well you know I don't know what my intestine is going to be because I I want to see what it is that I teach and what I learn what they learn first you go into a unit already knowing what it is that your assessment is going to be and when you know what that summative assessment is going to be you can easily pull just a couple of items out of there in order to be able to get a sense of what it is that they know and to give you a baseline so you can figure out what your impact is by the end of the unit great thank you so much for answering those questions Nancy and thank you to dogs as well everyone else who questions were not answered I'll be making a list of that and I'll be sending that over to Doug and Nancy so they can take a look thank you so much for joining us thank you have a great night everyone

2 thoughts on “Visible Learning for Literacy Practices That Best Accelerate Student Learning”

Leave a Reply

Your email address will not be published. Required fields are marked *