Objective: The Students Will Examine and Explain

The learner will demonstrate -- or TLWD.

It's the statement and acronym typically used to clarify and create learning goals.   This introductory statement was originally used with Bloom's Taxonomy to identify clearly in which cognitive category students were expected to demonstrate their learning - e.g. The learner will demonstrate knowledge, comprehension, application, analysis, synthesis, or evaluation.  When Anderson and Krathwohl (2001) revised Bloom's Taxonomy by renaming the cognitive categories from noun to verbs, the introductory statement became The student will be able to... followed by the newly named cognitive category - remember, understand, apply, analyze, evaluate, or create .  The push for student-centered objectives written in student friendly language once again changed the introductory statement for learning goals to be more direct and personal (I will...) or collaborative (We will...).  

However, with the instructional shift focusing on college and career readiness, it's time to once again rephrase the introductory phrase that set the learning goals for a lesson or unit.  Why?  Because learning is not only about demonstrating knowledge and thinking anymore.  Students are now also expected to communicate the depth and extent of their knowledge, understanding, and awareness of what they have learned.  In other words, learning by doing is no longer the goal.  Now students must be able to explain how it is done, express why it can be done, and expound upon what else can be done with the concepts and content they are learning.

Interestingly, for the most part, the college and career ready standards as they are written and presented do not foster and promote communication of knowledge and thinking.  While there are some performance objectives that begin with cognitive verbs that are synonymous with communication, such as define, describe, explain, present, represent, summarize, or write,  the majority of the cognitive verbs introducing the standards are more more intrinsic and cerebral than extrinsic and communicative.  Performance objective direct students to demonstrate how they can to analyze, apply, determine, evaluate, integrate, or interpret, but they neither inform nor guide students how to express and share their analyses, applications, determinations, evaluations, integrations, or interpretations.

This is why questions, not performance objectives, are an effective and integral means for demonstrating and communicating learning.  They prompt students to think about what they are about to learn.  They also encourage students to express and share the depth of their learning.

So where do we come up with these questions?  We rephrase the same performance objectives of academic standards as good questions that foster communication of learning using oral, written, creative, or technical expression.

How can we rephrase these performance objectives into questions?  We use the introductory statement The students will examine and explain and convert the cognitive verb of the standard into a question stem.  

The verb examine challenges and engages students to think deeply about what they are learning.  The verb explain prompts and encourages students to express and share the depth or extent of their learning.  These are the cognitive processes that not only address college and career readiness but also foster and promote cognitive rigor -- specifically, the demonstration of higher order thinking and communication of depth of knowledge.  

Now look at what happens when these performance objectives are rephrased as good questions.   They not only foster and promote demonstrating and communicating learning but also increase the cognitive rigor of the learning experience by having students think deeply and express and share the depth of their knowledge, understanding, and awareness of how, why, what influence, and how can you apply.
cess can be made simple by using by taking the following steps:

1. Identify the standard(s) that will be addressed.

1. Use the introductory statement The students will examine and explain...

1. Convert the cognitive verb to the correlating cognitive rigor question (C.R.Q.) stem using the Bloom's Revised Taxonomy Inverted Pyramid. (See the accompanying graphic).

1. Complete the question with the concept or content addressed in the standard.

These good questions not only serve as summative assessments but also set the instructional focus throughout a learning experience.  The phrases and words are the academic vocabulary, subject-specific terminology, and specific details and elements students will need to recognize and understand who, what, where, or when in order to address and respond to these questions and meet these performance objectives with the depth and extent they expect.

Turning performance objectives may seem easy and simple, but is actually difficult and complex - or rather, complicated.  It will take time and thinking to develop a good question that is so open-ended and thought-provoking that they will drive and determine the depth and extent of learning.  However, this pro

Use the formula for creating good questions from academic standards:

The students will examine and explain + C.R.Q. stem + subject / topic

Watch your students demonstrate and communicate deeper learning!

Erik M. Francis, M.Ed., M.S., is the lead professional education specialist and owner of Maverik Education LLC, providing professional development and consultation on teaching and learning for cognitive rigor. His book Now THAT'S a Good Question! How to Promote Cognitive Rigor Through Classroom Questioning will be published by ASCD in Winter 2016. For more information on this topic or how to receive professional development at your site, please visit www.maverikeducation.com.

Start the Year with Good Questions!

The summer is over, and the first day of school has come in many parts of the country.  It's time to for all us teachers and our students to head back to the classroom for a new year and for a deeper teaching and learning experience.

So how are you going to start and set that deeper teaching and learning experience that experience that first week?

Let's be realistic.  That first week of school is about getting-to-know-you.  You are getting to know your students academically by giving them pre-tests and placement tests. You are getting to know them personally by having them share who they are or what they did this summer using some form of oral, written creative, or technical expression. Your students are also getting to know you by understanding what your expectations are and the procedures and rules in your classroom for behavior and turning in work.

However, along with on getting to know each other and going over routines, what if you asked these good questions?

• What is the relationship between reading, writing, speaking and listening, and language?
• What is math?
• How does science explain our world and ourselves?
• What is history and whose is it?
• Why is it important to learn both the language and culture of a foreign country or society?
• What qualifies as art?
• What is music?
• What is fitness and health?

What if you spent that first day in class having them engage in a dialogue about what they think is the meaning or intent of these academic areas and subjects they are going to learn this year?  How could this serve as a pre-assessment for background knowledge and previous learning?  How could their responses provide you some insight into their their opinions, perspectives, and thoughts about these academic areas?

Instead of assigning an essay that asks who they are or what they did this summer or even completing a personal information sheet, what if you have them compose or create an academic autobiography in which they explain and express the following in a single or multi-paragraph essay:

• What kind of student are you?
• How strong do you think or how successful have you been in this particular academic area?
• What has been your greatest accomplishment or your fondest memory in learning this subject?
• What would encourage you to continue enjoying learning this subject or to enjoy learning this subject more than you have in the past?
• How could learning this subject continue to be or become a better experience for you?

Think about how much information and insight you would obtain about your students!  You will not only know and understand from where they are coming but also what kind of students they are (or perceive themselves to be) and how you could address their needs academically and even socioemotionally.  Have them write their Language Artsography, their Mathography, their Scienceography, their Historyography, or whatever subject they are currently learning and you are currently teaching them.  

If you're going to give a final exam as part of your class, what if you told your students what the questions will be on the final or even have them take the exam during the first week of school?  I learned this technique from Ken Blanchard, author of the One-Minute Manager book series who would give his exam on the first day of class.  It not only informed my students of what would be expected of them but also set the instructional focus for the entire semester.

As you return to your classrooms this year, shift your instructional delivery and focus.  Instead of spending that first week telling students what they need to know, understand, do, and what is expected of them, ask good questions to stimulate their thinking about what they are about to learn.  Then introduce the subjects and topics to which they will be address and responding.  Watch the learning environment shift from one that focuses on teaching and telling to learning through inquiry and interest. 

Let me know how your students react and respond, and have a great first day!

- E.M.F

Are We Teaching for Cognition or Compliance?

When my daughter was in 5th Grade, she brought home a worksheet that featured a series of multiplication problems she had to solve using this graphic.

I am an educator.  My experience has primarily been in teaching English and language arts at the middle and high school level.  However, I did a stint of middle school mathematics during my first year of teaching due to the school needing a math teacher and since I was the last hired ... well, you know how that goes.  While I don't consider myself to be a "math person", I have become skilled and proficient in mathematics mostly out of necessity and my own frustration with the concept.  However, I had never seen this graphic before.

When I showed this to my wife, who is an elementary teacher, she informed me that this is "one of those methods Khan Academy uses".  I went on Khan Academy's website and sure enough, there it was - the Lattice Box, the name of this foreign graphic my daughter brought home and was required to use to multiply multi-digit numbers.  I studied how to use the method and found it to be highly confusing, which surprised me because I am a visual learner but could not figure out how the pieces of this cube fit together.

I asked my daughter if she could just multiply the numbers using more conventional and traditional methods, and she said, "No, I have to do multiplication this way!"

I said, "Wait a minute.  What's more important here - that you understand what is multiplication and how and why it can be used to determine amount or that you use this method?"

She said, "Daddy, this is the way I have to do multiplication now."

That made this assignment much clearer to me - and also why there's such misconception and misinformation about this notion of "Common Core Math" and "Common Core English".

Interestingly, a week later after working on this assignment with my daughter (which we completed using the Lattice Box even though we both disliked the method), this story and the accompanying image was making the rounds on social media.

Perhaps some of you may remember this story about the father whose rant about the "Common Core Math" homework their child brought home became the viral sensation and caught the attention of magazines such as Time Magazine and even a featured spot on an episode of THE GLENN BECK SHOW.  The misinterpreted message became, "This is now how our children need to do math!"  No more could our children just borrow the one, regroup, perform long division, or use algorithmic formulas.  This was now the "New Math" our children needed to learn.

I became curious about what exactly what was this method students were now "required" to learn and use to subtract numbers, and I discovered through research and investigation that this graphic or tool featured in the homework assignment is  the empty number line, which is a model for addition and subtraction by researchers from the Netherlands in the 1980s.  I read about this methodology, how it was developed, and how it should be used, and it was fascinating.  I took one of my wife's elementary level math texts and worksheets and experimented with this methodology.  Sure enough, I attained the same differences I did when I subtracted using the more conventional and traditional method I was taught.  Still, I was fascinated to learn about the history and development of this tool and experience how it could be used.

After this experience, I decided to do some research and investigation into Lattice multiplication, and I found that Asian and European cultures have been using this strategy to multiply numbers as far back as the 13th Century.  I learned the history and development of the Lattice box and experimented with how it could be used to multiply multi-digit numbers.  Personally, I found the method to be confusing and even cumbersome, and I would not choose to use it.  However, again, I was fascinated by its history and development.

Both my daughter's homework and the Facebook father's viral posting piqued my interest to learn about the different methodologies that can be used not only to perform the four operations of arithmetic - addition, subtraction, multiplication, and division - but also help deepen conceptual and procedural understanding about mathematical practices, principles, and processes.  I learned that with math there is usually only one possible answer (which is what I already knew), but I discovered so many different methods and ways that the correct answer can be achieved and attained.  I did not find all of these methods and strategies and methods to be helpful or even useful, but I did enjoy experimenting with these practices and procedures and deciding which method would work best for me.

This experience reminded me of how I much I disliked not only instructing but also evaluating my students on how effectively they can use specific methodologies and strategies in English language arts.  I disliked teaching and grading my students on how to diagram sentences because I did not like using the method.

I also disliked requiring my students to use the Jane Schaeffer Writing Method because that was the writing program and process the district and school where I was working adopted.  While some of my students benefit from these methods, the majority of them did not, and I found myself frustrated teaching them not only because my students were struggling but I also struggled to use these methods and strategies that did not meet my learning style.  It actually caused me to have one of the worst teaching experiences in my career - and this was in an AP English Language class!  These kids hated writing style analyses for me not because they were frustrated in determining the tone and effect of the craft, structure, and language of the texts they were reading but because they had to use THE JANE SCHAEFER METHOD!  In fact, that became the name of the pain they felt from writing.

This is where teaching becomes a miserable experience for both the students and the teacher - when we are FORCED to use a prescriptive method, strategy, or technique to answer questions, address problems, and accomplish tasks.  We have to use THIS READING PROGRAM because that's what the school adopted.  We have to use THIS MATHEMATICAL PROCEDURE because that's what the district or charter has decided to use.  While these methods, strategies, and techniques are proven effective, they're not for everybody, and the mere mention of their names can cause both kids and adults to cringe.  Try it.  Mention Singapore Math or whole language and see the reaction and response you get.

Now we have a new name for our pain in education: Common Core.  "Common Core Math", "Common Core Reading", and "Common Core Writing" - the very name strike dread and disgust in the hears of many!  They are characterized to be as evile and vile as Darth Vader, The Joker, Freddy Krueger, or Hannibal Lecter.  Their names are spoked with a sneer or a tone of disdain and even fear and hatred  The parents and even the kids have said, "Can't we just use the old way to do math or read text?" and they are being told, "NO!  This is the way you must now read, write, and do math!"

According to whom?

I have reviewed the questions on the PARCC and SBAC exams extensively, and I have yet to see anything that resembles the practices and procedures provided in the textbooks, presented on the worksheets, or featured on Khan Academy.  I would presume this is not the intention of the textbook publishers or even Saul Khan.  However, the message has been grossly misinterpreted to state, "This is NOW HOW you must demonstrate and communicate your learning!"

So my question to consider is this - when we teach, are we teaching for cognition or compliance?  

Are we requiring students to answer questions correctly by following directions as explicitly and prescriptively as they are taught or are we encouraging them to think deeply and express and share how and why they can achieve and attain correct answers, desired outcomes, or specific results in different ways?

Are we challenging students students to think deeply and express and share how they could transfer and use what they are learning to answer a question, address a problem, or accomplish a task, or are we directing them to understand there is only one way to answer a question, solve a problem, and complete a task, and you must do it this certain way just as you were taught?

Perhaps you're saying, "But the curriculum features all these different methods and strategies the students must use to demonstrate and communicate deeper knowledge and thinking." True, but what if we approached these different methodologies, strategies, and techniques not as a mandated assignment they must complete but rather as a hands-on learning experience in which they experiment with using these practices and processes and decide whether they want to use this particular method or another strategy?  Would they not only become familiar with these different processes but also realize there may be more than one way to answer a question, address a problem, and accomplish a task and they have the freedom to choose the method that would work best given the circumstances, the context, or even their own personal preference?

If you truly want to understand how our children are expected to learn math and why they should experiment with these different methodologies, I recommend you watch this video by Dr. Raj Shah, who explains perfectly how we should teach math for cognition, not compliance: https://www.youtube.com/watch?v=X_CK1e0Lmxw

I ask you educators to consider when you send your students home with those homework sheets that feature the Lattice Box or the empty number line or require them to identify the parts of a sentence through diagramming or write an essay using the Jane Schaeffer writing method, are you teaching for cognition or compliance?

Erik M. Francis, M.Ed., M.S., is the lead professional education specialist and owner of Maverik Education LLC, providing professional development and consultation on teaching and learning for cognitive rigor. His book Now THAT'S a Good Question! How to Promote Cognitive Rigor Through Classroom Questioning will be published by ASCD in February 2016. For more information on this topic or how to receive professional development at your site, please visit www.maverikeducation.com.

How Authentic Are Your Assessments?

As our schools transition from implementing to evaluating instruction that addresses the cognitive rigor of college and career ready standards such as the Common Core State Standards, many states have decided to grant schools and students a reprieve from mandates that measure and monitor overall school performance based upon the results of the new state summative assessments such as the PARCC or SBAC.  Some state education agencies and charter school authority boards are allowing schools to use the site-based assessments they have implemented at their schools from testing corporations such as ATI-Galileo, NWEA, Acuity, and MAP for reporting student performance and progress.

However, how authentic are these assessments?  Are they expecting students to answer questions correctly based upon how effectively they can remember, understand, and use what they have learned or are they engaging students to express and share how they would use the education and experience - or expertise - they have acquired and developed to address and respond to the question?

Authentic assessments resemble reading and writing in the real world and in school (Hiebert, Valencia & Afflerbach, 1994; Wiggins, 1993).  These assessments generally challenge and engage students to demonstrate and communicate their deeper knowledge, understanding, and awareness using oral, written, creative, or technical expression. They also prompt students read, review, and respond to texts or comment upon and critique the ideas, incidents, individuals, and issues they are learning supported by relevant and sufficient evidence and valid reasoning.  They also encourage students to share and show what can they do or produce with the deeper and broader knowledge and thinking they have acquired and developed.

In other words, they resemble how students will address and respond to circumstances and situations not only academically but also personally, professionally, and socially throughout and beyond their formal K-12 education.  Think about it.  How are we "tested" in life or the depth and extent of our knowledge and thinking evaluated?  The only time we would be given a multiple choice test is for certification or licensure or unless we go on a game show like Who Wants to Be a Millionaire? (see my other blog entry on Let's Make a D.O.K.!) Our knowledge and thinking is "tested" and evaluated based upon how correctly, clearly, comprehensively, and creatively we can communicate our claims, conclusions, and contentions.

That's how we should be authentically assessing our students - based upon how their ability to communicate the knowledge and thinking they have acquired and developed through their education and experiences.

In education, authentic assessments are typically used in active learning experiences such as project-based and problem-based learning that prompt and encourage students to create, do, or produce something - a plan, a product, or a project - that reflects and represents how deeply and extensively they have learned the subjects and topics they are studying.  The assessment is generally based upon the quality of the project produced or the success of the student to come up with a solution.

However, even though project design and problem solving are active and authentic learning experiences, are the final products and solutions students produce truly or authentically mark and measure the level and depth of their learning?

Don't get me wrong!  Project-based and problem-based learning are excellent instructional methods and strategies that not only challenge but also engage students to demonstrate and communicate their learning in-depth, insightfully, and in their own unique way.  However, when it comes to assessment, most PBL experiences merely scratch the surface, focusing on what can you create, do, or produce without delving into how and why you created, did, or produced the project or solution.

That is true authentic assessment - evaluating not only how correctly but also clearly, comprehensively, and even creatively can students communicate their knowledge and thinking using oral, written, creative, or technical expression.

 It's how we are expected to answer questions, address problems, and accomplish tasks in the real world.  Think about it.  In our professional and personal lives, when we answer a question, address a problem, or accomplish a task, we're not just expected to "just do it".  We're also expected to delve deeper by expressing and sharing how and why we answered, addressed the problem, and and accomplished the task.  We need to defend, explain, justify, and support our actions and decisions.  We are also encouraged to pass on our education and experiences - or expertise - to others.

Authentic assessment is about communication and expression, not just activity, production, and design.   The quality of the response is determined based upon the following:

• Did the student answer the question, address the problem, or accomplish the task correctly, clearly, comprehensively, and even creatively?
• Did the student express and share their claims, conclusions, and contentions in-depth, in detail, insightfully, and inimitably?
• Did the student strengthen and support their responses with textual evidence, personal experience, recorded observations, or scientifically-based research?

So how does this translate into the classroom?  We need to move away from tests that use multiple choice and provide assessments that utilize open-ended questions that provide students the opportunity to express and share the depth and extent of their learning.  We need to refocus our evaluation of student learning from determining whether students can answer the question correctly to whether students to defend and support their response to what the question is addressing using their education and experience - or expertise - as evidentiary support.  

However, this does not involve having students demonstrate and communicate their learning primarily through PBL experiences.  We can convert the performance objectives of college and career ready standards into good overarching and topical essential questions they can address and respond using the texts and topics they are reading and reviewing as evidentiary support.

Good overarching questions are the inquiries students will examine and explore throughout and beyond their K-12 education.  They address the core ideas and enduring understandings of an academic area, discipline, or field of study.  These core ideas are addressed in the disciplinary core standards of the college and career ready standards.  We can use the performance objectives to develop good overarching questions that can serve as the formative and summative assessments for a grade level or subject area.

Take a look at the questions in the accompanying graphic.  These are derived directly from the ELA / Literacy CCSS Anchor Standards for reading.  These are the good questions students will examine and explore throughout and beyond their K-12 experience with reading.  Consider how these good questions can act as the final assessment at a particular grade level or subject area.  They can also serve as benchmark assessments that progressively measure and monitor how deeply and extensively students have learned these concepts and content throughout their K-12 education.   Think about it.  What if students were asked these same questions at the end of every school year starting in Kindergarten and through 12th grade and used the texts and topics they read and review in class that particular as their evidentiary support for their responses?  How could this serve as a true measure of how deeply and extensively students have learned these disciplinary core ideas in a particular subject area?

The performance objectives for grade level academic would serve as the topical essential question for a particular unit or lesson. Take a look at the good questions that are derived from the performance objectives of the following math standards for a 3rd grade unit on multiplication and division. The cluster serves as the topical essential question that sets the instructional focus and serves as the summative assessment for the unit.  The performance objectives listed under the cluster serve as the daily good question that sets the instructional focus and serves as the summative assessment for individual lessons or learning experiences.  The problems students will be presented as part of the unit will serve as the textual evidence that strengthens and supports their responses.  We can provide a learning experience that challenges and prompts students to address and respond to one of these questions and use the problems they are presented to examine and solve as their evidence.  Look at the accompanying graphic that would drive a lesson on understanding and applying the Pythagorean Theorem.  The question students need to address and respond is the one in green at the top of the graphic.  The problems they need to examine and solve will serve as their examples and evidence that will strengthen and support their response to the question at the header of the slide.  However, I would advise not having the students examine and solve all these problems in one setting.  Ask them to address and respond to the good topical question and pick one or two problems to examine and solve to support their response.  The next day, ask them to pick two or three more of these math problems and explain how they can be solved using the Pythagorean Theorem.  At the end of the unit, present that topical essential question as the assessment or "test" question and have them pick the one problem they left remaining to strengthen and support their response.  It's practically a given that the one problem students' chose not to solve is the one they perceived to be the "hardest" one.  Think about how much you would build a student's confidence if they were successfully able to express and share how they could use the math to solve that problem they perceived to be so hard.  Also, consider how you would be able to assess their learning authentically by observing how deeply and successfully they can express and share how they can "use the math" - or rather, think mathematically. 

In English language arts, the overarching essential questions are the grade level performance objectives of the academic standards that will set the instructional focus and serve as the summative assessment for the course.  The topical questions will directly address the text or topic being read in reviewed in class.  Look at the topical essential questions for this book study on Charlotte's Web by E.B. White.  These questions authentically assess how deeply and extensively students understand the ideas and information presented in the text by challenging them to express and share what they have learned using oral, written, creative, or technical expression.  They will use specific evidence from the text to strengthen and support their responses. Similarly, these topical essential questions for a unit on Shakespearean tragic hero also set the instructional focus and serve as the summative authentic assessment that measures and monitors how deeply and extensively the students understand the texts and topics they are reading and reviewing.

To create authentic assessments that measure and monitor the depth and extent of students' understanding of the core ideas of an academic area, discipline, or field of study, look at the performance objectives of the disciplinary anchor standards and practices for a particular subject area.  You can use the ELA/Literacy CCSS Anchor Standards for Reading, the CCSS Standards for Mathematical Practices, the Crosscutting Concepts of the Next Generation Science Standards, or the Historical Thinking Standards, or the conceptual standards of the C3 State Standards for Social Studies.  Convert those performance objectives into good questions that ask students to address and respond how or why.

To create authentic assessments that assess and evaluate deeper and extensive knowledge and thinking about a particular text or topic, look at the performance objectives of the grade level or subject area academic standards that students will address as part of a lesson or unit.  Other than the English language arts college and career ready standards, these performance objectives generally challenge and engage students to demonstrate and communicate deeper and extensive content understanding.  The topical essential question will serve as the single question students will continuously examine and explain over the course of the unit.  It will also be the single question which students will need to address and respond at the end of the unit.  They will need to address and respond to this question by demonstrating and communicating their learning by processing the information they have acquired and gathered into their personal or self-knowledge and use specific evidence from the texts and topics they are reading, reviewing, and responding to as support.

That's authentic learning - expressing and sharing depth and extent of knowledge and thinking supported by examples and evidence - and that's what it means to assess learning authentically. 

Erik M. Francis, M.Ed., M.S., is the lead professional education specialist and owner of Maverik Education LLC, providing professional development and consultation on teaching and learning for cognitive rigor. His book Now THAT'S a Good Question! How to Promote Cognitive Rigor Through Classroom Questioning will be published by ASCD in February 2016. For more information on this topic or how to receive professional development at your site, please visit www.maverikeducation.com.

Burn This! Why the D.O.K. Wheel Does NOT Address Depth of Knowledge

How many of you have seen this graphic?

Perhaps you were provided this graphic from your district or school to use as a frame of reference for planning instruction and assessment for depth of knowledge.

Perhaps this is the image that popped up when you conducted an online search about depth of knowledge.

Perhaps some of you are using this to develop lessons and units that you believe prompt and encourage students to demonstrate their depth of knowledge about the concepts and content they are learning.

Well, stop it!  Do not use this wheel!  In fact, BURN THIS if not physically then out of your memory because this is NOT depth of knowledge.  It's actually a (very poor) graphic for demonstrating higher order thinking.

What's the difference?

Higher order thinking correlates to the kind of knowledge and type of thinking that needs to be demonstrated in order to answer a question, address a problem, or accomplish a task.  When we  plan instruction and and assessment for higher order thinking, we educators typically mark and measure the level of thinking students are to demonstrate using Bloom's Taxonomy - specifically, the revised version by Anderson and Krathwohl.

Depth of knowledge is an entirely different means of measuring and monitoring rigorous learning.  It correlates more to how extensively students are to express and share their knowledge and thinking.  In other words, are students expected to express and share depth of factual and conceptual knowledge (What is the information that needs to be known and understood?), procedural knowledge ( How can the information be used to answer questions, address problems, and accomplish tasks accurately and appropriately?), strategic knowledge (Why can the information be used to produced a correct answer, desired outcome, or specific result?), and extended knowledge (What else can be done with the information and how else could it be used?).  Webb's Depth-of-Knowledge model is typically used to designate the extent students are to express and share their learning.

Unfortunately, depth of knowledge has been misinterpreted and incorrectly perceived as being similar to higher order thinking, and much of that misconception can be contributed to the D.O.K. Wheel.  If you look at the different pies in the wheel, you will notice that it categorizes depth of knowledge by the actions students will perform.

That's higher order thinking, and those verbs are cognitive actions or processes students will demonstrate.  Depth of knowledge deals with the setting, scenario, or situation in which thinking is demonstrated.  It addresses context rather than cognition -- in other words, it's not what the student is expected to do or demonstrate but rather the scenario or the situation in which students express and share their learning.

A D.O.K.-1 assignment or assessment is very content-driven, focusing recognizing, researching, and rephrasing who, what, where, when, and how about data, definitions, details, facts, figures, ideas, information, principles, and procedures.  The intent of D.O.K.-1 experiences is knowledge acquisition - acquiring and gathering the information students will need to strengthen and support their thinking.  These questions, problems, and tasks  would directly address and respond to the specific texts and topics being read and reviewed in class.  For example, the work of literary fiction current being read, the mathematical concept being taught, the scientific subject, or the historical topic.  Students can think deeply about the concepts and content; however, the context is more academic and factual.

A D.O.K-2 assignment or assessment is highly procedural, challenging students to understand, analyze, and evaluate how does it function, how does it work, or how is it used.  The goal of D.O.K.-2 experiences is knowledge application - demonstrating and communicating how information can be used to achieve or attain a certain answer, outcome, or result.  These learning activities prompt students to answer questions, address problems, and accomplish tasks correctly and successfully by applying practices, principles, and processes accurately and appropriately.   Examples of D.O.K.-2 items and tasks would be to use a mathematical procedure to solve mathematical and real world  algorithmic and word problems, explain how a natural event of phenomena occurs, describe how a text or author expresses and shares ideas and information, or how a historical event turned out as it did.  

A D.O.K.-3 assignment or assessment engages students to think strategically and use reasoning to analyze and evaluate what are the causes, connections, and consequences.  The purpose of D.O.K.-3 experiences is knowledge analysis - examining and explaining why is this information essential and relevant to know, understand, and be aware of in order to study phenomena, solve problems, and solifdify meaning.  These learning activities prompt students to express and share why can the knowledge be used to produce a certain result and how can the knowledge be used to categorize, classify, and clarify ideas, incidents, individuals, and issues.  Examples would be analyzing why an author chose to present ideas and information in a certain medium and evaluate what is the effect the author's choices has on the text and the reader.  In math and science, a D.O.K.-3 question, problem, or task engages a student to a combination of deductive and inductive reasoning to examine and explain outcomes and results.  In history, students are engaged to establish historical arguments about the claims and conclusions made about historical ideas, incidents, individuals, and issues.  They also encourage students to think hypothetically about what if, what would happen, or what could happen given certain criteria or factors.

A D.O.K.-4 assignment or assessment encourages students to expand their knowledge and extend their thinking beyond the topic, the teacher, the text, and even themselves to consider what else or how else.  The intent of D.O.K.-4 experiences is knowledge augmentation and transfer - recognizing and realizing how and why information is beneficial addressing and responding to circumstances, issues, problems, and situations in a variety of circumstances and contexts.  These are the items and tasks that generally foster and promote active learning such as project-based and problem-based learning that encourage students to analyze and evaluate what impact or influence do ideas, incidents, individuals, and issues have across the curriculum and beyond the classroom.  They also encourage students to think creatively about what can you create, do, or produce with the deeper thinking and extensive knowledge they have acquired and developed.

Notice how these levels are categorized not by the cognitive actions the students are to take but rather the context in which students demonstrate and communicate their thinking.   While the level of thinking varies, the depth of knowledge depends upon the extent in which student are to demonstrate and communicate what they have learned - academically (D.O.K.-1), procedurally (D.O.K.-2), strategically and reasonably (D.O.K.-3), or extensively and authentically (D.O.K.-4).

Still, many of us educators and our students need some kind of graphic or visual to help us clearly "see the picture" presented by depth of knowledge.  The D.O.K. Wheel unfortunately brings more confusion than clarity.  Therefore, I would like to present a visual that could provide some guidance and support - the D.O.K. Ceilings.

This is the graphic I use when I conduct my presentations on questioning for cognitive rigor, which is defined by the level of thinking and depth of knowledge students are challenged and engaged to demonstrate and communicate in their learning.  Look closely at how this image is constructed.  These are not steps but rather ceilings that indicate how extensively students are to engage with the subjects and topics they are learning.  

Hopefully, this graphic will help you design those D.O.K. lessons you are desperately trying to develop and your administrators are wanting to see - and finally rid us of the grossly inaccurate D.O.K. Wheel.

More Than a Ride: Why Every Educator, Student, Parent, and Politician Should See TOMORROWLAND

Don't dismiss TOMMOROWLAND as another marketing ploy by Disney to buy merchandise or visit their parks by making a movie about one of its rides.  

TOMORROWLAND is not SPACE MOUNTAIN: THE MOVIE.  It's also not THE HAUNTED MANSION or PIRATES OF THE CARIBBEAN.  There is no Jack Sparrow, and George Clooney's character is not cartoonish enough to turn into a character in the park or turn into an animatronic or hologram that will entertain park patrons as they neglect wait to ride Space Mountain, Mission to Mars, or Star Tours.   The only thing this movie truly has in common and connection with Disney is that it was produced by the studio and its title is named after the futuristic corner of their Disney parks - that and the portal to TOMORROWLAND is through one of its rides that was featured in the 1964 World's Fair (well, they couldn't use the saucer pillars since MEN IN BLACK already established they were flying saucers :D).

TOMORROWLAND is actually a statement about education -- particularly, what does it truly mean to demonstrate higher level thinking and communicate deeper knowledge, understanding, and awareness.  It talks about the demise of the dreamers, the innovators, the inventors, and the creative thinkers who are to look beyond factual knowledge and conventional wisdom and imagine and wonder what if.   In fact, the entire movie is an answer to what if questions.

What if the world and everything that is wrong or dysfunctional about it was able to be fixed?

What if the end of the world could be prevented or protected instead of perpetuated not only through our destructive actions and decisions but also our sensationalized and entertaining perspectives and points of view about how mankind and society will eventually self-destruct?

What if all this could happen because someone - in this case, an idealistic young girl who boldly attempts to ask her teachers who teach about the plight of mankind and society with their lessons about international conflicts, global warming, and even dystopian science fiction - was brave and bold enough to ask, "What can we do to fix it?"

Yes, all this in a Disney movie - or rather a movie produced and distributed by Disney.

However, this is not the typical Disney movie that portrays an idealistic, puritanical, and simplistic world in which everyone is animated - in appearance and action - and express themselves through song and dance.  There is also nothing insipid or pedantic about this movie.  It's not only a highly entertaining movie as well as one with a powerful message that says the following:

1) Girls are smart and wise!  Forget PITCH PERFECT 2.  This is the movie for girls this summer that is not only entertaining and enjoyable but also presents a powerful message about the potential of young ladies in the areas of science, technology, engineering, and mathematics - better known as S.T.E.M.  The main character is an intelligent teenage girl named Casey (Britt Robertson) who has an edge not because she is rebelling against the norms and rules of society but rather because she won't settle for society to lie down and surrender.  She's a believer, a dreamer, and (gasp!) an optimist who believes the world's problems can be fixed.  In fact, her ideals and hopes are the catalyst that earns her the Tomorrowland pin and has her become identified by another young girl, Athena (played by Raffey Cassidy) to recognize and recruit her to come to Tomorrowland, a utopian society where all the residents are great dreamers and thinkers like her.  These two girls drive the film forward and establish the central ideas and themes of this movie - even when the world is at its darkest and most down, there is always hope driven by ideas and dreams that it could get better.

2) Learning Through S.T.E.M. is academic, authentic, and awesome!  TOMORROWLAND is more of a promotion for a S.T.E.M. education than an advertisement for a Disney education.  The movie expresses the importance of developing deeper knowledge, understanding, and awareness of science, technology, engineering, and mathematics.  However, it does not preach.  The message is subtle yet evident through the actions and attitudes of Casey.  For example, there is a scene in which her father (played by Tim McGraw), who is an engineer who is about to be laid off by NASA once the local Cape Canaveral rocket launchpad where he works is demolished, is in his workshop tinkering with a technical device that won't work.  Casey enters, suggests to "try this", clips one of the ends of the cable to another line, and it works.  She doesn't provide an elaborate explanation or spout scientific terminology.  She solves the problem simply by "trying this" - which is truly what is the intent and purpose of teaching and learning with S.T.E.M.  It's not about answering questions, addressing problems, and accomplishing tasks by knowing, understanding, and "doing" the math and science.  It's about thinking how and why math and science - and technology and engineering - can be used to answer questions, address problems, and accomplish tasks.  However, they do it in a way that is both educational and entertaining rather than academic and austere.

3) Kids want to think and test their thinking, not be taught to think and take tests!  Interestingly, it's not the kids who frown or reject Casey for being smart.  It's her teachers who stand in front of the room teaching about nuclear war and world conflict, about the polar ice caps melting due to global warming, and the dystopian science fiction written by Ray Bradbury, Aldous Huxley, and George Orwell that paint a bleak forecast of our future and profess their own ideas about the impossibility of utopias.  Casey raises her hand high in each class, which goes ignored until her English class when her teacher groans and rolls his eyes as he calls on her.  That's when she asks, "Can we fix it?" Unfortunately, the class ends before a response can be given, and Casey is left on her own to think deeply about how could she answer her own question (which she eventually does).  These are the scenes that truly gripped me not only as an educator but as a parent and a citizen of the world.  Our curriculum and the teaching we do can be very negative, focusing on what are or have been the problems, how were they fixed or attempted to be fixed, and how can we learn what those steps so we can handle those problems if they happen again.  Unfortunately, not every problem can be solved; however, they can be addressed, handled, resolved, or settled, and that's what we need to teach our kids - to think about how to solve problems but test whether their ideas, hypotheses, and predictions are valid and viable.  That's how kids not only learn but also demonstrate and communicate learning, and that's what this movie is advising.  Let kids think about and test their thinking, not be talked to and tested.

4) The failures and flaws of the future can be fixed even before they happen. Even Tomorrowland, which was a utopian society, crumbles.  However, instead of being another story about the failures and flaws of the future as depicted in the printed and filmed texts of THE HUNGER GAMES trilogy, the DIVERGENT books, and the MAZE RUNNER novels, TOMORROWLAND focuses on how can these failures and flaws be fixed without revolutionary uprisings against the powers that be.  In fact, this movie could be perceived as presenting the cause behind the dystopian futures of this film.  We were so consumed with, exposed to, and stressed about different ways doomsday will happen that we were actually inspired and informed how to cause and create the apocalypse.  Simply put - we provided ourselves with the tools of our own destruction by educating and entertaining ourselves about it rather than challenging and engaging ourselves to think deeply about how to avoid it.  TOMORROWLAND takes more of a design approach to addressing the problem by accepting and acknowledging, Here are the flaws, so what can we do to innovate or invent to improve the world rather than work toward avoiding it or even ignoring it?  The future does not have to be flawed or even fatal.  It can be fixed - and that's what Casey believes and sets out to do.

I won't tell you if she's successful though.  You'll have to see this movie for yourself.  If you do, and you have children, take them.  I took my 12 year old daughter - who wanted to see the movie because of the Disney connection - and her 9 year old sister - who didn't want to see it because what she saw in the ads didn't interested her.  However, all three of us walked out of there blown away not only by how entertaining and enjoyable the movie was but also how it made us to enthused and encouraged.  The three of us walked out of there constantly saying, "WOW!" not because of the story or the special effect but because the meaning and message was so encouraging.  When I asked them what I thought the movie was about, their answers were the four items listed above, and the explanations were based on the conversation we had.

If you're a teacher, show this to your students - especially if you are implementing or attempting to implement a S.T.E.M. instructional focus at your school.  Ask your students, "What does this film infer and suggest about the following: girls learning science and math, learning for and through S.T.E.M., thinking to learn vs. learning to think, testing vs. taking tests, the future can be fixed?"  Then ask them, "How could you incorporate these ideas and themes to deepen your learning experience in and out of school?"  Make your goal for demonstrating and communicating not only to meet the performance objectives of academic standards but also to demonstrate and communicate deeper knowledge and thinking to earn that pin to Tomorrowland.

If you know an elected politician in your local community or who works in politics, show them this film.  Discuss with them what is the meaning and message of this film.  Then challenge them to consider how could they change and lead their community and constituents to fix the future by focusing on making the present a better place. Have them fulfill those promises they made in their campaigns.  Remind them what they promised and prompt them to follow through even in the face of doubt, discouragement, and disillusion.

That's what Casey did in TOMORROWLAND, that's what the movie teaches us, and that's why every educator, student, parent, and politician should see TOMORROWLAND - because it will challenge and engage you to think deeply about how to fix the future by focusing on today.

Is there a C.R.Q. 0 - or Should There Be One?

Should there be a C.R.Q. 0?

That was a question posed to me today from a teacher during a professional development training I was conducting for teachers who worked at the Santa Clara County of Education Community Schools. I was in the middle of explaining my Questions 4 Cognitive Rigor template that I developed to help teachers organize and scaffold their questioning. I was explaining how the template was structured in the following manner:

• C.R.Q.1 questions inform students of what are the facts they must acquire and gather in order to think deeply about what they are learning.
• C.R.Q.2 questions focus on the standards by challenging and engaging students to think deeply about how and why does the information be processed into deeper knowledge and thinking.
• C.R.Q.3 questions expand students' knowledge and extend their thinking further by asking how can knowledge, understanding, and awareness be transferred and used in a variety of circumstances and contexts.
• C.R.Q.4 questions ask students to think critically and creatively about how they can convey what do you think, believe, or feel; express what is your opinion, perspective, or thoughts; or share what can you create, design, develop, plan, or produce using what they have learned.

The purpose of this template is for teachers to organize their questions so students can process the information they have acquired and gathered into knowledge and thinking they can transfer and use to address and respond to a variety of circumstances and situations in-depth, insightfully, and in their own unique way. The questions are called cognitive rigor questions - or C.R.Q.s - because they challenge and engage students to demonstrate and communicate the depth of their knowledge and thinking.

For years, I have been training schools and teachers on how to use this template to organize and scaffold instruction through inquiry and questioning. However, today, I was asked the good question by a teacher stated at the beginning of this blog post, "Should there be a C.R.Q.-0?"

Immediately, I asked the question I have become known to ask whenever I want someone to clarify their response: "What do you mean?"

The teacher explained how a C.R.Q.0 could be a good question that asks students to share what they already know, understand, or are aware of the concept and content that they are going to learn. She did not see where there was a section in the Questions 4 Cognitive Rigor template to determine background knowledge and was wondering whether there should be a C.R.Q.0 that does not ask what do you need to know, understand, do, or think but rather what do you already know, understand, or are aware of this concept or content you're about to learn.

To be honest, I never thought about designating a category of a C.R.Q.0 that can serve as a check for background knowledge or understanding. In fact, I really never thought about checking for background knowledge and understanding because I associated those questions as more evaluative while the C.R.Q.s I developed were more instructional, driving as well as assessing learning.

However, the idea of a C.R.Q.0 does intrigue me, and for most of today I have been wrapping my head around providing a C.R.Q.0 category that teachers can use not only to determine but also drive what students already know, understand, and are aware of concepts and content.

The C.R.Q.s I developed were derived from the concept of cognitive rigor developed by Karin Hess, Dennis Carlock, Ben Jones, and John Walkup (2009) that aligned Bloom's Revised Taxonomy with Webb's four levels of Depth-of-Knowledge. Walkup addressed the idea of is there a D.O.K.-0 in his blog. He suggests how these activities are "pseudo-challenging" or "appear intellectually challenging" (Walkup, 2015). However, from the description of the task Dr. Walkup describes- baking a cake resembling a cell - it seems as if this task would qualify more as a D.O.K.-1 in that is asks students to recall and reproduce information but in a creative manner.

In regards to C.R.Q.'s, that activity would be a C.R.Q.4. However, the task would be prompted by asking either how could you bake a cake that details the structure and discusses the functions of the parts a cell or even how could you construct a model or graphic representation that details the parts of a cell and their function?

While the C.R.Q.'s I have developed are derived from Webb's Depth-of-Knowledge, these questions focus more on how students can express and share their knowledge and thinking. The depth of their knowledge, thinking, and disposition depends upon how they are expected to demonstrate and communicate their learning. C.R.Q.1's ask students what is the information. C.R.Q.2's ask how can an answer be attained using the information. C.R.Q.3's ask how and why can the information be used to attain a particular response or result. C.R.Q.4's ask what else can you do with the information.

So why not include a C.R.Q.0 that asks what do you already know, understand, and are aware of the information?

Am I going to include a C.R.Q.0 category to the Questions 4 Cognitive Rigor? That's not for me to decide. That's up to you. If you think a C.R.Q.0 would provide students with a deeper educational experience and engage them to think deeply as well as express and share the depth of their knowledge, thinking, and disposition, then I say go for it.

After all, it's your lesson and your students. I'm just providing a tool you can use and a means to challenge and engage your students. What you do with it is up to you.

Erik M. Francis, M.Ed., M.S., is the lead professional education specialist and owner of Maverik Education LLC, providing professional development and consultation on teaching and learning for cognitive rigor. His book Now THAT'S a Good Question! Questions for Cognitive Rigor will be published by ASCD in November 2015. For more information on this topic or how to receive professional development at your site, please visit www.maverikeducation.com.

Lesson Planning by What's Happening or Trending in the World

One of the things I often hear from the educators with whom I work is about the curriculum they are using or the lack of curriculum they have to address the cognitive rigor of college and career ready standards.  They often express their concern about the resources provided to them and how they do not seem to be aligned to the standards even though the textbook publishers proclaim they are.  They also discuss how they do not have the curriculum that truly reflects the kind of questions their students will encounter and experience on the PARCC or SBAC assessments.

However, what if I told you that lessons can be derived and planned just by checking what's trending on Yahoo! or watching the news this morning?

It's 5:15 am as I write this, and I am in my hotel room in Lake Havasu, AZ, (not on vacation) watching the news and surfing the net before I get ready to work with one of my schools for the day.  I go on Yahoo! and I see this is article the lead article on the page: http://news.yahoo.com/signs-alien-life-found-2025-nasas-chief-scientist-212655192.html

Immediately, I'm thinking about how this would be a great lesson to teach today in school and how I wish I had a classroom where I could have the students read not only this article but also other articles and editorials that are currently appearing on the internet discussing this topic.  Though I don't have a classroom to teach, I do immediately take out my cognitive rigor questions and decide to make some Daily Good Questions as if I would be teaching this today.

The first thing I think about is what is the topical good question that the article addresses.  I also want students to expand their knowledge and extend their thinking about the content, ideas, and messages presented in the article.

Then I check the Next Generation Science Standards and the English Language Arts Common Core State Standards to determine what overarching concepts and standards could be addressed in this lesson.

I also consider the universal ideas and themes the article and its subject matter broach.

Then I think about what exactly would I want my students to do with the information they have read.

This is what I came up with  - and it only took me approximately 15 minutes not including the time it took to read the article.

Now I not only have a standards-driven lesson for the day but also a student-centered experience that has students thinking deeply about the ideas and information stated in the article as well as expressing and sharing their own knowledge, thinking, and disposition about what they have read.

This is the benefit of teaching and learning for cognitive rigor.  It allows us educators to provide our students deeper educational experiences that stimulate their thinking and deepen their knowledge, understanding, and awareness not only about concepts and content but also the academic standards they must meet and exceed.  It also provides us educators to expand our students' knowledge and extend their thinking across the curriculum and beyond the classroom to recognize and realize the relevance of the academic skills and subjects they are learning.

We don't need to be beholden to the curriculum packages and textbooks provided to us.  All we need to do is consider the connection between academic concepts and real world circumstances.  This can be done not only in the literary based courses such as English language arts and history / social studies but also the mathematical and scientific courses that are more conceptual and procedural.  Consider the following example that involves using the Pythagorean Theorem.

This is a topical good question that was derived directly from the Mathematics Common Core State Standards - specifically CCSS.MATH.CONTENT.8.G.B.7.  The images were found from an image search I conducted looking for mathematical and real world problems involving the Pythagorean Theorem.  If I was to teach this lesson, I would ask my students to address and respond to the topical good question and choose to solve either 2 mathematical and 1 real world problem or 1 mathematical and 2 real world problems to use as their evidentiary support in their response.  They are still "doing the math" by solving these problems.  However, they are also thinking mathematically about how can the Pythagorean Theorem and its converse be used to determine unknown side lengths in right triangles and express and share their response using some form of oral, written, creative, or technical communication.

Again, this was all planned using the internet as well my own curiosity, interest. imagination, and wonder about the concepts and content I would be teaching.

So before we go to bed or as we wake up in the morning, take a moment to read the paper, watch the news, or search the internet - or even have the news playing in the background.  Look and listen what's going on in the world.  Think about how it relates to what you are teaching or how it could address the standards.  Then make those good questions that will not only stimulate students' thinking and deepen their knowledge, understanding, and awareness but also expand their knowledge, extend their thinking, and pique their curiosity, imagination, interest, and wonder.

That's what good questions do, and it can be easy and simple to provide students deeper educational experience if you ask good questions.

Erik M. Francis, M.Ed., M.S., is the lead professional education specialist and owner of Maverik Education LLC, providing professional development and consultation on teaching and learning for cognitive rigor. His book Now THAT'S a Good Question! Questions for Cognitive Rigor will be published by ASCD in November 2015. For more information on this topic or how to receive professional development at your site, please visit www.maverikeducation.com.