1. Deconstructing 21st Century CSOs
What Do We Expect Students to Learn?
Title I School Improvement Coordinators
Erin Sullivan & Gail Varney
WVDE Title I Coordinators
Limited paper copies will be used for this conference.  However, all of the presentations are now posted on the Title I website.  If you want hard copies of the sessions, you may want to print the PPs or create a file on your desktop and add the presentations.  That way you can take notes at the bottom of the slides using your computer. The presentation materials can be accessed at
Several presentations are available under the “PRESENTATIONS” tab on TEACH Provide background of this presentation – how it has been used by our office. Four driving PLC questions.

2. Session Goals
Understand the change in the rigor and relevance of the revised CSOs.
Participate in a process of deconstructing the CSOs that ensures teachers understand the depth of knowledge and what reasoning and skills students will be expected to master.
Understand that the deconstructed CSOs form the basis of designing assessments and selecting appropriate instructional strategies.
WV has gone to a Standards based instructional format. CSOs have been revised to align with the NAEP standards. CSOs address 21st Century real-world knowledge and skills.

3. Three Ways to Improve Student Learning
Raise the level of rigor in the content standards and objectives.
Increase the skill and knowledge of teachers in teaching the content.
Engage students in active learning designed around the standards for content, learning skills and technology tools.
The role of classroom teachers is to strive toward reaching these goals. By engaging students in active learning that is not text-book driven, rather is designed around the standards for content, learning skills and technology tools; teachers will increase the rigor of instruction in their classrooms.

4. The Rigor/Relevance FrameworkD G T A X O N M Y C
Assimilation 6 5 4 3 2 1 D
Adaptation
Evaluation
Synthesis
Analysis
Application A
Acquisition B
Application
Understanding
This visual captures our goal for 21st century curriculum in West Virginia schools. Our goal for the 21st century learner is to move both learning experiences and assessment into the D Quadrant of the Rigor and Relevance Framework. This Framework is a tool developed by the staff of the International Center for Leadership in Education to examine curriculum, instruction and assessment. The Rigor/Relevance Framework is based on two dimensions of higher standards and student achievement.
First, there is the continuum of knowledge that describes the increasingly complex ways in which we think. The Knowledge Taxonomy is based on the 6 levels of Bloom’s Taxonomy. The low end of the continuum involves acquiring knowledge and being able to recall or locate that knowledge in a simple manner. The high end of the Knowledge Taxonomy labels more complex ways in which individuals use knowledge. At this level, knowledge is fully integrated into one’s mind. They can take several pieces of knowledge and combine them in both logical and creative ways. Assimilation of knowledge is a good way to describe this high level of the thinking continuum. Assimilation is often referred to as a higher-order thinking skill. At this level the student can solve multi-step problems and create unique work and solutions.
The application Model, or the second continuum, is one of action. The five levels of this continuum describe putting knowledge to use. While the low end is knowledge acquired for its own sake, the high end signifies action-sue of that knowledge to solve complex real-world problems and to create projects, designs, and other works for real-world situations.
Awareness
Apply
across
disciplines
Apply to
real world
predictable
situations
Apply to real-world
unpredictable
situations
Knowledge
Apply in
discipline
APPLICATION MODEL

5. RLA CSO Comparison – Grade 4
Previous Policy
RLA Summarize the author’s purpose (e.g., to persuade; to inform; to determine a specific viewpoint).
Revised Policy
RLA.O Determine author’s purposes in literary and informational texts and use supporting material to justify author’s intent:
To persuade
To entertain
To inform
To determine a specific viewpoint
Now, let’s take a brief look at how the COSs have changed – how complex they now are.
Show the previous CSO (left) and ask if that could be “taught” in one period. Show the new CSO and ask the same thing.
In the previous policy, “e.g.” were suggestions, not requirements. A teacher could have simply selected one and taught it in isolation. However, with the revised policy, the instruction must be developed over a period of time.

6. Mathematics CSO Comparison - Grade 3
Previous Policy
MA.3.1.6
Compare and order fractions with like and unlike denominators using concrete models.
Revised Policy
M.O.3.1.6
Create concrete models
and pictorial
representations to
compare and order fractions with like and unlike denominators,
add and subtract fractions with like denominators, and verify results.
Show the previous CSO (left) and ask if that could be “taught” in one period. Show the new CSO and ask the same thing.
Students actually have to create a product in the revised objective. Previously, the objective could have been “checked off” as being “covered” in a single lesson. However, this new objective must be “taught” over a series of lessons throughout the year.

7. What is Depth of Knowledge?
The degree of depth or complexity of knowledge reflected in the content standards and assessments
How deeply a student needs to understand the content for a given response/assessment
It is essential all educators recognize the Depth of Knowledge in the revised CSOs. We need to understand the DOK of the CSOs to comprehend what we want kids to know, understand, and be able to do. If teachers don’t understand the DOK of the CSOs, they won’t be able to effectively instruct or assess. They won’t be able to prepare students for the DOK the WESTEST.

8. Depth of Knowledge Levels
Level 1: Recall Recall, recognition; skill, behavior or sequence of behaviors learned through practice and easily performed Level 2: Skill/Concept Engagement of some mental processing beyond recalling; the use of information or conceptual knowledge; requires making some decisions regarding how to approach a question or problem Level 3: Strategic Thinking More sophisticated reasoning and analysis; deep understanding; students are required to solve problems & draw conclusions Level 4: Extended Thinking Requires integration of knowledge from multiple sources and ability to represent knowledge in a variety of ways; usually requires work over an extended period of time
Refer to handout on DOK levels. The DOK has 4 levels. Depth of Knowledge, or DOK, is a way to think about content complexity, not content difficulty. That is the first thing with which people must come to understanding. Complexity is different from difficulty. For example, if students have not seen the word or content before, it might be difficult for them, but it is not complex.
Dr. Norman Webb (University of Wisconsin) completed a depth of knowledge and alignment study of the content standards and objectives. As you see on this slide, there are 4 depth of knowledge levels and each objective is assigned a DOK level based upon the information on this slide. Dr. Webb is currently completing the official alignment study between the WV CSOs and the WESTEST 2 items. When he has completed his work, we will post the DOK levels to the Teach 21 site.
Level 1 DOK is recall and recognition.
Level 2 – is about using a skill or a concept, i.e. Paraphrase. Conceptual understanding generally refers to the integration and application of concepts and other ideas within a content area. Procedural understanding denotes knowledge about skills and sequence of steps, when and how these should be used appropriately, and their efficient and accurate applications.
Level 3 DOK requires strategic thinking. Analysis and other examples are given here. Non-routine problem solving like in reading and determining author’s purpose is Level 3.
Level 4 DOK requires extended thinking usually requires work over a period of time, including gathering information, analyzing findings, preparing reports, and presenting findings.
The new rigor of the WESTEST means it contains more levels 2 & 3 items.

9. Understanding Depth of Knowledge
DOK is about intended outcome, not difficulty.
DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.
We must understand that DOK is not about the difficulty of the objective, rather, it is about the intended outcome or the complexity of the mental processing that must occur in order to get to the intended outcome. Consider the example of adding This is easy. Students can do this by rote. It’s a DOK 1.
Increasing the difficulty of the problem by asking a student to add 4, ,530 does not increase the complexity. To solve this problem requires a child to recall the sequence of steps. The level of cognitive difficulty is still a DOK 1. It’s still recall. Knowing the rules of adding larger numbers - increasing the difficulty of the addition problem - does not affect the intended outcome. We still want the student to be able to add, just larger numbers.

10. Understanding Depth of Knowledge
Words like explain or analyze have to be considered in context.
“Explain to me where you live” does not raise the DOK of a simple rote response.
Even if the student has to use addresses or landmarks, the student is doing nothing more than recalling and reciting.
Copyright © 2007 Mississippi Department of Education
Even though the verbs (or action words) are listed on graphics explaining DOK, it is not the verb that is critically important, rather it is the words that follow the action. What is the objective really asking the student to do? In the above example, the word “Explain” may be perceived as a higher level objective, however, it simply asks the student to recall information.

11. Understanding Depth of Knowledge
Difficulty is a reference to how many students answer a question correctly:
How many of you know the definition of exaggerate?
DOK 1 - recall
If all of you know the definition, this question is an easy question.
How many of you know the definition of prescient?
If most of you do not know the definition, this question is a difficult question.
Copyright © 2007 Mississippi Department of Education
Let’s look at another example of how the level of difficulty can be increased without increasing the level DOK.
Raise your hand if you know the definition of exaggerate.
Raise your hand if you know the definition of prescient.
A common confusion is that in order to make an objective more difficult, we must increase the level of DOK. This is simply NOT the case. The questions are asking the student to recall information…one definition might be considered less likely to be known that the other, but the objective remains the same…provide a definition. (RECALL). We can raise the level of difficulty WITHOUT raising the DOK level of the objective.
Prescient: press-ee-ant, preshent, pree-see-ent: perceiving the significance of events before they occur. To know beforehand.

12. Activity: Determining DOK Level
DOK ? Describe three characteristics of metamorphic rocks
DOK ? Describe the difference between metamorphic and igneous rocks.
DOK ? Describe a model that you might use to represent the relationships that exist within the rock cycle.
Please refer to your handout on DOK levels:
Verbs alone do not determine the DOK’s level. This is an example of the same verb being used for different objectives. Use chart to determine DOK.
Same verb – three DOK levels Key= 1,2,3
1 = (simple recall)
2 = (requires cognitive processing to determine the differences in the two rock types)
3 =(requires deep understanding of rock cycle and a determination of how best to represent it)

13. Practice Activity: Sample One
Locate and interpret key information in illustrations, titles, chapter headings, glossaries and maps to answer questions.
What is the DOK level? Answer is level two. What makes this a DOK 2 – not a DOK 1?
(the word interpret) We can make it a level one by removing the word “interpret”.

14. Practice Activity: Sample Two
Solve one-step linear equations and inequalities with one variable, interpret the solution or solutions in the context from which they arose, and verify the reasonableness of the results.
A one-step linear equation sample might be: 20 = 3X X= 6 is not a reasonable answer for this sample equation.
This objective is an example of Level 3. The expectation expressed in this objective is that students will not only solve and verify the results. This will require students to do some reasoning in order to interpret the solution and could be fairly complex depending on the context. If students were only required to solve linear equations and verify solutions, then the expectation would be Level 2.

15. Practice Activity: Sample Three
Design a statistical experiment to study a problem and communicate the outcomes.
This is a DOK 4 because to plan a statistical experiment, a student must define the problem and develop a procedure for solving it. This involves identifying the correct statistical model, applying the model to data and communicating the outcome of the selected model. The student must interpret finding and make reasonable and rationed inferences from obtained data. This represents a complex, multistep reasoning task.

16. No longer will teachers be able to “check off” CSOs as being “covered” in one class period.
Depth of knowledge in the CSOs requires extended learning experiences in order to master them. The next slide is a picture that represents a teacher trying to teach a 21st Century CSO in one class period.

17. This is why the CSOs to be deconstructed.

18. Where do we begin?
In order to provide instruction at the appropriate DOK, you must deconstruct the CSOs into smaller pieces (learning targets). Ask yourself, what does the CSO really ask my students to do? How will I know they have mastered the CSO?

19. Deconstructing the Standards is…
…a systematic process to identify embedded learning targets in standards and objectives so that nothing essential is missed during instruction.

20. Learning Targets
What students should know, understand and be able to do to master the standards/objectives

21. We Need Clear Targets to. . .
Correctly identify what students know and don’t know
Select appropriate assessments
Plan effective instruction
Keep track of student learning target by target or standard by standard
Have students self-assess or set goals likely to help them learn more
Why should we deconstruct?
© 2007 Educational Testing Service
Assessment Training Institute

22. Types of Learning Targets
Knowledge
knowing and understanding facts and concepts
learned outright or via reference
Reasoning
mental processes we want students to engage in
USING knowledge to solve problems
Performance Skills
process is most important
Products
using knowledge, reasoning, and skills to create a product
Remember, step one was choosing a CSO that needs to be deconstructed because it’s unclear due to complexity.

23. Performance Skill Objective
Product Objective
Product
Performance Skill
Reasoning
Knowledge
Learning Target
Hierarchy
Performance Skill Objective
Performance Skill
Reasoning
Knowledge
Reasoning Objective
Reasoning
Knowledge
There is a hierarchy to learning targets. Knowledge targets have no reasoning, performance, or product underpinnings. Reasoning targets require knowledge but no performance or products. Performance requires underlying knowledge and reasoning but not products. Product targets might be underpinned by all four types of learning targets. The exception is that a product might not have a performance underpinning.
Many of WV’s standards/objectives contain more than a single type of learning target. When this happens, each part needs to be deconstructed separately.
Knowledge Objective
Knowledge

24. Examples: Knowledge Targets
Identify sight words
Identify similes and metaphors
Know defining characteristics of various literary genres
Count and group concrete manipulatives by ones, tens, and hundreds to 1,000

25. Examples: Reasoning Targets
Make a prediction based on evidence
Distinguish between fact and opinion
Evaluate information from a variety of resources
Classify and compare triangles by sides and angles

26. Examples: Performance/Skill Targets
Read aloud with fluency and expression
Use self-correction strategies
Use inductive reasoning to find and justify the laws of exponents with numeric bases
Model, identify and describe square, prime and composite numbers

27. Examples: Product Targets
Produce a grammatically correct sentence
Develop a proper paragraph form in a written composition
Compose a written composition using the five-step writing process
Create a design with more than one line of symmetry

28. Deconstruction Steps
Choose a standard/objective for which the embedded learning might not be consistently identified.
Identify its ultimate type: knowledge, reasoning, performance skill, or product.
Ask four questions:
What knowledge do students need ?
What reasoning proficiencies (if any) do students need?
What performance skills (if any) do students need?
What products (if any) do students need to practice?
The four questions are all asking what students need to MASTER the standard or objective.

29. Performance Skill Targets
Step One
Standard/Objective:
Step Two
Type:  Knowledge  Reasoning  Performance Skill  Product
Learning Targets
What are the knowledge, reasoning, skill or product targets underpinning this objective?
Knowledge Targets
Reasoning Targets
Performance Skill Targets
Product Targets
What must students know to master this standard?
How are students using knowledge to solve a problem, make a decision, form a plan, etc.?
What must students be able to do ? How are they using knowledge and reasoning to perform a task?
What are students asked to produce or create?
Step Three
This is a worksheet designed to help teachers in the deconstruction process. Go through a review of the steps, showing them what questions they need to ask as they do steps two and three. Emphasize again the underpinning/scaffolding concept so they know if they determine a CSO includes performance skill, it will also have reasoning and knowledge underpinnings.

30. Activity: Learning Target Verb Sort
Categorize verbs under correct learning target headings:
Knowledge
Reasoning
Performance Skill
Product
Ask teachers to sort according to what they think the verb is asking students to know, understand, or do. (Activity packets of verbs and Learning Target headings)

31. Explain Predict Observe Design Understand Infer Perform Produce
Knowledge
Reasoning
Performance
Product
Explain
Predict
Observe
Design
Understand
Infer
Perform
Produce
Describe
Classify Do
Make
Identify
Compare
Conduct
Write
Define
Summarize
Speak
Draw
Recall
Analyze
Operate
Represent
Recognize
Evaluate
Investigate
Display
Select
Generalize
Collect
Model

32. Tips for Deconstructing the CSOs
Analyze the wording of the standard/objective to determine key concepts and key skills
Read through indicators
Circle verbs to identify key skills
Underline nouns and noun phrases to identify key concepts
Example M.O :
Create grade-appropriate real-world problems involving any of the four operations using multiple strategies, explain the reasoning used, and justify the procedures selected when presenting solutions.
When presenting this slide, open it up for discussion as to different words they would circle and underline. Good opportunity to point out how complex the CSOs are and the many learning targets embedded in some of them.

33. How Many Types of Learning Targets?
EXAMPLE 1: 8th Grade Reading/L.A., RLA.O.8.1.5
Use pre-reading and comprehension strategies (e.g. generating questions and previewing, activating and evaluating prior knowledge and scanning or skimming texts) to critically analyze and evaluate the comparison of literary and informational texts for
Making judgments
Hypothesizing
Making complex or abstract summaries
Have teachers refer to their hard copy of this CSO. Ask them to circle the key verbs and nouns/noun phrases. Have them use this process to determine the TYPES of learning targets are in this one.
This one has at least three types of targets. Be sure that all targets are included on the chart.
Knowledge
know what strategies are considered to be pre-reading strategies
Understand the types of comprehension skills and (example) how to locate a main idea and details
Understand the process for inferring, drawing generalizations etc.
Understand the difference in skimming and scanning processes
Reasoning
generating questions
activating and evaluating prior knowledge
critically analyze and evaluate
making judgments
hypothesizing
making complex or abstract summaries
Performance
scanning or skimming
previewing

34. How Many Types of Learning Targets?
EXAMPLE 2: Grade 3 Mathematics, M.O.3.5.1
Collect and organize grade-appropriate real-world data from observation, surveys, and experiments, and identify and construct appropriate ways to display data.
Have teachers refer to their hard copy of this CSO. Ask them to circle the key verbs and nouns/noun phrases. Have them use this process to determine the TYPES of learning targets are in this one.
This one has 4 types of targets:
Knowledge
What is data?
Define a survey and identify its usefulness.
What type of data must be collected for a survey, observation and experiments?
Reasoning
Identify appropriate ways to display data.
Performance Skill
Collect data.
Product
Construct appropriate ways to display data.

35. Activity Let’s deconstruct one together!
Provide teachers with handout worksheets of the RLA CSO on the following slide and have them work at their tables to deconstruct. Provide copies of the Title I Office’s deconstruction of this one.

36. Standard/Objective: RLA. O. 4. 1
Standard/Objective: RLA.O Determine the author's purpose in literary and informational text and use supporting material to justify the author’s intent to persuade, entertain, inform and determine a specific viewpoint.
Type:  Knowledge
 Reasoning
Performance Skill
 Product
Learning Targets
What are the knowledge, reasoning, skill or product targets underpinning this objective?
Knowledge Targets
What must students know?
Reasoning Targets
How are students using knowledge to solve a problem, make a decision, etc.?
Performance Skill Targets
What must students be able to do? How are they using knowledge and reasoning to perform a task?
Product Targets
What are students asked to produce or create?
Allow groups time to do the handout worksheets of this slide at their tables, then discuss in whole group, comparing their work.

37. Performance Skill Targets
Standard/Objective: RLA.O Determine the author's purpose in literary and informational text and use supporting material to justify the author’s intent to persuade, entertain, inform and determine a specific viewpoint.
Type:  Knowledge
 Reasoning
X Performance Skill
 Product
Learning Targets
What are the knowledge, reasoning, skill or product targets underpinning this objective?
Knowledge Target
Reasoning Targets
Performance Skill Targets
Product Targets
Identify main idea
Identify supporting details.
Understand the meaning of persuade, entertain, & inform.
Identify the 2 types of writing (informational & narrative)
Understand why authors write.
Understand the concept of compare/contrast.
Understand summarization techniques.
Compare and contrast two types of reading genres
Draw a conclusion about author’s purpose by identifying words in text to justify the intent.
Draw a conclusion about author’s pt. of view based on key words in text.
Use supporting materials to justify the author’s purpose.
Allow groups time to do the handout worksheets of this slide at their tables, then discuss in whole group, comparing their work.

38. Your Turn to Deconstruct
Divide into teams.
Choose one standard/objective that everyone in the group will tackle.
Each group deconstructs the chosen standard/objective.
Groups compare learning targets and come to consensus.
Ask groups to work together to deconstruct one of their own CSOs.

39. Choose a CSO to Practice Deconstructing
Estimate, measure, compare, order and draw lengths of real objects in parts of an inch up to 1/8 of an inch and millimeters. (M.O.5.4.1)
write to persuade using order of importance, classifying differences and similarities, classifying advantages and disadvantages. (RLA.O.4.2.6)
use examples, and details in practical texts to make inferences and logical predictions about outcomes of procedures in such texts. (RLA.O )

40. QUESTIONS?????