1. Levels of Thinking and Reasoning
The Bloom-Marzano Hierarchy
Scott King-Owen, Instructional Services

2. Welcome! Make sure you sign the Schoollink roster!
Pick up your workshop materials.
Scott King-Owen,

3. Objectives
Define the levels of thinking and reasoning used in the Standard Course of Study and testing in North Carolina.
Generate content-specific examples of the levels of thinking and reasoning.
Apply the levels of thinking and reasoning to classroom practice.
Evaluate effective use of the levels of thinking and reasoning in classroom practice.

4. Pre-Assessment On your own paper, construct a circle map.
Within your circle write down all you know about Bloom’s Taxonomy.
Bloom’s Taxonomy

5. Bloom’s Taxonomy
1956: Benjamin Bloom publishes a small volume called Taxonomy of Educational Objectives, The Classification of Educational Goals, Handbook I: Cognitive Domain.

6. Bloom’s Taxonomy
Educators use the taxonomy to write instructional objectives.
Benjamin Bloom wanted a way to classify educational goals.
Bloom publishes his Taxonomy of Educational Objectives.
Assessments are designed according to the taxonomy.
Multi-Flow Map
Cause and Effect: The major function of this Thinking Map is to show causes—which can be sorted by root, immediate, proximal, or long-term causes—and their effects in relation to a major event. The map can be indefinitely expanded to include multiple causes—which in this example are illustrated with historical terms.
State curricula were written according to the taxonomy.

7. So, why do we still not use Bloom’s Taxonomy in North Carolina?
Bloom’s Taxonomy was updated in 2001 by Anderson and Krathwol.
So, why do we still not use Bloom’s Taxonomy in North Carolina?

8. Taxonomy Redefined Problems with Bloom’s taxonomy:
Oversimplifies the nature of thought and its relationship to learning (Furst, 1994).
The taxonomy implied a hierarchy of thought processes running from lower to higher which did not hold up in research studies (Marzano, 2001).

9. Learning: What We Now Know
Learning must take into account the types of knowledge as well as the thinking processes required for manipulating that knowledge.
Learning must take into account the learner’s control of his or her own thinking as well as the role of emotion and interest in acquiring knowledge.

10. The Birth of Marzano’s Hierarchy
Robert Marzano set out to repair the deficiencies of Bloom’s Taxonomy in the 1990s.
The result was Designing a New Taxonomy of Educational Objectives (2001).
Revised as The New Taxonomy of Educational Objectives (2006)

11. Marzano and North Carolina
North Carolina’s ABCs program required testing, and the test designers needed a taxonomy for aligning the goals and objectives to test questions so that we would teach what we test and test what we teach.
The resulting “Understanding North Carolina Tests: Thinking Skill Level” was published in 1999.

12. NC Thinking Skills

13. Marzano and North Carolina
Misconception Alert:
North Carolina did not fully adopt Marzano’s hierarchy.
Instead, state officials hybridized Marzano’s work with the original Bloom’s Taxonomy.

14. So What? Why does the hierarchy matter?
The goals and objectives of the Standard Course of Study are written with the levels of thinking and reasoning based on Marzano’s work.
So, to effectively teach the SCOS, teachers must be intimate with the hierarchy in order to know how to really teach the cognitive demands in the objectives.

15. So What? Why does the hierarchy matter?
All North Carolina EOGs and EOCs are written with the hierarchy.
So, in order to effectively prepare students for state assessments, teachers must be able to transform objectives into measurable test items.

16. Reflection
What is the most important “big idea” uncovered in the previous slides? What would be different about the way we teach and assess if we all knew this “big idea”?
In groups, come to consensus about one “big idea” from the material.

17. A Closer Look at Bloom and Marzano
Who can name the levels of Bloom’s Taxonomy?
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Bloom’s Taxonomy

18. Bloom-Marzano Hybrid Knowing Organizing Applying Analyzing
Generating
Integrating
Evaluating
Hybrid Taxonomy

19. Similarities and Differences
In groups, construct a Double-Bubble map comparing the Bloom Hierarchy to the hybrid Marzano-Bloom hierarchy.

20. Bloom – Marzano Hybrid: Definitions
LEVEL
DEFINITION
Knowing
Gathering, storing, and recalling information
Organizing
Arranging and classifying information
Applying
Using information in a new situation
Analyzing
Examining relationships among parts
Generating
Producing new meaning or ideas
Integrating
Combining information
Evaluating
Judging the quality of information

21. Bloom-Marzano Hybrid: Definitions
LEVEL
DEFINITION
Knowing
Gathering, storing, and recalling information
Organizing
Arranging and classifying information
Applying
Using information in a new situation
Analyzing
Examining relationships among parts
Generating
Producing new meaning or ideas
Integrating
Combining information
Evaluating
Judging the quality of information
Low-Level Cognitive Tasks
High-Level Cognitive Tasks

22. Part 1: Low-Level Cognitive Tasks
DEFINITION
Knowing
Gathering, storing, and recalling information
Organizing
Arranging and classifying information
Applying
Using information in a new situation

23. What does “Knowing” mean?
describe
label
identify
use the senses to observe
Knowing
Just because this is very low level does not mean that its not important. Information that is stored in memory can be used with ease if it can be recalled. This fluency of information usage allows us to read and do a host of over tasks without devoting enormous brain power to them.
recall information
store information in memory
list

24. “Knowing” Objectives
ESL 3, NL 3.02 – Recognize some high frequency words in simple or patterned text.
Science 2, 1.03 – Observe the different stages of an insect life cycle.
Social Studies 5, 4.03 – Describe the contributions of people of diverse cultures throughout the history of the United States.

25. “Knowing” Questions What is federalism?
a form of government with elected representatives
a division of power between the national and state governments
a belief in the idea of national sovereignty
a form of government in which elected officials make all economic decisions
10th Grade Civics. 2.01, B

26. “Knowing” Questions What led to the decline of soapstone mining?
Dwindling supplies
Discovery of mica
Preference for pottery
Interest in gold mining
Grade 8 EOG, 2.01, C

27. What does “Organizing” mean?
comparing
arrange information in an ordered way
sequencing in order or by a criteria
Organizing
Why would comparing and contrasting be a form of “organizing”?
The brain must describe first in the act of comparing and contrasting. But the act of bringing together two objects for comparison requires the brain to go beyond merely describing and to organize the similarities and differences between those objects.
Classifying builds upon comparing and contrasting by arranging information into logical groups by similarity or difference.
Changing the format of information merely refers to putting the information into another form. This would be akin to taking data and putting it in a chart, or on a graph, or even in pictorial form. The substance of the information does not change.
changing the format of information
contrasting
classifying

28. “Organizing” Objectives
Social Studies 6, 9.01 – Trace the historical development of governments
ELA 8, 3.03 – Evaluate and create arguments by arranging details, examples, and reasons effectively.
Advanced Functions, 1.02 – Compare distributions of univariate data.

29. “Organizing” Questions
Why are parallel circuits, rather than series circuits, commonly used in wiring houses?
They allow appliances to operate separately.
They are cheaper.
They have a higher total resistance.
The voltage drop varies with each resistor.
Physical Science, 3.03, A

30. “Organizing” Questions
5th Grade Math, 2.02, B

31. What does “Applying” mean?
knowing when to use knowledge
transferring knowledge from one field to another
using prior knowledge in a new situation
Applying
Using prior knowledge in a new situation requires students to recognize that the situation is similar to a case that they’ve already studied, as well as to bring together the appropriate information needed to solve the problem within that new situation. Much of this usage is propositional – “if, then” formatted. “If the problem is constructed this way, then I use this part of the formula, but if the problem only requires this answer, then I use another formula.” These propositions require students to know how as well as when to use information.
At its broadest, application involves transferring knowledge from one field to another, such as when a historian uses an insight from literature to solve a historical problem, or when a historian uses an idea discussed in physics to illuminate a particular historical problem (such as the Heisenburg Uncertainty Principle).
recognizing similarities and differences between situations
bringing together information to solve problems

32. “Applying” Objectives
Healthy Living K, 2.06 – Demonstrate how to get help in an emergency.
Theatre Arts 6, Plan and create a simple set for formal or informal dramatic presentations.
Spanish for Native Speakers 1, Apply understanding of conventional written and spoken expressions in a variety of settings by using appropriate and exact words to influence reactions, perceptions, and beliefs.

33. “Applying” Questions
Which propaganda technique is represented by the quote “My opponent is soft on crime!”?
Name calling
Bandwagon
Glittering generality
Stacked cards
Civics, 4.02, A

34. “Applying” Questions
A boulder falls from a cliff. What is the boulder’s acceleration during the fifth second of the fall?
2.0 m/s2
9.8 m/s2
20.0 m/s2
98 m/s2
Physical Science, 1.02, B

35. Review and Application
In your groups, you will be given some sample objectives and test questions. Identify the thinking skill level for the objectives and test questions.
Be prepared to defend your thoughts to the whole group.

36. Categorizing Objectives and Questions - A Caveat
OFTEN, categorizing a question or an objective requires that you understand the amount of prior knowledge needed to process the objective at a particular level.
Objectives in higher grades often assume prior knowledge, so what may be “knowing” to you is categorized as “applying” by the state.

37. Strange Verbs in the SCOS
To what levels do you assign the following verbs?
Acquire...
Interact...
Share knowledge...
Investigate...
Develop...
Explore...
Compose...
Observe...
Verbs that often call for the student to do something imply procedural knowledge. Procedural knowledge requires the use of information to complete a task. Many times this is nothing more than application.
Have tables debate these verbs. What skills or cognitive demands are implied by each verb?

38. Part 2: High-Level Cognitive Tasks
DEFINITION
Analyzing
Examining relationships among parts
Generating
Producing new meaning or ideas
Integrating
Combining information
Evaluating
Judging the quality of information

39. What does “Analyzing” mean?
identifying the attributes of things
cause and effect
identifying parts of objects
Analyzing
identifying logical errors
identifying relationships and patterns
identifying the main idea

40. “Analyzing” Objectives
Physics, 8.01 – Analyze the nature of electrical charges.
US History, 6.01 – Examine the factors that led to the United States taking an increasingly active role in world affairs.
Visual Arts 7, Explore and discuss the value of intuitive perceptions in the problem-solving process.

41. “Analyzing” Questions
A cell with 5% solute concentration is placed in a beaker with a 1% solute concentration. What will happen to the cell over time?
The cell will gain H2O and expand.
The cell will lose H2O and shrink.
The cell will both gain and lose equal amounts of H2O; thus, it will remain the same size.
The cell will undergo no exchange of H2O with its surroundings.
Biology, 1.04, A

42. “Analyzing” Questions
Why did the horse in “The Runaway Stallion” most likely run through the water to the opposite bank?
He was afraid that Jeff was trying to capture him.
He wanted the oats in the bucket.
He needed to cool down in the water before eating.
He was frightened by a noise that Hank made.
4th Grade Reading EOG, 2.04, A

43. What does “Generating” mean?
producing new meaning or ideas
exploring hypothetical situations
inferring new meaning
Generating
drawing a conclusion
Generating is really about transfer. In many ways, it picks up where organizing and analyzing leave off – it asks students to produce some new knowledge or arrangement of information by drawing conclusions based on the organization of that information as well as an analysis of its parts and relationships. It is analogous to applying in that students are asked to go beyond what is given to produce something. However, it differs from applying in that application strictly involves usage of knowledge (mostly procedural) in new situations. Generating more closely means creation of new knowledge and not just usage of information. This is done in several ways:
Inferring – Producing new knowledge by reading between the lines of text and uncovering what is implicit, but not explicitly stated.
Predicting – Using information to make a reasoned guess about what might happen next.
Elaborating – Adding to an existing structure by coming up with new examples or details that reinforce the structure.
Drawing a conclusion – Going beyond mere analysis of information to draw a conclusion about the importance or meaning of the information.
Exploring hypotheticals – Going beyond the known to explore what “might have been;” drawing inferences about situations by imagining “what if.”
predicting outcomes
elaborating by adding examples or details

44. “Generating” Objectives
Theatre Arts 6, Infer meaning from a script to create characters.
Science 8, Use maps, ground truthing and remote sensing to make predictions regarding changes over time, land use, urban sprawl, and resource management.
Civics, 2.02 – Explain how the United States Constitution grants and limits the authority of public officials and government agencies.

45. “Generating” Questions
Which best explains why the author includes this line in the play?
HANNAH: There are tougher questions we have to deal with than baseball movies or which river is the longest or . . .
To create sympathy for Jay
To reveal Hannah’s wisdom
To foreshadow the choice Jay will have to make
To reduce the tension building between Jay and Hannah
English 1, 4.02, C

46. “Generating” Questions
While cleaning a saltwater aquarium, students placed the aquarium plants in a container of distilled water. What effect will this have on the plants?
The plant cells will separate.
The plant cells will shrink.
The plant cells will swell.
The plant cells will remain the same.
Biology, 1.04, C

47. What does “Integrating” mean?
connecting and combining information
Integrating
generalizing from examples or details
summarizing
The line between generating and integrating is a fine one. However, the distinction depends on the ultimate product of each. Whereas generating is about simply creating new information, integrating is about putting that new information together in a cohesive, meaningful statement.
When connecting, students are not just realizing the connections between ideas (a form of analysis), but they are constructing a meaningful statement about those connections.
Summarizing requires analysis of information, but focuses on creating a cohesive whole from the parts.
Restructuring means reorganizing ideas or paradigms to include new information.
Generalizing refers to creating summary or conclusive statements based on a study of examples.
Integration in a large measure depends on generating new knowledge before it can be combined.
restructuring ideas to include new information

48. “Integrating” Objectives
ESL 9-12, IH Write an organized and focused composition with supporting details on familiar and previously studied topics.
US History, – Summarize significant events in foreign policy since the Vietnam War.
Science 7, Summarize the genetic transmittance of disease.

49. “Integrating” Questions
Which of the following statements concerning diffusion and active transport is correct?
Both diffusion and active transport require cell energy.
Neither diffusion nor active transport require cell energy.
Diffusion requires cell energy while active transport does not.
Active transport requires cell energy while diffusion does not.
Biology, 1.04, D

50. “Integrating” Questions
A house plant is wilting; however, it is still green. What vital resource is it most likely lacking?
Plant food
Water
Sunlight
Air
Grade 3 Science, 1.01, B

51. What does “Evaluating” mean?
judging the reasonableness of information
Evaluating
establishing criteria for judging
checking the accuracy of claims
rating ideas by quality

52. “Evaluating” Objectives
Civics, 6.08 – Evaluate methods used by society to address criminal and anti-social behaviors.
Algebra II, Use rational equations to model and solve problems; justify results.
ELA 7, Develop the stance of a critic by considering and presenting alternative points of view or reasons, remaining fair-minded and open to other interpretations, and creating a critical response/review of a work/topic.

53. “Evaluating” Questions
The scatterplot shows the number of absences in a week for classes of different sizes. Trevor concluded that there is a positive correlation between class size and the number of absences.
Which statement best describes why Trevor’s conclusion was incorrect?
The largest class does not have the most absences.
The smallest class does not have the least number of absences.
The data show no relationship between class size and number of absences.
The data show a negative relationship between class size and number of absences.
Math EOG 8, 4.03, C

54. “Evaluating” Questions
Which of the following pieces of information is most helpful in supporting the writer’s request for a replacement?
The writer knew she had the perfect gift for her mother.
The writer gave the hand vacuum to her mother for her fortieth birthday.
The vacuum had no suction.
The writer and her father read the owner’s manual together.
English 1, 3.01, C
Based off a passage entitled “Neat n Clean Hand Vacuum”

55. Review and Application
In your groups, will you be given some sample objectives and test questions. Identify the thinking skill level for the objectives and test questions.
Be prepared to defend your thoughts to the whole group.

56. Thoughts for the Classroom
Should teachers start at the bottom of the hierarchy and work their way up to more complex levels of thinking?
In other words, should we always start with knowing and end up at evaluating?

57. Some Thoughts for the Classroom
“[The] view [that learners must start at the bottom and work toward the top] may be characterized as the ‘climbing the ladder’ model of cognition.”
This model is “at odds with contemporary views of the learning process” and may confine students to a boring regimen of “low-level, skill-drill activities, rote memorization of discrete facts, and mind-numbing test prep worksheets.”
Quotes from Carol Ann Tomlinson and Jay McTighe, Integrating Differentiated Instruction and Understanding by Design (pages ).

58. Some Thoughts for the Classroom
Teachers should incorporate Marzano’s vocabulary into questioning:
Use a variety of questions at different cognitive levels.
Ask different students questions.
Vary the questions asked for each student.
Challenge all students to answer complex questions.

59. Some Thoughts for the Classroom
Use the Marzano structure for developing assessment items:
Write or re-write multiple choice items to match the hierarchy.
Construct open-ended items according to the cognitive demands of the objective.

60. Some Thoughts for the Classroom
Embed using the vocabulary of the hierarchy in everyday work without focusing on the structure itself:
Students do not need to know the levels of thinking and reasoning.
Students do need to know what common terms like “verify,” “assess,” “analyze,” and “infer” mean.

61. Some Thoughts for the Classroom
Does posting the objective daily mean that the teachers or the students actually understand it?
NO! NO! NO! NO!
The evidence of understanding of the objective comes from how it is “unpacked” in the classroom.
Teachers should work at making objectives “student” friendly and encouraging students to personalize objectives.

62. Marzano and CITW
How does this hierarchy connect to Marzano’s Classroom Instruction that Works?
Similarities and Differences
Comparing & Classifying = Organizing
Metaphors & Analogies = Analyzing and Generating
Nonlinguistic Representation
All forms of NR can be used at each level of the hierarchy.
Setting Objectives and Providing Feedback
Objectives (NC SCOS)
Feedback = Analyzing and Evaluating
Have groups do this on their own first.

63. North Carolina Thinking Skills Levels: Thinking Maps
Knowing
Organizing
Applying
Analyzing
Generating
Integrating
Evaluating

64. A Few Big Ideas
Categorizing an objective or a question often depends on the interpretation of that item as well as the background knowledge of the student.
Since the Bloom-Marzano hierarchy is the language of the SCOS of testing, it is essential that it be understood and applied to classroom practice.

65. A Few Big Ideas
Objectives must be “unpacked” to get at what is implied for students to know, understand, and be able to do in the classroom.
The NC SCOS did not adopt the full Marzano hierarchy, so teachers must pay attention to other critical issues such as metacognition, emotion, and interest and the roles they play in learning.

66. Conclusion -- Administration
As a result of this information,
How will my administrative practice change?
What will I look for in lesson plans and observations?
How has my understanding of the curriculum and testing changed?

67. Conclusion -- Teachers
As a result of this information,
What will I do differently in my classroom assessments?
What will I do with my lesson planning?
How will I “unpack” my curriculum to get at what students should know, understand, and be able to do?

68. Questions? Comments? Next Steps?
Conclusion
Questions?
Comments?
Next Steps?
For more information, contact Scott King-Owen,