Why Paired Confidence-Competence Measures Could be the Most Important Measures of All for Assessment of Student Success AABSS 2019 Conference – Las Vegas, Nevada, February 25-26 Dr. Steven Fleisher, Psychology Faculty California State University Channel Islands Dr. Edward Nuhfer, Professor, Director of Faculty Development California State University (retired)

... co-researchers Chris Cogan, Memorial University Rachel Watson, University of Wyoming Kali Nicholas Moon, University of Wyoming Rick Zechman, Humboldt State University Paul Walter, St Edwards University Karl Wirth, Macalester College Ami Wangeline, Laramie County Community College Eric Gaze, Bowdoin College And over 25,000 participants – “Cast of thousands!”

Confidence-competence measures have a long history of being done poorly.

Destructive Message 1 Self-Assessment is Random Nonsense Peer-reviewed literature Implications for people People’s feelings about their own abilities are simply random guessing. Their feelings have no value as a source of information to assess real knowledge or skill. You may believe that you know yourself, but don't take that belief too seriously. Peoples’ beliefs about themselves don’t match reality.

Destructive Message 2. People have overly inflated views of their own abilities Peer-reviewed literature Implications for people The vast majority overestimate what they actually know or can do. Those who are the least skilled have the most overinflated opinions of their abilities. People may collectively believe that they can accomplish a shared aspiration, in reality that aspiration will not likely be practical. However, experts in authority can show members of society where their limits are.

Message 3. People are pretty ____ good at self-assessment, and collectively groups are awesome at it. Peer-reviewed literature Implications for people Most people have the ability to self-assess their present knowledge and skills with reasonable accuracy. Only a few people have highly overinflated views of their abilities. If people believe that they are capable of achieving a shared aspiration, it is very likely that they can find a way to achieve it.

We are either mostly like... or we are mostly like...

...unskilled and unaware of it

Seminal Paper 1999 Journal of Personality and Social Psychology First serious effort to quantify self-assessment. First to demonstrate self-assessment is teachable. We now know more, but only because of the foundation that they began to build.

The Dunning-Kruger Effect “People are typically overly optimistic when evaluating the quality of their performance on social and intellectual tasks. In particular, poor performers grossly overestimate their performances because their incompetence deprives them of the skills needed to recognize their deficits” (Ehrlinger, Johnson, Banner, Dunning, & Kruger, 2008).

The Dunning-Kruger Effect in a K-D Graph Blue – Self-assessed competency Red – Measured competency The Dunning–Kruger effect expresses that relatively unskilled people suffer illusory superiority and mistakenly assess their abilities to be much higher than they really are. Conversely, persons with high ability tend to underestimate their relative competence. “...their incompetence deprives them of the skills needed to recognize their deficits.”

What “perfect signal” looks like... What random noise looks like...

...what strong associations between self-assessed competence and actual competence look like...

...and what strong associations do not look like.

How well do people self-assess?

Individuals’ Self-Assessment Accuracies from Knowledge Survey

Individuals’ Self-Assessment Accuracies from Postdicted

Groups (n =50): Self-Assessment Accuracies

Accuracy by Means of Groups of Size N for Paired Measures of Confidence with Competence

Now, What can we learn about our institutions from such paired measures?

What we ought to see...

Well...POOP!

Some places do better’n others

Best Case

Good Case

Fair Case

How about bringing this to our students?

My Spring 2019 Classes

Show of Hands!! How Many of You... Scored higher than 50% on the SLCI? Scored higher than 70% on the SLCI? Scored Higher than 80% on the SLCI?

Show of Hands!! How Many of You... Were accurate in your self-assessment within ± 30 ppts or less? Were accurate in your self-assessment within ± 20 ppts or less? Were accurate in your self-assessment within ± 10 ppts or less?

Show of Hands!! How Many of You... Have ever been asked to estimate your self-assessment accuracy on a college test? Have received a measure of your self-assessment accuracy as feedback?

Correlation Relationships of My Classes, Spring 2019

Look at Groups by Science Courses Taken

How Close Do Groups’ Confidence Track SLCI Scores?

How Close Do Groups’ Confidence Track SLCI Scores?

Genders – Who scores highest overall (24,000 Participants)? Neither men nor women score significantly differently.

Genders – Who scores highest in my classes?

Why in my classes?

Who self-assesses more accurately?

Dominant Ethnicities Who scores highest in my classes?

Why in my classes?

Who self-assesses more accurately?

How well did we do?

How well did we do? Second measure on postdicted self-assessment

NCC Metacognition & Motivation Workshop Wirth - Cutting Edge Metacognition Workshop 5/1/2019 Pedagogical Challenge Metacognition (including self-assessment) is a “self-imposed internal conversation.” Metacognition is essential to becoming an expert learner (strategic, self-regulated, and reflective). Easily assumed that students are doing it, or can develop on their own; both assumptions are wrong. Challenge is to keep students in constant contact with their metacognition. Instruction must be explicit (Pintrich, 2002). Metacognitive self-assessment may be the most beneficial skill to develop (Pintrich, 2002; Rivers, 2001). 46

END