Presentation on theme: "A Framework for Assessing Professional Dispositions"— Presentation transcript:
1 A Framework for Assessing Professional Dispositions Mary Lee HallUniversity of Tennessee at MartinBrenda GulledgeAccreditation, Accountability, and Quality Institutional OrientationSponsored by AACTE and NCATESpring 2005
2 Framing Questions What are dispositions? What are the primary sources of disposition outcomes for programs?How can disposition outcomes be included in the program curriculum?What are important decisions in assessing dispositions?How can disposition assessment data be gathered and used?What are dispositions?What are the primary sources of disposition outcomes for programs?How are dispositions part of the curriculum?How can we assess dispositions in action?How can we use disposition assessment data?
3 What Are Dispositions? Toward a Working Definition of Dispositions Dispositions as a psychological constructTraits and DispositionsThought Processes and DispositionsSkills and DispositionsAttitudes and DispositionsHabits and DispositionsBeliefs and DispositionsNCATE and DispositionsA working definition of Dispositions
4 NCATE’s Definition“DISPOSITIONS. The values, commitments, and professional ethics that influence behaviors toward students, families, colleagues, and communities and affect student learning, motivation, and development as well as the educator’s own professional growth...” (NCATE 2000)NCATE’s definition of Dispositions as a construct also includes the theme of behavior; and focuses attention on candidate behaviors toward students, families, colleagues, and communities that might have an impact on student learning and development and on the candidate’s own professional growth.
5 NCATE explains further… “…Dispositions are guided by beliefs and attitudes related to values such as caring, fairness, honesty, responsibility, and social justice. For example, they might include a belief that all students can learn, a vision of high and challenging standards, or a commitment to a safe and supportive learning environment.” (NCATE 2000)
6 Dispositions are similar to but different from: Traits, thought processes, skills, habits, attitudes, and beliefs.For more on these distinctions see:Katz, L. G. (1993). Dispositions: Definitions and Implications for Early Childhood Practice. ERIC
7 What are Sources of Disposition Outcomes? Conceptual FrameworkKnowledge basePolicies
8 Conceptual Framework: The Primary Source for Dispositions What are the values, commitments, and professional ethics that guide and influence faculty behavior?How does the unit’s mission, vision, philosophical orientation, and professional knowledge base influence outcomes?
9 Vision and Dispositions Vision - the view of what professional practice in your area should look like. A vision statement represents the program’s future; what the program wants to become. The vision statement responds to the question: What should [social studies education, counseling, etc.] look like?What would you like to see the program become? What reputation would it have?What contributions would the program make to candidates and its community?What characteristics or values would the program embody?
10 Given this Vision…What are the patterns of observable behavior that our candidates would be demonstrating that are consistent with our vision?For example, if we envision teaching as a profession where practitioners demonstrate respect for human diversity, what could we observe candidates doing that would be consistent with this vision?
11 Values and Commitments as Dispositions What values and commitments about professional practice do the faculty hold? For example,Faculty value human diversity and are committed to expressions of respect for human diversity in its various forms.Faculty value and are committed to professional ethical responsibility.Faculty value and are committed to belief in the ability, talent, and learning potential of all students.
12 Given these values and commitments… What are the patterns of observable behavior that our candidates would be demonstrating that are consistent with these values and commitments?For example, if we value and are committed to a profession where practitioners value human diversity and are committed to expressions of respect for human diversity in its various forms, what could we or others observe candidates doing that would be consistent with these values and commitments?What could we observe candidates doing to indicate that they are committed to ethical responsibility?
13 Philosophical orientation and Dispositions… If the program philosophy is the lens through which the program sees the world of its professional practice, then what is the philosophical orientation, or system of principles that guides decision-making in your program?
14 For example…The mission of the Special Education Teacher Education Program is to prepare teachers who demonstrate the commitment and capacity to reform schooling and education for children with special educational needs from birth through young adulthood. This mission is built upon a set of principles that guides faculty teaching, research and service:• To collaborate with faculty within the COE and across the University, candidates, practitioners, and other community members and agencies;• To model and promote research-based and reflective practice;• To act as advocates for children with special educational needs and their families;• To support healthy development and well-being of children special educational needs and their families; and,• To promote social justice in the school and larger community.
15 Given this orientation or these principles… What are the patterns of observable behavior that our candidates would be demonstrating that are consistent with this orientation and principles?For example, if our program is guided by the principle of ethical responsibility, what could we or others observe candidates doing that would be consistent with this principle or orientation?
16 Other Sources of Disposition Outcomes State and Institutional PoliciesProfessional StandardsWhat disposition outcomes are specified in unit and program specialized professional association standards?For example, From professional standards: Candidates engage in continuous, collaborative learning to inform practice. [NAEYC, 2001]See also INTASC Disposition outcomes.Conceptual Framework(s)Mission of unit and programPhilosophy, purposes, goals, commitments, and valuesKnowledge basesCandidate proficienciesPerformance assessment systemKnowledge BaseStandardsInstitutional standardsState standards (e.g., professionalism)National (specialized professional associations), INTASC, NBPTSWisdom of best practiceResearchTheoriesPoliciesUnit (e.g., background check)Institution (e.g., Academic honesty policies)State (e.g., mandated state evaluation model)
17 How can Disposition outcomes be included in the curriculum? First, Dispositions as a frame for outcomes--
18 Re-thinking Disposition outcomes in teacher education programs If we adopted the notion of dispositions as the frame for our outcomes, we could ask and expect candidates to behave in ways consonant with the selected dispositions, regardless of what they believed about them (Raths, 2001).[For example, the disposition to engage in continuous professional development.]Which means we have to specify disposition outcomes in terms of observable behaviorsThen, we have toProvide Opportunities to Learn, Practice, and Demonstrate DispositionsBy assuring that candidates have knowledge and skills needed to demonstrate the disposition.By providing systematic opportunities to observe others modeling dispositionsBy providing systematic opportunities to practice and demonstrate dispositions over time and in a variety of situations
19 Including Dispositions in the Program Curriculum Opportunities to learn pre-requisite knowledge and skills [For example, knowledge about professional organizations, journals, and conferences; skills necessary to conduct action research]Opportunities to see target dispositions modeled [For example, faculty share their conference activities with classes; conduct action research on their own teaching and share insights, processes, and findings with classes; refer in class to professionally related local newspaper articles]Opportunities to practice and demonstrate target dispositions [For example, faculty make a point of letting students know about professional conferences, letting them miss class to attend; help initiate a literacy circle]
20 What are important decisions in assessing Dispositions?
21 When should Dispositions be assessed? Admission* - Like many meaningful knowledge and skill outcomes, there are limited disposition outcomes that can be assessed before candidates enter the program. Assessment at admission may establish a baseline for measuring future growth. Assessment at this point may be viewed as a needs assessment.Appropriate transition points* - Keep pre-requisite knowledge and skills in mind.Program completion* - Opportunity for assessing advanced and complex disposition outcomes.
22 What Dispositions should be assessed? Dispositions that have been defined in terms of clearly observable behavior (e.g., What behaviors show respect for human diversity?).Dispositions that candidates have an opportunity to learn, practice, and demonstrate throughout the program.Dispositions that are appropriate to expect at each program phase.
23 Defining Dispositions as Observable Behaviors The process of defining dispositions should include professional education faculty and members of the professional community.Together, this group should agree on a list of actions candidates would be taking when demonstrating the target disposition.For example, What actions could candidates take to demonstrate a commitment to continuous professional development?
24 “Candidates are engaging in professional development and learning when they…” read current professional journals and magazinesread professionally related articles in popular magazinesread professionally related articles in local newspapersread professional books not required for school or workparticipate in literature circles or study groupsattend professional training sessions, workshops, seminars, etc.attend local professional conferencesattend state, regional or national professional conferencesconduct classroom-based action research to inform their practiceregularly prepare professional development goals, plans to meet those goals, and assess progress toward attaining those goalsparticipate in classroom observations of other professionalsinvite observation of their own teaching by othersdemonstrate use of feedback to change teaching for the better
25 How Should Dispositions be assessed? Using clearly observable (low-inference) behaviors that are meaningful indicators of the disposition outcomeUsing different types of assessments and over timeUsing internal and external sources of assessment dataUsing assessment strategies characterized as accurate, reliable, meaningful, valid, objective, and defensible
26 Assessments are Less Accurate, Reliable, Meaningful, Valid, Objective, & Defensible… When the assessment is based on a continuous criteria. For example, “The candidate is self-motivated to engage in professional development. Circle one of the following: Never / Some of the time / Much of the time / Always” Or, “Assess candidate’s professional involvement on a scale of 1 to 5 where 1 is low and 5 is high.”When assessments are based on evaluating performance using global rather than specific indicators. For example, rating the candidate by responding to this statement: “The candidate engages in continuous professional development and learning” rather than by rating them on specific behaviors that characterize “engaging in continuous professional development.”When the assessment is based on many (5-10) performance levels.
27 Assessments are More Accurate, Reliable, Meaningful, Valid, Objective, & Defensible… When the assessment consists of a fixed measurement scale, and a list of criteria that describe the characteristics of performance for each score point. For example, as is now the case in NBPTS, NCATE, and most SPA program standards.When the assessment expresses performance criteria in terms of observable behaviors When the assessment uses fewer (2-4) performance levels
28 Constructing Low-inference Assessments of Dispositions The process of constructing assessments should involve members of the P-12 professional community. [improves validity]Use a list of observable behaviors [low-inference] generated by the faculty and P-12 professional community. [meaningful indicator]First identify those behaviors that describe an Acceptable performance standard for the professional role and time being assessed. Clear distinction between Acceptable and Unacceptable is important.Finally, identify those behaviors that describe an Optimal performance standard for the professional role and time being assessed.
29 Does not yet meet expectations A RUBRIC TO ASSESS THE DISPOSITION TO ENGAGE IN CONTINUOUS PROFESSIONAL DEVELOPMENTUNACCEPTABLEDoes not yet meet expectationsACCEPTABLEMeets ExpectationsOPTIMALExceeds ExpectationsCandidates show little or no evidence of an orientation toward continuous, self-motivated inquiry aimed at professional learning and development. They do not establish and follow through with annual professional development plans. They do not show that they read professional literature beyond that required for school or work. They do not take advantage of professional training opportunities. They show little or no evidence of using feedback from others to improve their teaching, and they do not observe others in order to learn.Candidate’s work shows clear evidence of an orientation toward continuous, self-motivated inquiry aimed at professional learning and development. Candidates regularly establish professional development goals, take action to attain those goals, and assess the outcomes of action they have taken. They read a variety of professionally related periodical literature, including professional journals, professional magazines, and articles about professional topics in popular magazines and newspapers. Candidates attend professional training sessions, workshops, seminars, including attending local professional conferences. They demonstrate that they use feedback to change their professional practice. They participate in classroom observations of other professionalsCandidates show clear and consistent evidence of an orientation toward continuous, self-motivated inquiry aimed at professional learning and development. In addition to reading a variety of professionally related periodical literature, candidates read professional books not required for school or work, or participate in collaborative literature circles or study groups focusing on professional topics. Candidates attend state, regional or national professional conferences or other training opportunities. Candidates conduct classroom-based action research to inform their practice. They invite observation of their own teaching by others
30 Disposition Evaluation Form Admission to Student Teaching Candidate: Bobbi Sue Vygotsky Date: Fall, 2003Rate candidates on each item as either: Ob (Observed) or Nob (Not Observed)_____Regularly establishes professional development goals, takes action to attain those goals, and assesses the outcomes of action they have taken_____Reads a variety of professionally related periodical literature (e.g., professional journals, professional magazines, and articles about professional topics in popular magazines and newspapers_____Attends professional training sessions, workshops, seminars, including attending local professional conferences_____Uses feedback to change professional practice_____Participates in classroom observations of other professionals
32 Put Dispositions in the Catalog “In programs where candidates are specializing in a professional area, awarding a degree or recommending for a professional license does not merely attest to the accumulation of the specified number of hours in the classroom or other professional setting but also to the demonstration of professional knowledge, skills, and dispositions. The faculty has the responsibility to both the public and the profession to award a degree or license only when the candidate has demonstrated a satisfactory level of professional knowledge, skills, and dispositions as judged by the program faculty. Further, candidates must exhibit integrity and character consistent with the standards of ethical principles set forth by appropriate professional associations and Tennessee law.”
33 Adopt Technical Performance standards For example,Candidates display evidence of commitment to the profession of teaching by Displaying exemplary attendance in class and field experiences, completing work in a timely manner for classes and field experiences, and accepting feedback from university faculty and school practitioners for professional growth and development
34 Be clear and document!Assessments based on clear, observable behavioral indicators of the disposition outcomesAssessments linking observable behavioral indicators with performance standards for each level.Documented multiple assessments, at appropriate times and from internal and external sources*Documented feedback of assessment data to the candidate and documented opportunities for improving performance*
35 How Can Disposition Assessment Data be Gathered and Used?
36 Gathering Disposition Assessment Data Internal sourcesCandidates’ demonstrated routine behavior (e.g., responsible professional behavior regarding attendance, meeting deadlines, working as part of a team, etc.)Periodic use of disposition rating forms by university faculty (e.g., at end of each semester, at identified gateways)Candidate performances in response to portfolio prompts (e.g., document and reflect on on-going professional development).Candidate performance in response to case studies or simulations (e.g., What would you do in this situation? Comment on what the teacher did in this situation.)Candidate reflections, journals, self-evaluationsCandidate surveys and exit interviews
37 Conceptual Framework What is the basis for your dispositions? P-12 educators were asked for their input regarding dispositions for our teacher education candidates as we reviewed our CF for impending NCATE visit.What is your vision of what a professional educator should look like as relates to attitude, values, beliefs, behaviors?
38 Standards, Standards, Standards Which professional organizations guide our dispositions?INTASC Standards (basis for State Knowledges, skills, and dispositions)NEA Code of EthicsSpecialized Professional Association Codes of Ethics (ACA, ISSLC, CEC)
39 Data used that relates to Dispositions Survey of 400 K-12 teachers in rural Northwest Tennessee revealed that 67% had taught students with limited English proficiency. However, 85% indicated that an improved understanding of international cultures would enhance their effectiveness in the classroom. How does this relate to dispositions?
40 Diversity, cultural awareness and community service learning A field experience developed to increase teacher education candidate awareness of diversity, cultural awareness and community service learning was the result of the above data. You will see how this fits into our outcome for dispositions related to the CF and the list of dispositions we are developing in candidates
41 Conceptual FrameworkThe theme “Educators as Facilitators of Learning” is a modification of the original theme, representing the Unit’s belief that teachers do more than deliver content. The purpose of the Teacher Education Program (TEP) at UTM is to provide education professionals with the knowledge, skills, and dispositions necessary to become effective facilitators of learning.
42 Outcome that relates to Dispositions Professional and Ethical Behavior – the candidate will be able to articulate their basic values and beliefs, value professional interactions, demonstrate self-respect and respect for others and help students examine and understand the ethical dimensions of the content of study. The candidate will be expected to discuss such matters as intellectual freedom, equity, tolerance, and due process. They will also know the professional codes of ethics in their specialty and the consequences of violating those propositions. Professionally, the candidate is expected to participate in collegial activities designed to make the entire school a productive learning environment while maintaining some type of activity in professional organizations.
43 Our Dispositions at UTM Interacts positively with peers, faculty, and P-12 personnel.Demonstrates self-respect and respect for othersAccepts constructive criticism and is willing to modify one’s behaviorAssumes responsibility when appropriateAttempts to solve problems independently and in a fair-minded mannerExhibits interest in the learner and enthusiasm for the learning processBehaves in a professional manner and in accordance with TEP guidelines
44 A Little QuizHow do our dispositions correlate with Educators as Facilitators of Learning?
45 Observable BehaviorsLet’s look at some examples of observable behaviors and how we assess them.How do our candidates learn about the expected dispositions?TCED 301 – Introduction to Teacher Education, foundations, portfolio processWebsite tutorial
46 Admissions to TEPWhere do dispositions fit into the process of admission to TEP?Candidates are interviewed by a team of unit faculty members and faculty or administrators for P-12 schools.
47 TEP InterviewLikert-type scale that has 4 indicators from 1 (Weak) to 4(Strong) and a fifth indicator “Not Observed”C. Professional/Personal AttitudesAwareness of responsibilities which are a necessary part of the professionDisplays an interest, enthusiasm, and commitment to teaching and continual professional growthFriendliness/manner/presentation of selfWillingness to accept the responsibility to adhere to the highest ethical standardsDisplays maturity in judgment and ability to make realistic decisionsCapacity for development
48 Gathering Disposition Assessment Data External SourcesCandidates’ demonstrated routine behavior in field and clinical settings (e.g., responsible professional behavior regarding attendance, meeting deadlines, working as part of a team, response to supervision, etc.)Periodic use of disposition rating forms by P-12 field and clinical teachers (e.g., at end of field experiences and student teaching)Alumni surveys can generate data about candidate preparation for work, program satisfaction, and curriculum relevancyEmployer surveys can generate data about employer satisfaction levels with the abilities,skills, and dispositions of recent graduates
49 Dispositions Rubric 4 point scale Not exhibited Exhibited some of the timeExhibited most of the timeClearly exhibited
50 Dispositions Rubric Used in key classes Positive interaction w/peers, faculty and P-12 personnelDemonstrates self respect & respect for othersAccepts constructive criticism & willing to modify behaviorAssumes responsibilitySolves problems in a fair minded mannerExhibits interest & enthusiasm in the learner & learningBehaves professionally in accordance with TEP guidelines
51 Early Field Experience Evaluation Field Experience form used in Level I Field Experiences (Early)Likert-type scale:1. Below Expectations2. Needs Improvement3. Meets Expectations4. Exceeds ExpectationsN/A Not applicable
52 C. Professional and Ethical Behavior Notice how even the titles for indicators relate back to CFWhat do the indicators remind you of?Interacts positively and appropriately with peers and P-12 school personnelDemonstrates self-respect and respect for othersAccepts constructive criticism and is willing to modify one’s behavior accordinglyAssumes responsibility when appropriateExhibits interest in the learner and enthusiasm for the learning processConducts oneself in a professional manner and in accordance with Teacher Education Program (TEP) guidelines
53 Level II Field Experiences Form is used during midpoint evaluationLevel II methods classesLikert-type scale same:1. Below expectationsNeeds ImprovementMeets ExpectationsExceeds ExpectationsN/A Not applicable
54 Professional and Ethical Behavior How should they be assessed differently in Level II? Or should they?Interacts positively and appropriately with peers and P-12 school personnelDemonstrates self-respect and respect for othersAccepts constructive criticism and is willing to modify one’s behavior accordinglyAssumes responsibility when appropriateExhibits interest in the learner and enthusiasm for the learning processConducts oneself in a professional manner and in accordance with Teacher Education Program (TEP) guidelines
55 Teacher Education Continuous Evaluation This form is applicable to candidate’s seeking teacher licensure concerning their retention in the Teacher Education Program at UTM. Any instructor of a teacher education program course may file a Teacher Education Continuous Evaluation form if a candidate’s suitability to enter or continue in the program is questioned. This form will be used in such instances as when there is evidence of a candidate’s inability to perform the professional competencies of ethical conduct, professional attitudes and behaviors, or essential mental or physical functions of a teacher. Copies of this form signed by the instructor and preferably by the candidate must be submitted to the Office of Education-Services, which will distribute all appropriate copies.
56 Continuous Evaluation Course Enrolled: Date:Candidate’s NameID Number Cumulative GPAMajor Teacher Education AdvisorPlease state specifically the nature of the deficiency. Provide specific examples which will support the evaluation.Suggested Action/RemediationSignature, faculty DateSignature, Candidate DateThe candidate’s signature indicates only that s/he has read the report, not that s/he agrees with it.
57 What happens next?How do we plan to use the Continuous Evaluation form?We are working on that – 3 copies in the candidate’s file indicates need for hearing before a retention committeeWho monitors this? Good question
59 Examples Relations with others Exceptionally well accepted Works well with othersGets along satisfactorilyHas some difficulty working with othersWorks very poorly with others
60 More Examples Attitude/Application to Work Outstanding in enthusiasm Very interested and industriousAverage in diligence and interestSomewhat indifferentDefinitely not interested
61 Back to CF and Dispositions How does the mid-session report relate to the list of dispositions and the CF?The candidate is also ranked on attendance as regular or irregular and on punctuality as on time or tardy.
62 Student Teaching Comprehensive Assessment Performance Observation Evaluation By University SupervisorBased on Tennessee Framework for Evaluation and Professional GrowthScale is:4. Exceeds Expectations3. Meets Expectations2. Requires Improvement1. Unacceptable
63 Professional GrowthCollaborates with colleagues and appropriate othersEngages in professional developmentPerforms professional responsibilities efficientlyProfits from evaluative feedback
64 Professional Responsibilities University supervisor also uses the four point scale to assess this statementCandidate’s performance reflects disposition expected of professional educator
65 What about follow-up of candidates and dispositions? Teacher Warranty UT-Martin has with area school systems – signed agreementWe guarantee the quality of teachers who complete our programOne of the statements we make is that: all candidates completing an approved teacher education programCan demonstrate success in accommodating students from diverse cultural, ethnic, international, and socioeconomic groups.How does this relate to dispositions?
66 Using Dispositions Data Program faculty can monitor candidate growth and development throughout the programSelf-assessment by candidates at key points in their program (e.g., At admission, where do I stand with respect to these expected disposition standards?)Verify change or lack of change in dispositions demonstrated by candidates engaged in documented efforts to improve their performanceProgram evaluation and improvement
67 Teacher Education Interview Summary Example of using dataUniversity of Tennessee At MartinOral Communication and Dispositions Interview EvaluationsSem/year Admitted Admitted with Conditons Rejected TotalFallSpringFallSpringFallSpring
68 Summary and QuestionsCan you see some difficulty as I talked about our dispositions and the rating instruments, etc. as far as using the data for program evaluation and improvement?How could that be changed?In closing, thank you for attending our session and isn’t this a wonderful time to be in Teacher Education?