Presentation on theme: "C16 – Tools for Assessing Learning in Engineering Teri Reed-Rhoads Purdue University School of Engineering Education Inventions and Impact 2: Building."— Presentation transcript:
C16 – Tools for Assessing Learning in Engineering Teri Reed-Rhoads Purdue University School of Engineering Education Inventions and Impact 2: Building Excellence in Undergraduate Science, Technology, Engineering, and Mathematics (STEM) Education August 2008 Washington, DC
Engineering Education Grant Writing Workshop ASEE Section Meeting April 3, 2008 Teri Reed-Rhoads Purdue University
Assessment Formative – checking along the way Summative – final analysis of achieving research goals Consider an outside evaluator – unbiased, measurable feedback Consider an external advisory board Give details, actual tools, timelines, etc.
Assessment Course Curriculum Program Project
Types of Assessment –Cognitive Domain Content knowledge (Concepts) Process knowledge, e.g., teamwork (Skills, Abilities) –Student interest, perceptions, and attitudes (Affective) –Rate change (Achievement)
PD’s Response – Goals on Cognitive Changes Goals on cognitive changes –Increase understanding of concepts Ability to solve statics problems Ability to construct free-body diagrams Ability to describe verbally the effect of external forces on a solid object –Increase processing skills Ability to solve out-of-context problems Ability to visualize 3-D problems Ability to communicate technical problems Connie Della-Piana, Russ Pimmel, Bev Watford “Project Evaluation”, Workshop for Faculty from Minority Serving Institutions, Feb. 8 –10, 2006.
Concept Inventories https://engineering.purdue.edu/SCI/worksho p/tools.html Engineering Related –20 instruments Science and Mathematics Related –8 instruments Physics and Astronomy Specific –12 instruments
Engineering Related CI
Science and Mathematics CI
*Rebecca S. Lindell, Elizabeth Peak and Thomas M. Foster, “Are They All Created Equal? A Comparison of Different Concept Inventory Development Methodologies”, 2006 Physics Education Research Conference Proceedings, American Institute of Physics. Physics and Astronomy CI*
Goals on Process Knowledge Teamwork –Team Effectiveness (Imbrie) –Team Developer (McGourty) Communications –3-D Visualization –Etc. Leadership
PD’s Response – Goals on Affective Changes Goals on affective changes –Improve students’ attitude about Profession Curriculum Department –Improve students’ confidence –Improve students’ intellectual development Connie Della-Piana, Russ Pimmel, Bev Watford “Project Evaluation”, Workshop for Faculty from Minority Serving Institutions, Feb. 8 –10, 2006.
PD’s Response – Goals on Achievement Rate Changes Goals on achievement rate changes –Improve Recruitment rates Retention or persistence rates Graduation rates Connie Della-Piana, Russ Pimmel, Bev Watford “Project Evaluation”, Workshop for Faculty from Minority Serving Institutions, Feb. 8 –10, 2006.
Know the Motivation or Background
URMs 30% of population, few in labor force Women 50% of population, 10% in labor force # of BS Engineering Degrees % of Freshman Engineering Class Under Represented Minorities 1,915 (4.8%) 7,971 (11.6%) Women 146 (0.4%) 1,961 (4.9%) 14,102 (20.5%) Chubin and Babco (2003), Walking the Talk in Retention to Graduation
Engineering Disciplines Retention Rates Engineering Discipline Proportion Women Proportion URM Total URM Non- URM Chemical Civil Computer Electrical Industrial Mechanical Total Chubin and Babco (2003), Walking the Talk in Retention to Graduation National Science Foundation. Women, Minorities, and Person with Disabilities in Science and Engineering: Arlington, VA, 2000 (NSF )
Authentic Teaching Alliance Assessment Cycle
Group Questions 1.What are the known instruments/methods in each of the three areas? (Cognitive, Affective, and Rate Change) 2.Where have these been used successfully/unsuccessfully? 3. Where are the gaps in each area?
Process One note taker One reporter Everyone discusses We will report out with 15 minutes left
There is no failure. Only feedback. -Anonymous Happy grant writing! https://engineering.purdue.edu/SCI/Workshop
Affective National academy of engineering website with book of surveys
Rate Change Longitudinal measurements, courses, rate changes of minorities, retention, enrollment, GPA for performance measures Use Associate Dean’s offices as resources for stats, changes, etc.
Wish Critical Thinking – one that is free Definition of engineering changes for students as they go through the curriculum Digital Systems within computer engineering Teaming attitude by individual Measuring how well the student can deal with an out of box situation. Clearinghouse that explains what already exists, how long is it, how it was used in the past, cliff notes version
Wish ABET could sponsor (or someone) or post tools that are being used More CI https://engineering.purdue.edu/SCI/workshop /tools.html https://engineering.purdue.edu/SCI/workshop /tools.html Tools for writing and communication Life-long learning tools (continuous learning) Procedural knowledge on engineering design FREE Critical Thinking
Wish Meta-cognition Societal context of knowledge, global competencies, service learning, research skills, discourse and collaborative dialogue, oral communication, problem solving, leadership (some instruments from Business schools), ability to integrate knowledge, longitudinal data, qualitative data analysis Evaluation of case studies
Cognitive Math placement test English Placement test Concept Inventories https://engineering.purdue.edu/SCI/workshop/tools.ht ml FE, GRE – standardized tests Critical thinking –Criticalthinking.org – has tools, and rubric developed by Gary Brown at Washington State, Watson Glazier (also not free), U of South Carolina EFFECTS (web-based) –http://www.ce.sc.edu/effects
California Critical Thinking Collegiate Learning Assessment CAAP – from ACT SALG – Student Assessment of Learning Gains IDEA – teaching assessments MSLQ – Motivated Strategies for Learning Questionnaire NSF User Friendly Guide to Assessment
Rogers and Sando, Rose-Hulman Learning Styles Instruments (Kolb, Felder) Career Instruments (Holland) FMRI, Caps measuring brain activity Experience sampling method – student wears a bracelet and when it goes off the student writes down what they are doing (engagement measures) Calibrated Peer review Teamwork effectiveness (Imbrie, Smith, Layton) National Center for Case Studies (Buffalo)