Presentation on theme: "C16 – Tools for Assessing Learning in Engineering Teri Reed-Rhoads Purdue University School of Engineering Education Inventions and Impact 2: Building."— Presentation transcript:
1C16 – Tools for Assessing Learning in Engineering Teri Reed-Rhoads Purdue University School of Engineering EducationInventions and Impact 2: Building Excellence in Undergraduate Science, Technology, Engineering, and Mathematics (STEM) Education August 2008 Washington, DC
2Engineering Education Grant Writing Workshop ASEE Section MeetingApril 3, 2008Teri Reed-RhoadsPurdue University
3Assessment Formative – checking along the way Summative – final analysis of achieving research goalsConsider an outside evaluator – unbiased, measurable feedbackConsider an external advisory boardGive details, actual tools, timelines, etc.
5Types of Assessment Cognitive Domain Content knowledge (Concepts)Process knowledge, e.g., teamwork (Skills, Abilities)Student interest, perceptions, and attitudes (Affective)Rate change (Achievement)
6PD’s Response – Goals on Cognitive Changes Increase understanding of conceptsAbility to solve statics problemsAbility to construct free-body diagramsAbility to describe verbally the effect of external forces on a solid objectIncrease processing skillsAbility to solve out-of-context problemsAbility to visualize 3-D problemsAbility to communicate technical problemsConnie Della-Piana, Russ Pimmel, Bev Watford “Project Evaluation”, Workshop for Faculty from Minority Serving Institutions, Feb. 8 –10, 2006.
7Concept Inventorieshttps://engineering.purdue.edu/SCI/workshop/tools.htmlEngineering Related20 instrumentsScience and Mathematics Related8 instrumentsPhysics and Astronomy Specific12 instruments
10Physics and Astronomy CI* *Rebecca S. Lindell, Elizabeth Peak and Thomas M. Foster, “Are They All Created Equal? A Comparison of Different Concept Inventory Development Methodologies”, 2006 Physics Education Research Conference Proceedings, American Institute of Physics.
11Goals on Process Knowledge TeamworkTeam Effectiveness (Imbrie)Team Developer (McGourty)Communications3-D VisualizationEtc.Leadership
18URMs 30% of population, few in labor force Women 50% of population, 10% in labor force# of BS Engineering Degrees% of Freshman Engineering Class19661977200219752001Under Represented Minorities1,915(4.8%)7,971(11.6%)17.415.8Women146(0.4%)1,961(4.9%)14,102(20.5%)19.918.3Chubin and Babco (2003), Walking the Talk in Retention to Graduation
19Engineering Disciplines Retention RatesEngineering DisciplineProportion WomenProportionURMTotal URMNon-URMChemical32.411.535.048.1Civil126.96.36.1994.2Computer8.847.658.7Electrical11.814.336.347.2Industrial29.417.751.3Mechanical12.011.038.048.7Total19.011.738.949.5Chubin and Babco (2003), Walking the Talk in Retention to GraduationNational Science Foundation. Women, Minorities, and Person with Disabilities in Science and Engineering: Arlington, VA, 2000 (NSF )
22Group QuestionsWhat are the known instruments/methods in each of the three areas? (Cognitive, Affective, and Rate Change)Where have these been used successfully/unsuccessfully?Where are the gaps in each area?
23Process One note taker One reporter Everyone discusses We will report out with 15 minutes left
24There is no failure. Only feedback. -Anonymous Happy grant writing! https://engineering.purdue.edu/SCI/Workshop
25AffectiveNational academy of engineering website with book of surveys
26Rate ChangeLongitudinal measurements, courses, rate changes of minorities, retention, enrollment, GPA for performance measuresUse Associate Dean’s offices as resources for stats, changes, etc.
27Wish Critical Thinking – one that is free Definition of engineering changes for students as they go through the curriculumDigital Systems within computer engineeringTeaming attitude by individualMeasuring how well the student can deal with an out of box situation.Clearinghouse that explains what already exists, how long is it, how it was used in the past, cliff notes version
28Wish ABET could sponsor (or someone) or post tools that are being used More CI https://engineering.purdue.edu/SCI/workshop/tools.htmlTools for writing and communicationLife-long learning tools (continuous learning)Procedural knowledge on engineering designFREE Critical Thinking
29WishMeta-cognitionSocietal context of knowledge, global competencies, service learning, research skills, discourse and collaborative dialogue, oral communication, problem solving, leadership (some instruments from Business schools), ability to integrate knowledge, longitudinal data, qualitative data analysisEvaluation of case studies
30Cognitive Math placement test English Placement test Concept Inventories https://engineering.purdue.edu/SCI/workshop/tools.htmlFE, GRE – standardized testsCritical thinkingCriticalthinking.org – has tools, and rubric developed by Gary Brown at Washington State, Watson Glazier (also not free), U of South Carolina EFFECTS (web-based)
31California Critical Thinking Collegiate Learning AssessmentCAAP – from ACTSALG – Student Assessment of Learning GainsIDEA – teaching assessmentsMSLQ – Motivated Strategies for Learning QuestionnaireNSF User Friendly Guide to Assessment
32Rogers and Sando, Rose-Hulman Learning Styles Instruments (Kolb, Felder)Career Instruments (Holland)FMRI, Caps measuring brain activityExperience sampling method – student wears a bracelet and when it goes off the student writes down what they are doing (engagement measures)Calibrated Peer reviewTeamwork effectiveness (Imbrie, Smith, Layton)National Center for Case Studies (Buffalo)