Outcomes based education: EnvE Outcomes based education: CONTINUOUS IMPROVEMENT Benno Rahardyan Environmental Engineering Study Program Faculty of Civil and Environmental Engineering Institut Teknologi Bandung

Goals Faculty efficiency and efficacy Improved student learning Knowledge Skills Attitudes Behavior Improved learning environments Accountability Accreditation

*EnvE

The Process of Continuous Improvement* *EnvE

Continuous Improvement Action Process Outcomes Course Assessment Course Questionnaire Exit Survey Employer Average a. Scientific Expertise 2.36 3.30 2.92 3.35 2.98 b. Experimental skill 2.62 3.28 3.06 3.18 3.17 c. Design Abilities 2.28 3.36 3.02 3.09 2.94 d. Teamwork 2.49 2.84 2.71 e. Problem Solving 2.54 3.73 2.99 3.22 3.31 f. Professional Responsibility  2.86 3.38 g. Communication Skill 2.93 3.54 3.04 3.27 h. Understanding of Engineering Impact 2.73 2.46 2.77 i. Long life Learning  2.85 2.47 3.50 3.08 j. Contemporary Issues 2.35 2.96 2.88 2.86 k. Technical Know-how 2.55 2.05 3.11 2.72

Performance Indicator Continuous Improvement Action Process 1 2 3 4 Student Outcome Performance Indicator 1-under expectation 2-below average 3-meet expectation 4-exceed expectation An ability to apply knowledge of mathematics, science, and engineering. Proficient in mathematics through differential equations, probability and statistics, calculus-based physics, general chemistry, relevant to the program study (earth science, biological science, fluid mechanics) 1. Describe mathematical, physical, chemical and biological relations for given facts/occurences in environmental engineering. Student can not describe and can not see any relation Student can describe but can not see any relation Student can describe and can see the relation Student can describe, see the relation and extend the potential impact of problem   2. Recognize basic engineering problems, identify the data, and classify the related knowledge for solution Student can not recognize, identity and classify the related knowledge of solution Student can recognize, but can not identity or classify the related knowledge of solution Student can recognize, identity and classify the related knowledge of solution Student can recognize, identity, classify the related knowledge of solution and also mention update invention. 3. Demonstrate application of differential equation, statistic, probability, calculus-based physics, general chemistry and other relevant science to solve the problem Student can not demonstrate solution of given problem Student can only mention several solutions without solving the given problem Student can demonstrate application to solve the given problem Student can demonstrate application to solve the given problem and also explain the interpretation of solution. Outcome A (EXAMPLE)

Comparison of Outcomes Measurements Student Outcome Assessment results Comparison of Outcomes Assessment Results Comparison of Outcomes Measurements Attainment standard of 3.00 Area of Improvement Area of Improvement Area of Improvement

Comparison of Outcomes Assessment Results Student Outcome Assessment results Comparison of Outcomes Assessment Results Comparison of Outcomes Measurement Results (2012 Data) Outcomes Course Assessment Course Questionnaire Exit Survey Employer Average a. Scientific Expertise 2.36 3.30 2.92 3.35 2.98 b. Experimental skill 2.62 3.28 3.06 3.18 3.17 c. Design Abilities 2.28 3.36 3.02 3.09 2.94 d. Teamwork 2.49 2.84 2.71 e. Problem Solving 2.54 3.73 2.99 3.22 3.31 f. Professional Responsibility  2.86 3.38 g. Communication Skill 2.93 3.54 3.04 3.27 h. Understanding of Engineering Impact 2.73 2.46 2.77 i. Long life Learning  2.85 2.47 3.50 3.08 j. Contemporary Issues 2.35 2.96 2.88 2.86 k. Technical Know-how 2.55 2.05 3.11 2.72

Closing the loop •What do the findings tell us now? •Did our “treatment” improve learning? •What else do the findings show? •What’s the next step? •What have we learned about our assessment process? Infrastructure? What can be improved?

ITB: Exit survey (Directorate of Education) Alumni survey (Career Center) User survey (Career Center)

Continuous Improvement Outcomes Assessment and Improvement Plan* Outcomes 2013/2014 2014/2015 2015/2016 2016/2017 2017/2018 d. Teamwork * 2018 Curriculum Preparation g. Communication Skill j. Contemporary Issues a. Scientific Expertise h. Understanding of Engineering Impact k. Technical Know-How f. Professional Responsibility i. Life-long Learning b. Experimental Skill c. Design Abilities e. Problem Solving Note: spesific observation scientific expertise: TL2101 Fluid Mechanics I, TL2104 Engineering Mathematics, TL2201 Fluid Mechanics II, TL2202 Environmental Chemistry, TL2203 Environmental Microbiology experimental skill: TL3103 Environmental Laboratory design abilities: TL3105 Water Supply Engineering,TL3202 Drainage & Sewerage,TL4101 Water Treatment Plant Design, TL4102 Wastewater Treatment Plant Design Teamwork: TL4201 Environmental Impact Assessment, TL4202 Project Planning Problem Solving: TL3101 Unit Operation, TL3102 Unit Processes, TL3203 Water Management Professional Responsibility: TL4098 Field Work Communication Skill: TL4098 Field Work, TL4099 Seminar and Final Task Understanding of Engineering Impact: TL2105 Environmental Health, TL2205 Environmental Epidemiology TL4201 Environmental Impact Assessment Long life Learning: TL4099 Seminar and Final Task Contemporary Issues: TL3104 Solid Waste Management, TL3201 Air Pollution, TL4103 Air Pollution Control Technical Know-how: TL2103 Engineering Drawing , TL3103 Environmental Laboratory *EnvE

Optimizing assessment of student work Daina Briedis, 2013

OUTCOMES IMPROVEMENT PLAN Program Outcomes Courses Improvement a. Scientific Expertise Action: maintain Quiz Assignment Book preparation Reading textbook material up dating Laboratory improvement b. Experimental skill   Action : maintain Field trip Guest lecturer c. Design Abilities    Better assignment Collaborative Learning Course Benchmarking Course Relevance Improvement Curriculum Development Evaluation General Indicators Faculty Capacity Building Improvement Students Project Improving Scientific Expertise Maintaining Class Attendance Maintaining Learning Process Performance Training ABET Accreditation

OUTCOMES IMPROVEMENT PLAN Program Outcomes Courses Improvement Long life Learning   Environmental Epidemiology Air Pollution Control Seminar and Final Task Action : maintain Note taking Problem based learning type assignment Collaborative learning type assignment Student centered learning collaborative learning j. Contemporary Issues Engineering Drawing Environmental Health Solid waste Management More case studies of problem Content enrinchment k. Technical Know-how Engineering Mathematics Unit Process Water Treatment Plant Design Air Pollution Monitoring Audio visual content Supporting equipment Laboratory improvement Availability of facility. ABET Accreditation

CURRENT STATUS POSSIBLE IMPROVEMENT EVALUATION Priority/Representing Courses CURRENT STATUS POSSIBLE IMPROVEMENT scientific expertise: TL2101 Fluid Mechanics I, TL2104 Engineering Mathematics, TL2201 Fluid Mechanics II, TL2202 Environmental Chemistry, TL2203 Environmental Microbiology experimental skill: TL3103 Environmental Laboratory design abilities: TL3105 Water Supply Engineering,TL3202 Drainage & Sewerage,TL4101 Water Treatment Plant Design, TL4102 Wastewater Treatment Plant Design Teamwork: TL4201 Environmental Impact Assessment, TL4202 Project Planning Problem Solving: TL3101 Unit Operation, TL3102 Unit Processes, TL3203 Water Management Professional Responsibility: TL4098 Field Work Communication Skill: TL4098 Field Work, TL4099 Seminar and Final Task Understanding of Engineering Impact: TL2105 Environmental Health, TL2205 Environmental Epidemiology TL4201 Environmental Impact Assessment Long life Learning: TL4099 Seminar and Final Task Contemporary Issues: TL3104 Solid Waste Management, TL3201 Air Pollution, TL4103 Air Pollution Control Technical Know-how: TL2103 Engineering Drawing , TL3103 Environmental Laboratory Outcome-related Courses

Course Assessment Plans for Improving Outcomes

IMPROVEMENT Course Portfolio + Lab development C.2 Learning Process Improvement C2.1 Maintaining Class Attendance C2.2 Maintaining Learning Process Performance C.3 Academic Atmosphere Improvement C.4 Course Relevance Improvement C.5 Advising Improvement C.6 Teaching Method Improvement-Improving Soft Skills C.6.1 Faculty Capacity Building C.6.2 Implementation of teaching method improvement C.6.3 Improvement of Communication Skill C.7 Improving Scientific Expertise C.7.1 International Guest Lecture Series C.7.2 Environmental Engineering Seminars Series CURRENT STATUS POSSIBLE IMPROVEMENT

Portofolio online Metode Perkuliahan Metode Evaluasi Sistem Penilaian Uraian terhadap Hasil Kuesioner dan Statistik Kelas Refleksi Pelaksanaan Perkuliahan Rencana Tindak Lanjut

Courses Weakness and Improvement action Continuous Improvement Weakness Evaluation and Improvement action Courses Weakness and Improvement action

Continuous Improvement The Result of Weakness Evaluation Related to Outcomes a) through d) Outcome Weakness Evaluation (Improvement plan) a. Scientific Expertise Generally, students understanding about course material is less than 75% Students still have difficulty to obtain literatures relevant with the course The class room facilities cannot optimally support the learning atmosphere. Method of teaching, improved supporting facilities, b. Experimental skill Lack of modern laboratory equipment and very limited in number Some of the laboratories are not clean and tidy enough Students are less accustomed to perform experiments independently Maintained c. Design Abilities The opportunity to practice about good design is still limited Students are lack of knowledge about the design process d. Teamwork The students have not been able to communicate effectively Increased opportunities in rehearse of team work, collaborative learning

Continuous Improvement The Result of Weakness Evaluation Related to Outcomes e) through k) Outcome Weakness Evaluation (Improvement plan) e. Problem Solving The students do not understand the learning material integrally Maintained f. Professional Responsibility Students having lack knowledge about environmental engineering professionalism g. Communication Skill Students are not used to state their opinion well Increased opportunities in group work presentations, collaborative learning h. Understanding of Engineering Impact Students’ knowledges on environmental engineering application are limited Introduction of more Guest lectures in the courses, site visits i. Long life Learning j. Contemporary Issues Students awareness about environmental related issues are low Students too much focusing on learning environmental engineering academically Guest lectures and site visits k. Technical Know-how Students’ knowledge and understanding about the engineering tools usage are low Students do not understand the engineering tools application More workshop sessions

Continuous Improvement Activities of Continuous Improvement Lab development C.2 Learning Process Improvement C2.1 Maintaining Class Attendance C2.2 Maintaining Learning Process Performance C.3 Academic Atmosphere Improvement C.4 Course Relevance Improvement C.5 Advising Improvement C.6 Teaching Method Improvement-Improving Soft Skills C.6.1 Faculty Capacity Building C.6.2 Implementation of teaching method improvement C.6.3 Improvement of Communication Skill C.7 Improving Scientific Expertise C.7.1 International Guest Lecture Series C.7.2 Environmental Engineering Seminars Series

Continuous Improvement Activities of Continuous Improvement C.8 Improving Contemporary Issues and Access to Modern Tools C.8.1 Invited Speakers C.8.2 Field Trips C.8.3. Field Work Student Activity C.8.4Training C.8.5 Laboratories Visit C.9 Students Competition C.9.1. Internal Student Competition C.9.2. External Student Competition C.10 Improvement of Evaluation Method C.10.1 Improvement of Course Portfolio C.10.2 Course Benchmarking C.10.3 Evaluation General Indicators C.11Curriculum Development C.12 Improvement Students Project

Continuous Improvement Course Z Prev Year Year-0 Year-1 ... Activities of Continuous Improvement Lab development C.2 Learning Process Improvement C2.1 Maintaining Class Attendance C2.2 Maintaining Learning Process Performance C.3 Academic Atmosphere Improvement C.4 Course Relevance Improvement C.5 Advising Improvement C.6 Teaching Method Improvement-Improving Soft Skills C.6.1 Faculty Capacity Building C.6.2 Implementation of teaching method improvement C.6.3 Improvement of Communication Skill C.7 Improving Scientific Expertise C.7.1 International Guest Lecture Series C.7.2 Environmental Engineering Seminars Series √ √ √ √ √ √ √ √ √ √ √ √ - √ √ EXAMPLE √ √ √ √ √ √ - √ √ √ √ √ √ √ √

Continuous Improvement Course Z Prev Year Year-0 Year-1 ... Activities of Continuous Improvement C.8 Improving Contemporary Issues and Access to Modern Tools C.8.1 Invited Speakers C.8.2 Field Trips C.8.3. Field Work Student Activity C.8.4Training C.8.5 Laboratories Visit C.9 Students Competition C.9.1. Internal Student Competition C.9.2. External Student Competition C.10 Improvement of Evaluation Method C.10.1 Improvement of Course Portfolio C.10.2 Course Benchmarking C.10.3 Evaluation General Indicators C.11Curriculum Development C.12 Improvement Students Project √ √ √ √ √ √ - - √ - √ √ - √ √

EXAMPLE OUTCOME G Continuous Improvement Teaching Method Improvement-Improving Soft Skills Improvement of Communication Skill Year Writing skills Presentation skills Moderator Questions Paper Summary Slide Poster Oral presentation -2005 Indonesian - Ind 2005 Eng. 2006 Ind.&Eng Eng (optional) 2007 2008 Eng (obligatory) 2009 2010 Process-oriented Improvement Improvement in Final Project Seminar EXAMPLE OUTCOME G Result/Output-oriented Improvement English Proficiency Score (predicting TOEFL score)

Recommendation summary General improvement scheme Feedback information delivery General continuous improvement schedule Improvement online portfolio Program development General improvement action