Presentation on theme: "Experiences on Course Development Practices in a Power System Analysis Master Course a.k.a. tales of a journey through the valley of death in providing."— Presentation transcript:
1Experiences on Course Development Practices in a Power System Analysis Master Course a.k.a. tales of a journey through the valley of death in providing a high quality learning environmentAssoc. Prof. Dr. Ing. Luigi Vanfretti and Maxime BaudetteKTH SmarTS LabKTH Scholarship of Teaching and Learning Stockholm, March 12th 2015.
2AcknowledgmentThere has been a lot of former students involved in this efforts, through teaching assistant duties and beyond, which we can’t list here – but you know who you are, and we thank you very much! A very special thanks is expressed to Mostafa Farrokhabadi, who was the first student helping with the research work related to this talk until Maxime Baudette got involved.
3Outline Motivation Constructive Alignment Theory Implementation: T&L activities, Assessment ActivitiesEvaluation of the implementation: Feedback, R-SPQ-2F, Ranking AlgorithmFiltering Feedback: Repertory GridIntroducing Peer Instructions with Clickers
4EG2100 in Brief Introductory course on Power System Analysis 6 ECTS (7.5 ECTS before 2014)Given on 2 periods (September to December)About 85 studentsThe course is divided in five important topics:Modern Power Systems with an Introduction to Sustainable Energy Technologies and Smart GridsFundamental Principles for Power System Analysis AC circuitsElectrical Modeling of Generators, Transmission Apparatus and NetworksMethods for Analysis and Design of Power Networks in Steady State and Unbalanced OperationSteady State Stability Analysis using the Power-Flow FormulationMethods for Analysis of Power Distribution Systems in Steady State
5“I never teach my pupils, I only attempt to provide the conditions in which they can learn.” A. EinsteinMotivationThe previous course (until 2010) that was replaced by this one had several drawbacks in its pedagogic design and material:The course material was outdated, inconsistent and lacked appeal.No teaching method was used to encourage the learning process The failure rate of the course was very highThese drawbacks required a complete overhaul of the course: from preparing new material, to the design of a new implementation using teaching methods.
6How to address a large variety of background? Students’ BackgroundStudents come with a very different background, and often lack any exposure to power systems at all:Mandatory for M.Sc. Program in Electrical Power Engineering (TELPM), and M.Sc. Joint Program in Smart Electrical Networks and Systems (SENSE)Elective in other M.Sc. Programs at the school of Electrical Engineering and some Erasmus Mundus M.Sc. ProgramsAlso offered to power and energy professionals within industry collaborations and PhD students (including those from other universities)This variety in the students’ background makes it challenging to design well aligned and adequate T&L activities and proper assessment methods.
7Redesign of the Course - Goals In 2011 it was decided re-design the course to achieve the following goals:Increase the success rateLargely increase the depth and moderately the coverage of the course contentLink the course to the current developments in the industry and academia, including the “Smart grid” conceptRe-Design Approach:Implement the constructive alignment theory (CAT) as the didactic teaching approach of choice for the courseAdopt the consensus-oriented decision-making (CODM) model to design the courseDevelop effective multiple feedback channels to gather the students’ perception and select relevant feedback
8Constructive Alignment – Basic Principles (Biggs and Tang) The theacher’s fundamental task is to get students to engage in learning activities that are likely to result in their achieving the desired outcomes in a reasonably effective manner.What the student does is more important in determining what is learned than what the teacher does.Constructive Alignment Theory:Provides a set of principlesThese principles can be used to devising Teaching and Learning Activities (T&Ls)The activities can help in achieving the Intended Learning Outcomes (ILOs).
9Using Principles of Constructive Alignment CourseObjectives(Intended Learning Outcomes)Teaching and Learning ActivitiesAssessmentWhat should the student beable to doas a result of the course?Using the principles from constructive alignment:Start by carefully aligning T&LAs and AssessmentsThese activities should to support the students to fulfill the ILOs.The students role (MAJOR):The students use the activities to construct their knowledge and achieve desired outcomes.The teacher’s role (minor):To design a learning environment that encourages the student to perform the T&Ls that aid the students to construct their knowledge.What should the students do to demonstrate that they reached the objectives?What work is appropriatefor students to doto reach the ILOs?
10Variety of Background – Affects the proper implementation of CAT Question% of correct answers1992403904705536847598629691027113212131415771617671863192074214422682351Total64.8Variety of Background – Affects the proper implementation of CATAffects student’s readiness to engage in the Teaching and Learning Activities Diagnostic test results show a low average in the students preparation Fundamental knowledge varies a lot.
11CODM – Design Process (Hartnett) Links the design group to the consensus model.Five 2-hour sessions (10 hrs)Allows to deliver a shared proposal:Important if the design group will also be the implementation group (needed commitment).ClosingFraming the ProblemOpen DiscussionIdentifying Underlying ConcernsDeveloping ProposalsDeveloping a Preferred SolutionChoosing a Direction
12Design Rule for CAT Implementation Example of using the consensus based model Step 1 (Framing the problem): design rule used for one of the T&L activities using Cause-and-Effect analysis.
14T&L Activities - Lectures Lectures are intended to develop conceptual understanding:
15T&L Activities- Invited Lectures Invited Lectures are intended to develop the interest of students by providing insights from the industry and top researchers from Academia.Sample of Invited LecturesMagnus Danielsson. System Design, R&D, Net Insight AB, Stockholm, Sweden.Lecture on Communication Issues in Smart Grid ApplicationsSvein Harald Olsen, Statnett SF, Oslo, Norway.2 Lectures on IEC CIMSonja Berlijn, SVP R&D, Statnett S, Oslo, Norway.Lectures active R&D projects (Lean Line + Voltage upgrading)David Petesch/Yannick Fillon, RTE - National Center for Grid Expertise, Paris La Défense – FranceLecture on EMTP and Real-Time Simulation at RTEProf. Federico Milano, UCDChallenges for power system modelling and simulation
16T&L Activities- Daily Reading Quiz, Conceptual Quiz In total, they are worth 10 BONUS points of the final grade.Reading Quiz consists of some very basic questions about the content of the lectureThe questions are designed in a way that a student should go through the lecture content before arriving to the lecture!Peer Instruction ImplementationConceptual Quizzes asked about the lecture content during the lecture.The questions addressed different learning levels, focusing on developing deep learning. They sometimes involved a small amount of numerical calculations.The students answered the questions using a remote clicker.
17T&L Activities – Homeworks For the students to practice the methods and study the concepts covered during the lectures. There are 3 Homeworks (30 points) covering the contents learned in the respective lectures. The homework are aimed to prepare the students for the assessment tasks (tests and final exam). In 2014 computer assisted assignments were introduced.ClassroomHome!
18Assessment – Tests and Final Exam 2 Tests during the semester .Final examThe tests and exam were built in two parts, with theory questions in the first part and problems in the second part. The problems were similar to those solved in the homework.The assessment of the course is done though the different graded activities of the course.All the grades are added to give a total grade on 100 pointsTotal PointsGrade Letter94 – 100A87 – 93B81 – 87C74 – 80D65 – 73E60 – 64Fx00 – 59FCourse ActivityPointsDaily Quiz10 (Bonus)Homeworks30TestsFinal Exam40Total100 (+10 Bonus points)
19Evaluation of the implementation Three Course EvaluationsOne after each test (end of P1 and end of P2)One after the final examR-SPQ-2F Evaluation (at the beginning of the course)Two different question types (surface and deep approach)Students are graded using their answersThe results will reveal the students approachInterviews using the Repertory Grid TechniqueA form of structured interview to find out a participant’s preference on a given topic and the way these preferences are ordered on a rating scaleAllows to link qualitative to quantitative feedback.
20Explanation on KTH Social Quantitative DataInstructions on FormQualitative Data
21Sample Feedback using Google Forms Quantitative Feedback Qualitative Feedback
22Course EvaluationsAfter 2011, the form was improved for including an introductory part with instructions, explaining the students the grading scale they should use.
24Course Evaluations - YOY improvements The feedback provided by students that retook the course shows that the changes improved the course.The acceptance of the changes comes after the students realize that it has a positive impact on their learning.Note: very reduced population for statistical analysisSample QuestionIs the new course structure (lectures, daily exercises, weekly exercises, and weekly tests) preferable to the previous course structure (lecture, assignments, and final examination)?YOY = year over year
25Activity No. 1 – Design of a course evaluation process Instructions: Discuss in small groups (no larger than 3) the following questions. Using your answers, propose a design to evaluate a “sample” course.What feedback do you need for your particular course?T&L activities: do they contribute to the student’s understanding? Do the T&Ls have an impact on the Assessment Activities?In what form do you want it and how much?Quantitative, Qualitative, BothHow do you gather it in an efficient way?What tools do you use.How often do you gather it?Number times and type of feedback to be gathered – (a) that allows to trace changes, (b) of new information (originating from the changes).How do you use it?How do you show the students that their opinion is considered.
26The Revised two-factor Study Process Questionnaire: R-SPQ-2F Allows to determine the students’ learning approach The process is exposed to some drawbacks!
27R-SPQ-2F: A new ranking algorithm Surface LearningDeep Learning1st Quarter3rd Quarter2nd Quarter4th Quarter
28Grade distribution and its correlation with students’ learning approach
29Filtering Feedback: Repertory Grids The repertory grid method is a “person-centered” approach that allows the student to develop their own constructs using their own perception of the subject. Repertory grids serve as a feedback mechanism where the students can assess course elements using well-defined constructs and considering different scales. The construction of each grid allows a methodological analysis of the grids to identify constructive feedback to improve the course: a formal quantitative method to select and discard feedback.Good FeedbackFurther AnalysisBad FeedbackDiscardGood GridBad Grid
30Implementation of repertory grids The elements of the grid were determined by the T&L activities.The constructs were built from interviews with some studentsA small group of students (about 10%) was carefully selected to represent the different learning approachesEach student was interviewed individually about the different elements (qualitative data collection)The constructs were identified by analyzing all interviewsAn empty grid was sent to each student who should fill it in
31Repertory grid analysis Different software provide methods for analysis of the grids (Rep IV, Idiogrid), such as:Principal Component Analysis (PCA): allows better interpretation of feedback.Cluster Analysis: re-orders elements in a tree, similar objects are placed together. Allows to interpret the feedback.Descriptive analysis (Mean, Mode)PCA:In Fig. 2, the position of the elements and constructs is a goodrepresentation of their correlation. For example, this student believesthat the elements “daily exercise” and “class test solution”are similar in the case that both are stressful and should be removedfrom the course design. Note that these analyses add nonew data to the repertory grid, but simply help in the interpretationof the grids rated by the students.Cluster Analysis:For example, it can be observedthat in the first level, there are twin branches of “course textbook—lecture slides,” “Lennart’s lectures—Luigi’s lectures,” and “binder—daily exercises.”This is quite logical as these elementsshare common characteristics. Cluster analysis revealsthose elements and constructs that are in tight correlation; thiswill be valuable in modifying the course. For example, thisstudent believes that constructs of “time consuming—stressful”are in the same group. Other feedback or PCA analysis can bescrutinized to identify those elements that this student sees asstressful. Then, a decision can be made as to whether to modifythe timing of those elements.
32Activity No. 2 – Feedback Analysis Instructions:Assume that you have a high volume course (many participants). In this activity, you should analyze the method for feedback analysis presented, and propose alternatives to relate the students to their learning approach and classify which feedback to address in a systematic (reproducible) way.Are there other alternatives (methods) to analyze the course feedback and relate it to the student’s learning approach? How do you check if your implementation of CAT is working?How can you use the proposed method to filter the different feedback and choose which to address?
34Peer Instruction and the use of “Clickers” Reading Quiz and Conceptual Quiz: To implement peer instruction there is very good technology availableThese are called clickers and are used for gathering students feedback in real-time.Clickers were given to each student and kept by them throughout the course.
35A flexible tool for different uses! In the EG2100 course, clickers are used for the following:Unique identifier for each studentAttendance registration during LecturesPre-requisite test at the beginning of the course (not graded)In-class questions:Tests on the mandatory pre-readings (5 questions)Questions during the lecture (~5 questions)Rewarded by bonus points
36In-class interactions Tests on the mandatory pre-readings (5 questions)Questions about the content of the pre-readingsRegular questions, all done in a rowQuestions during the lecture (~5 questions)Questions about the subject being taughtRegular questions, often one by oneIf “average results” are obtained, students are invited to discuss with their neighbors and answer the question once again (peer instruction)If “bad results” were obtained, the lecturer re-explained the material.
37Preparation of Teaching Materials for Peer Instruction Material has to be prepared to introduce the peer instruction as a change of activity.The lectures need to be prepared in min modulesQuestions need to be prepared to assess deep learning (or surface when needed)Instead of a review at the end of the lecture, peer instruction allows for review of deep learning concepts during the lecture.This has a cost: Your time! – Think if you can afford it before doing it!
38Pros/Cons Pros: Greatly appreciated by the students: Higher attendance and better preparation of the studentsBetter in-class attentionHelps the teacher to get direct feedback (e.g. unclear notions)Flexible !Cons:Additional Data ManagementTechnical issues may happen in class (crash, bad reception, empty battery, etc.)Different lecturing style (might be hard to accept by students in early phases)
39Student’s feedback about the clickers The feedback received is very positive and encouraging for continuing this peer-instruction implementationThe mandatory pre-readings helped your understanding of the course material.It was helpful to discuss your answers with your classmates (peer instruction)The daily quizzes helped you keeping up to date with the course materials.It was fun to use the clickers. It helped you be more attentive in class
40Activity No. 3 – Peer Instruction Method Using Clickers Instructions: Assume that you have already your teaching prepared from previous course deliveries. In this activity, you should analyze how to modify the course materials to adapt it to the peer instruction method using clickers. Discuss the following questions, provide a proposal on how to implement peer instruction and an example slide for your particular course?At what learning depth level do you intend to use peer instruction?When using the peer instruction method, what factors do you need to consider so that you can use clickers to obtain quick feedback of the student’s understanding of a particular topic?How do you transform (timing, amount of material and content) your course material to adapt it to the peer instruction method?
41Highlights and Words of Caution A wider variety of stakeholders can participate in CODM process – this would allow for a very good course design addressing different points of view!CAT is a good vehicle to enhance the students’ learning. It’s implementation has to be considered carefully. Implementing CAT is extremely expensive – DO NOT BELIEVE the propaganda that says the contrary!R-SPQ-2F new ranking algorithm efficiently classifies the students’ learning approach depth. Useful for sampling feedback from students with different learning approaches.The two main challenges for the course designers are:To continuously modify the course so the surface approach students move toward deep approach.To do carry out (1) when there is absolutely no benefit to the teaching staff (a lot of work that brings no reward)
42Discussion (1/2) – Continuity and Effort The teaching staff changes from year to year depending on the availability of PhD students (Luigi is always involved and the only one to provide continuity)Course planning based on consensus requires continuity, in order to focus on improvements instead of challenging the whole structureEffort: Workload and TimeThe T&L activities require a large workload for the teaching staff to prepare and correct the assignments (very expensive and not properly funded)Course planning based on consensus requires time and involvement from the teaching staffThe feedback process requires a lot of time for gathering and analysis of the resultsIntegration of clickers in the lectures requires a lot of time (very expensive!) in order to provide a good experience for the students (questions should be well organized and well formulated)
43Discussion (2/2)Systemic factors: brace yourself to a large inertia and resistance to changeStudents and other faculty will oppose different didactic approaches to those that they are used too – you will face a large resistance to change.Not having a “single exam” to be able to pass the course has been criticized and brought up to the program coordinator year after year…This attitude is hard to change in the 4th year. However, we have shown that the acceptance of the students improves over time when they are able to reap the benefits of a good learning environment.You will waste your time with complaints from the student union, program coordinators, department head… just for trying to do things differently (I would say BETTER!)There is no means to enforce student feedback at KTH, this weakens statistical analysis and the amount of samples.Changes in rules or flexibility is needed if KTH wants to take education seriously.No pain, No gain? More like lot’s of pain, and no gain!The one most important thing we have learned is that there is no reward for good teaching in KTH – do it at your own risk and for your own self development.The only benefit you could obtain, by investing even more time, is to try to generate some publications for your teaching portfolio (in case you want to find somewhere else to teach with quality).
44Epilogue – A tale just like that of the one ring Even though this has been a wonderful adventure in teaching and learning, all good things must come to an end.Luigi can’t afford to continue to subsidize the delivery of the course by using time from his PhD students (which translates to use the funds from research projects!) to deliver the course.A loss of SEK was generated in 2014.A good learning environment comes at a cost: we do not get paid enough to deliver a high quality learning environment.Luigi will not longer be the responsible for this course nor involved in it, and with that goes the 4 years of experience…But at least we enjoyed the walk through the valley of death, and we have learned a lot about how to deliver a high quality learning environment!
45Thank you!“It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them.” — Steve Jobs Questions? Good luck with the rest of your day!
46References L. Vanfretti and M. Farrokhabadi, “Consensus-Based Course Design and Implementation of Constructive Alignment Theory in a Power System Analysis Course”, European Journal of Engineering Education, https://dx.doi.org/ /  L. Vanfretti and M. Farrokhabadi, “Evaluating Constructive Alignment Theory Implementation in a Power System Analysis Course through Repertory Grids”, IEEE Transactions on Education, vol. 56, no. 4, pp. 443 – 452, Nov DOI:  L. Vanfretti and Federico Milano, “Facilitating Constructive Alignment in Power System Engineering Education using Free and Open Source Software,” IEEE Transactions on Education, vol. 55, no.3, August 2012, pp DOI:  S. Taylor, “Managing postgraduate research degrees”. In H. Fry, S. Ketteridge and S. Marshall (eds) The Effective Academic: A Handbook for Enhanced Academic Practice. London: Kogan Page.  S. Taylor and N. Beasley. A handbook for doctoral supervisors. Routledge, New York, 2005.
47Course Objectives (Intended Learning Outcomes) After completing the course, the students should be able to: Using first principles derive the basic concepts and methods used for power system analysis. Describe the methods used for modeling and simulation of different power system phenomena, andExplain how computer methods affect the modeling processConstruct mathematical models for computing the steady state performance, and basic unbalanced performance of power systems.Describe the use of the Unified Markup Language for data modeling of power system components, and interpret the Common Information Model and its relation with the power system models used in class.Describe, derive, and compare different models of the most common equipment used in power network models.Using different methods, to compute, analyze, and reflect on the performance of a power system under steady state and unbalanced operation, for transmission and distribution networksApply a power system and general purpose software tools to analyze the performance of a power system, and evaluate it for different operating conditionsContrast different software development methods that are used in the creation of power system analysis softwareUsing first principles derive the basic concepts and methods used for power system analysis.To construct data models and mathematical models for computing the steady state performance, and basic unbalanced performance of power systems.To derive, describe and compare different models of the most common equipment used in power network models.Using different methods, to compute, analyze, and reflect on the performance of a power system under steady state and unbalanced operationTo describe basic characteristics of renewable and distributed energy sources, as well as monitoring, analysis and control technologies used in Smart Grids.Keep it in case of questions
48Evaluation Diagnostic test at the beginning of the course (non graded) Raise student awareness about their individual level of required backgroundIn-class questions are rewarded with a very flexible bonus point systemTechnical issues require to have a flexible systemAny evaluation activity requires to maintain a database of students