1. Bieber et al., NJIT ©2004 1 Internship/Ph.D. Opportunity PLA: Participatory Learning Approach Michael Bieber Jia Shen, Dezhi Wu, Roxanne Hiltz Information Systems Department College of Computing Sciences New Jersey Institute of Technology http://web.njit.edu/~bieber http://web.njit.edu/~bieber - http://is.njit.edu/pla/http://is.njit.edu/pla/ September 2004

2. Bieber et al., NJIT ©2004 2 Motivation To increase learning of course content Learning through active engagement –involve students as active participants –with the full problem life-cycle –through peer evaluation Minimize overhead for instructors

3. Bieber et al., NJIT ©2004 3 PLA Process (e.g.,Participatory Exam) Each student creates 2 exam problems Instructor edits the problems if necessary Each student solves 2 problems Students evaluate (grade) the solutions to the problems they authored, writing detailed justifications Ph.D. students evaluate each problem a second time Instructor gives a final grade optional: Students can dispute their solution’s grade, by evaluating it themselves and writing detailed justifications Instructor resolves the dispute All entries posted on-line

4. Bieber et al., NJIT ©2004 4 Exam Process Control Assign ID Edit questions Assign who answers questions Assign level-2 graders Course Design Determine Final Grades Set up on-line environment Dispute final grade Level-1 and Level-2 graders grade solutions Make up problems Read - other problems - other solutions - grade justifications - disputes Solve problems Instructor Control ProcessStudent Learning Process Resolve Disputes Process Flow: Learning from doing the PLA activities additional learning from reading everything peers write

5. Bieber et al., NJIT ©2004 5 Exam Process Control Assign ID Edit problems Assign who solves problems Assign level-2 graders Course Design Determine Final Grades Set up on-line environment Dispute final grade Level-1 and Level-2 graders grade solutions Make up problems Confirmation ID, understand process Read - other problems - other solutions - grade justifications - disputes Solve problems Instructor Control ProcessStudent Learning Process Resolve Disputes

6. Bieber et al., NJIT ©2004 6 Enjoyability (2 semesters with CIS677) QuestionsSAANDSDMeanS.D.# I enjoyed the flexibility in organizing my resources 26.2%48.9%16.7%3.6%4.6%3.881.00221 I was motivated to do my best work 23.5%42.9%28.2%3.4%2.1%3.82.92238 I enjoyed the examination process 17.2%42.3%22.6%10.5%7.4%3.511.13239 SA - strongly agree (5 points); A - agree (4); N - neutral (3); D - disagree (2); SD - strongly disagree (1); the mean is out of 5 points; S.D. - standard deviation Cronbach’s Alpha=0.68

7. Bieber et al., NJIT ©2004 7 Perceived Learning QuestionsSAANDSDMeanS.D.# I learned from making up questions 17.9%42.5%21.3%13.8%4.5%3.551.08240 I learned from grading other students answers 17.7%48.1%19.4%9.3%5.5%3.631.06237 I learned from reading other people’s answers 15.8%45.0%22.1%11.3%5.8%3.541.07240 I demonstrated what I learned in class 13.6%50.2%22.6%10.9%2.7%3.61.95221 My ability to integrate facts and develop generalizations improved 21.8%49.2%25.6%2.1%1.3%3.88.83238 I learned to value other points of view 17.6%51.9%27.6%1.3%1.6%3.82.81239 I mastered the course materials 7.4%51.6%31.4%6.9%2.7%3.54.84188 Cronbach’s Alpha=0.88

8. Bieber et al., NJIT ©2004 8 Recommendation: Do Again! QuestionSAANDSDMeanS.D.# Would you recommend in the future that this exam process used? 20.7%40.1%24.5%8.9%5.8%3.601.10237 Similar results for CIS365: undergraduate file structures course using short essay questions (Fall 2002)

9. Bieber et al., NJIT ©2004 9 PLA Environment Software Guide the process Form groups Assign problem solvers, evaluators, dispute arbitrators On-line templates to ensure full entries Guide people to post entries in correct place Incorporate group process tools Handle problems as much as possible –Remind people who are late –Reallocate who does what Based on a workflow management tool…

10. Bieber et al., NJIT ©2004 10 Extensions: Activity Support Can we build in tools to support each activity –How to construct a problem –How to research and construct a solution –How to evaluate a solution –How to resolve a disputed evaluation –Technical term: scaffolding (educational literature) –Initial approach: inquiry-based support (ed. literature)

11. Bieber et al., NJIT ©2004 11 Extensions: Full Collaboration Groups for: –Problems, solutions, evaluation, dispute arbitration Requires group process support –Process: work on each activity together or separately, internal review –Grading of individual group members –Process Tools: brainstorming, voting, etc.

12. Bieber et al., NJIT ©2004 12 Extensions: Scope Which activities? –so far: exams –what about: quizzes, homeworks, larger projects, in-class projects Which problem types? –so far: short and long essay questions –what about: multiple choice, short answer, computer programs, semester projects –Sub-problems: computer program design & implementation semester project outline & execution

13. Bieber et al., NJIT ©2004 13 Extensions: Scope, cont. Course Level –Graduate, undergraduate, secondary school (high school, junior high) Disciplines –IS/IT, business, science, engineering, humanities, medical, all of secondary school

14. Bieber et al., NJIT ©2004 14 Extensions: Scope, cont. Degree of Evaluation (assigning grades) –Currently: solutions –What about: quality of problems quality of evaluations/grades –All could be disputed Degree of Participation –students could evaluate each –students could arbitrate disputes

15. Bieber et al., NJIT ©2004 15 Internship/Ph.D. Opportunity Literature review and suggestions for all of the extensions For more information on PLA see http://is.njit.edu/pla/ http://is.njit.edu/pla/