Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations.

Similar presentations


Presentation on theme: "1 When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations."— Presentation transcript:

1 1 When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners. Oosterhof, et al. (2008) Assessing Learners Online

2 2 Donald Staub, Title III Director Carteret Community College ? ? ? ? ? ? Assessing Distance Learning: Are they learning what we think were teaching?

3 3

4 4 The Lineup CCC Why Worry about DL Assessment? How Were Doing It The DL Program DL in Programs & Courses Possibilities & Challenges

5 5 CCC 33 CCC Enrollment ~1800 DL is 10+ years old Spring total sections 261 seated sections 90 Internet (23%) 40 Hybrid (10%) 8 Web-based 5

6 6 Why worry about assessing DL? Student Success Persistence (btw: success vs. retention vs. persistence) Good assessment guides good the College & Program-Level:

7 the Course-Level: Generalizability (c.f. Oosterhof, et al, 2008) …the degree to which learner performances that are observed generalize to performances that are not observed. [i.e. are they learning what we think were teaching?]

8 8 So, how are we doing this? Caveat: Were not saying this is right or wrong…its just the way that were currently approaching it.

9 9 Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction. (CS 3.3.1) COC Distance Education Policy Statement

10 10 The (former) Guidelines

11 11

12 Strategic Plan We will create a culture where all institutional actions are focused on improving student learning and success and leading to student goal completion. Q1: Is the distance education program reflected in the institutions strategic (or master) plan? PM 16Success and Withdrawal Rates by Instructional modalities

13 13 Q2: Is there evidence that outcomes for the program have been identified? DL-AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. DL-AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.

14 14 AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. Assessment: Quality Assessment Plan, Success/ Persistence Rates, Student Surveys/Evaluations. AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC. Assessment: The number of internet/hybrid sections offered. Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program?

15 15 QAP

16 16 The QAP (Quality Assessment Plan) Developed in-house (using models) Peer-reviewed by content specialist Stipends (T3) All current online courses have been reviewed All new online courses must be reviewed/ certified

17 17 The QAP Score Card

18 18 The QAP Score Card: Learning

19 19 QAP Success/Persistence

20 20 Success - Spring sections seated: 75% success (persist: 87.1%) internet: 66% success (persist: 81.7%) hybrid: 73% success (persist: 87.8%) FA09 internet: 62 sections; 68% success FA08 internet: 61 sections; 67% success FA07 internet: 59 sections; 66% success FA06 internet: 48 sections; 66% success 20

21 21 QAP Success/Persistence Program/Course Evaluations

22 22 Surveys & Evaluations: DL Program

23 23 Surveys & Evaluations: DL Program 23

24 24 Surveys & Evaluations: Course-Level

25 25 Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? Modality Comparison Grade Distributions Program Reviews ILLOs (Gen Ed outcomes)

26 26 Modality Comparison

27 27 Modality Comparison Institutional-level

28 28 Modality Comparison Program-level

29 29 Modality Comparison Course-level

30 30 Modality Comparison Grade Distributions

31 31 Grade Distributions institution-level %s

32 32 Grade Distributions course- level

33 33 Modality Comparison Grade Distributions Program Reviews

34 34 Evaluate Instructional Modalities

35 35 Modality Comparison Grade Distributions Program Reviews ILLOs

36 36 ILLOs (Gen Ed outcomes) 1. Communication 2. Computer Literacy 3. Critical Thinking 4. Humanities & Fine Arts 5. Information Literacy 6. Personal Growth & Responsibility

37 37 Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment: Final Exam in CIS 110.

38 38 Computer Literacy - CIS 110 Section analysis

39 39 Computer Literacy - CIS 110 Same instructor teaching one section of each

40 40 Q4: Has the institution developed student learning competencies for the courses/programs offered by distance education? If these are the same competencies for courses/programs offered by traditional methodologies, is assessment identified for distance learning students separate from students taking courses by traditional methodologies?

41 41 Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction. So, are they learning what we think were teaching?

42 42 Yes! … No?…maybe?? PLLOs CLLOs And the answer is…

43 43 PLLOs

44 44 PLLOs (SLOs) Same instructor teaching one section of each (common assignment from each section) Business Administration ACC 120 BUS 110 ECO 251/252

45 45 PLLO: preparation of financial statements as part of mid-term Assessed in: ACC

46 46 BUS 110 ECO 250

47 47 PLLOs CLLOs

48 48 Course-level assessment - CLLOs ACC 120 ART 111 EDU 119 ENG 111 HIS 111 PSY 150 & 241 RCP 114 SOC

49 49 Course-level assessment ~ CLLOs BUS 110 … Students will be able to: A: Explain Business Ethics and Social Responsibility B: Describe the major concepts of business, such as Entrepreneurship, Management, Marketing, Human Resource Management, and Financing C: Describe Business Globalization

50 50

51 51 Use of Results: The on-line section needs some extra material to reinforce or explain globalization.

52 52 Students placed more value on the CMS as an effective learning tool when they perceived the CMS as supporting interactivity among student, instructor, peers, content. (Chou, Peng, Chang, 2010)

53 53 ART 111

54 54 A - ( ): - You posted 3 or more times and you completed the Assignment with a well thought through response. (The more you post MEANINGFUL responses over 3 the higher your grade. Though welcome and expected, please understand that a One or Two sentence "feel good" post such as (... Great response Michaela!!) or (... I love how you put the meaning and then went down and talked about them. it made it easy to understand!!) will count for very little towards your grade. 54

55 55 HIS 111 Essays (online) vs. Research Paper (seated) … same rubric

56 56 RCP 114

57 57 Other Possibilities Downstream assessment E-ports 57

58 58 Challenges

59 59 I got rid of [the discussion board]…its a pain in the ass. I have to pull teeth to get them to post even when they get a score Value of discussion board is suspect at best.

60 60 My first goal is to get the instructors to agree on common CLLOs. Then well work on common assessments. I dont know [what texts and assessments the other instructors are using in this multi-section course] Outcome-Assessment alignment

61 61 Donald Staub Title III Director Carteret Community College


Download ppt "1 When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations."

Similar presentations


Ads by Google