1. 1

2. 2 Evaluation Report A preliminary report to the faculty and administrators of the online distance learning program in the Department of Educational Research in the College of Education at Florida State University. April, 2002 There is no royal road to learning Euclid, 300 B.C.

3. 3 Agenda I.Evaluation Team II.Description of the Students III.Procedures Used IV.Results V.Conclusions VI.Preliminary Recommendations VII.Reactions – Discussion and Written

4. 4 Key Evaluation Results 1.Overall, How Would You Rate the Program? 2.Overall, How Would You Rate the Curriculum? 3.Overall, How Would You Rate the Faculty? M = 3.29 SD =.68 M = 3.29 SD =.72 M = 3.49 SD =.68 (4 = Excellent)

5. 5 I. The Evaluation Team

6. 6 Michael Simonson, Ph.D. Evaluation Coordinator

7. 7 Judy Converso, Ph.D. Comparisons With Similar University Programs

8. 8 Maggie Crawford Information Specialist Telephone Interviews

9. 9 II. The Students

10. 10

11. 11 Ages of Students Responding Median = 41

12. 12 Median = 6 Number of Online Courses Taken + Taking

13. 13 Hours/Week Studying – One Course Median = 10 hours/week

14. 14 III. Procedures 1.Student Survey (online) 2.Telephone Interviews 3.Faculty Interviews and Meetings 4.Literature Searches and Program Comparisons 5.Lurking

15. 15 IV. Procedures V. Conclusions

16. 16 1. Student Survey Response = ~70%

17. 17 1. A. Are You Satisfied with Your Courses and Curriculum? 1.Grading Requirements?13.3 2.Required Course Content?13.2 3.Required Course Selection?13.2 4.Course Difficulty?13.2 5.Elective Course Content?103.1 6.Relevance of Courses?23.1 7.Online Teaching?23.0 8.Elective Course Selection?102.9 Not SatisfiedMean N = 41 4 = Excellent

18. 18 1. B. Are You Satisfied with Your Instructors? 1.Content Knowledge?13.6 2.Feedback?33.2 3.Skills at Online Teaching?13.1 4.Interaction?32.9 Not SatisfiedMean N = 41 4 = Excellent

19. 19 1. C. Are You Satisfied With? 1.Number of Required Courses?23.4 2.FSU Technical Support43.2 3.Prep. For Profession?63.1 4.Portfolio Experience?103.1 5.Number of Electives?53.1 6.FSU Library Support?92.9 Not SatisfiedMean N = 41 4 = Excellent

20. 20 1. D. General Questions: How Would You Rate: 1.Faculty?3.5 2.Program – Overall?3.3 3.Curriculum?3.3 4.FSUs Support?3.3 5.Texts and Printed Materials?3.1 6.Program Support Services3.0 Mean N = 41 4 = Excellent

21. 21 Open-Ended Questions-Student Survey What Improvements Would You Recommend? More electives More teaching and less discussion More courses each semester Improve Blackboard Eliminate discrepancies in course policies

22. 22 Open-Ended Questions-Student Survey What Do You Like Best? Content Faculty Reputation of the program Freedom Flexibility Relevance

23. 23 Open-Ended Questions-Student Survey Why Is Your Favorite Course Your Favorite? Quality interaction with the instructor Excellent professor Interesting Content Various instructional Strategies used Relevant content

24. 24 Open-Ended Questions-Student Survey How Likely Would You Be To Choose An Online Program, Given Free Choice? Almost evenly split between those who prefer face-to-face and online learning

25. 25 Open-Ended Questions-Student Survey Other Comments ? Do something about Blackboard Offer courses in the summer Provide more interaction with faculty

26. 26 2. Telephone Interviews 10 Students Called ~ Randomly 5 Females and 5 males (30 Females/18 Males) Age = ~45 15 – 45 Minutes Each 3 Questions

27. 27 Summary Statements – Telephone Interviews Why did you enroll? Reputation of the program Convenience Content Skill Building

28. 28 Summary Statements – Telephone Interviews Major Program Strength? Faculty Organization Flexibility Students

29. 29 Summary Statements – Telephone Interviews What Changes Would You Recommend? Get a better system than Blackboard Improve consistency of policies between courses More immediate feedback Help us connect with other students Be realistic about time commitments Consider synchronous instruction

30. 30 3. Faculty Interviews 7 Interviews Face-to-face and Telephone 30 – 60 Minutes

31. 31 Summary Statements Faculty Interviews Instructional Systems Design is the key to the program Rigor and quality are critical The program is very labor intensive Synchronous instruction should be considered Various instructional platforms should be available Identify program benchmarks Provide training in distance education for faculty Support for faculty is needed – graduate assistants and local production equipment Camaraderie among students is important – add synch. 3 faculty promised, only one hired

32. 32 4. Literature & Program Review 11 programs reviewed Titles of courses compared Similarities and differences noted

33. 33 VI. Preliminary Recommendations Policy & Management Faculty & Curriculum Instruction & Delivery

34. 34 Recommendations Policy & Management Tuition and Fees Program Policy Statements Incentives and Involvement of Faculty Entrance Requirements – Skills AND Resources 5-Year Plan

35. 35 Recommendations Faculty & Curriculum Move from text-based instruction to media Explore various management platforms in addition to Blackboard Review the capstone experience Consider specializations w/in the degree Involve full-time faculty

36. 36 Recommendations Instruction & Delivery Offer Summer courses Increase selection of online materials Consider use of adjuncts, supervised by full-time faculty Secure local production resources for faculty Offer synchronous instruction, and consider face-to-face activities

37. 37 VII. Reactions

38. 38