Presentation on theme: "Dr. Mac Adkins, President SmarterServices"— Presentation transcript:
1 Dr. Mac Adkins, President SmarterServices Measuring GritDo Non-Cognitive Attributes Impact Academic Success, Engagement, Satisfaction and Retention?Dr. Mac Adkins, President SmarterServicesProvided by
2 Measuring GritsWe recommend a glass beaker for measuring grits. We are from Alabama, we know about these things.
3 What is Grit?Why does one student who had straight A’s in high school drop out of college after one year, and another one excel?Why does one single mom with three children graduate Summa Cum Laude and another one drop out?
4 What is Grit?Grit is that elusive quality that prompts one student to stick with it while others quit.For over ten years we have measured levels of grit in over 2,300,000 students at over 500 colleges and universities.Today I want to share with you the results of research related to the impact that grit has on student success.
5 Types of Data Used To Predict Learner Success APTITUDEATTITUDESITUATION
7 Some Students Seem To Have More Grit Than Others
8 Three Approaches to Measuring Grit Stick your head in the sand.Use a brief, non-prescriptive survey.Use SmarterMeasure Learning Readiness Indicator.Is Distance Learning For Me?
9 SmarterMeasure Learning Readiness Indicator A 124-item online skills test and attributes inventory that measures a student’s level of readiness for studying onlineUsed by over 500 Colleges and UniversitiesSince 2002 taken by over 2,300,000 students
11 What Does The Assessment Measure? INTERNALINDIVIDUAL ATTRIBUTESMotivation Procrastination Time Management Help Seeking Locus of ControlLEARNING STYLESVisual Verbal Social SolitaryPhysicalAuralLogicalEXTERNALLIFE FACTORSAvailability of TimeDedicated PlaceReasonSupport from FamilySKILLSTECHNICALTechnology UsageLife ApplicationTech VocabularyComputing AccessTYPINGRateAccuracyON-SCREEN READINGRecall
26 Adjusting Readiness Ranges Adjusting the cut points can make the reporting a more accurate predictor of success.
27 How Do Schools Use It? Orientation Course Enrollment Process Information WebinarPublic WebsiteClass ParticipationFacebook68% of client schools administer the assessment to all students, not just eLearning students
28 Thermometer AnalogyMore important than taking your child’s temperature is taking appropriate action based on their temperature.More important than measuring student readiness is taking appropriate action based on the scores.
30 Progression of SmarterMeasure Data Utilization PredictiveCorrelationComparisonDescriptiveStudent Service
31 Research Ideas on the Research Page of the Website
32 Professionally Assisted Approaches to Research ProjectsInternally ConductedCompany AssistedProfessionally Assisted
33 Middlesex Community College 6% to 13% more students failed online courses than on-ground courses.Intervention PlanAdminister SmarterMeasureIdentify which constructs best predicted successProvide “Success Tips” as identifiedDistributed by website, , orientation course, records office, library, posters, and mail
34 Research Findings Analyzed 3228 cases over two years Significant positive correlation between individual attributes and gradesMotivationImpactsGrades
35 Results of Middlesex Research Before SmarterMeasure™ was implemented, 6% to 13% more students failed online courses than students taking on-ground courses. After theimplementation, the gaps were narrowed: 1.3% to 5.8% more online students failed than on-ground students.
36 Results of Middlesex Research Failure rates reduced by as much as 10%
37 Action Plan Three areas of focus Empower eLearning staff, faculty advisors, and academic counselors with student dataMotivationSelf DisciplineTime ManagementThree areas of focus
38 Project Summary“In summary, the implementation of SmarterMeasure has helped students to achieve better academic success by identifying their strengths and weaknesses in online learning.”In essence, with various strategies implemented to promote SmarterMeasure™, a “culture” was created during advising and registration for students, faculty, and support staff to know that there is a way for students to see if they are a good fit for learning online.
39 CEC - The NeedWe need to know which students to advise to take online, hybrid or on-campus courses.We need to know which students to direct to which student services to help them succeed.We need to know how to best design our courses so that new students are not overwhelmed.
40 The AnalysisWhat is the relationship between measures of student readiness and variables of:Academic Success - GPAEngagement – Survey (N=587)Satisfaction – Survey (Representative Sample based on GPA and number of courses taken per term)Retention – Re-enrollment data
41 The Analysis Phase One – Summer 2011 Phase Two – Fall 2011 Included data from all three delivery systems – online, hybrid and on-campusAnalyzed data at the scale levelPhase Two – Fall 2011Focused the research on online learners onlyAnalyzed data at the sub-scale levelA neutral, third-part research firm (Applied Measurement Associates) used the following statistical analyses in the project:ANOVA, Independent Samples t-tests, Discriminant Analysis, Structural Equation Modeling, Multiple Regression, Correlation.
42 The Findings Academic Achievement The scales of Individual Attributes, Technical Knowledge, and Life Factors had statistically significant mean differences with the measures of GPA.
43 The Findings Retention The measure of Learning Styles produced a statistically significant mean difference between students who were retained and those who left.A 73% classification accuracy of this retention measure was achieved.The scales of Individual Attributes and Technical Knowledge were statistically significant predictors of retention as measured by the number of courses taken per term.
44 The Findings Engagement The scales of Individual Attributes and Technical Competency had statistically significant relationships with the four survey items related to Engagement.The scales of Life Factors, Individual Attributes, Technical Competency, Technical Knowledge, and Learning Styles were used to correctly classify responses to the survey questions related to engagement and satisfaction with up to 93% classification accuracy.
45 The Findings Satisfaction Structural equation modeling was used to create a hypothesized theoretical model to determine if SmarterMeasure scores would predict satisfaction as measured by the survey.Results indicated that prior to taking online courses, student responses to the readiness variables were statistically significant indicators of later student satisfaction.Therefore, the multiple SmarterMeasure assessment scores are a predictor of the Career Education survey responses.
47 The Findings Student Categorizations Enrollment Status Positive – active/graduated (34.3%)Negative – withdrew/dismissed/transfer (65.7%)Academic Success StatusPassing – A, B or C (48.9%)Failing – D, F or Other (21.1%)Transfer Credit – (21.8%)Not reported – (8.2%)
48 The Findings - Correlates Readiness DomainReadiness Domain SubscalesPositive vs. NegativePass vs. FailLife FactorPlace, Reason, and SkillsPlaceLearning StylesSocialandLogicalN/APersonal AttributesAcademic, Help Seeking, Procrastination, Time Management, and Locus of ControlTime ManagementTechnical CompetencyInternet CompetencyComputer CompetencyTechnical KnowledgeTechnology UsageTechnical Vocabulary
49 The Findings - Predictors Readiness DomainsGPAFpLife FactorPlace and Skills12.35.0001Learning StylesVerbal a and Logical3.95.02Personal AttributesHelp Seeking, Time Management, and Locus of Control21.11Technical CompetencyComputer and Internet Competency22.75Technical KnowledgeTechnology Vocabulary38.76
50 The Findings - Predictors Readiness DomainsCredit Hours EarnedFpLife FactorPlace12.37.0001Learning StylesVisual6.81.01Personal AttributesAcademic Attributes, Help Seeking, and Locus of Control13.40Technical CompetencyComputer Competencyand Internet Competency12.23Technical KnowledgeTechnology Usage and Technology Vocabulary26.97
51 The RecommendationsWe need to know which students to advise to take online, hybrid or on-campus courses.A profile of a strong online student is one who:Has a dedicated place to study onlinePossesses strong time management skillsDemonstrates strong technical skillsExhibits a strong vocabulary of technology terms
52 The RecommendationsWe need to know which students to direct to which student services to help them succeed.An online student who should be directed toward remedial/support resources is one who:Has a weak reason for returning to schoolHas weak prior academic skillsIs not likely to seek help on their ownIs prone to procrastinateHas low, internal locus of controlHas weak technology skills
53 The RecommendationsWe need to know how to best design our courses so that new students are not overwhelmed.Limit advanced technology in courses offered early in a curriculumFoster frequent teacher to student interaction early in the courseRequire milestones in assignments to prevent procrastinationClearly provide links to people/resources for assistance
54 Argosy University Required in Freshman Experience course Students reflect on scores and identify areas for improvement in their Personal Development PlanGroup reflection with others with similar levels of readiness
55 Argosy University - COMPARE Compared the traits, attributes, and skills of the online and hybrid students.Substantial differences between the two groups existed.Changes were made to the instructional design process for each delivery system.OnlineHybrid
56 Argosy University - EXPLORE Correlational analysis between SmarterMeasure scores and student satisfaction, retention, and academic successSatisfactionSuccessRetentionTechnicalMotivationTimeStatistically Significant Factors:Technical Competency MotivationAvailability of Time.
57 Argosy University - TREND Aggregate analysis of SmarterMeasure data to identify mean scores for students.Comparison made to the national mean scores from the Student Readiness Report.National ScoresArgosy Scores
58 Argosy University - APPLY Findings were shared with the instructional design and student services groups and improvements in processes were made.For example, since technical competency scores increase as the students take more online courses, the instructional designers purposefully allowed only basic forms of technology to be infused into the first courses that students take.
59 J. Sargeant Reynolds Community College Required as admissions assessmentIntegral part of their QEPComputed correlations with grades and SmarterMeasure sub-scales of over 4000 students.PGradesAttributesTechnicalLearning StylesLife Factors
60 Findings Statistically significant correlations: Dedicated place, support from employers and family, access to study resources, and academic skills (Life Factors)Tech vocabulary (Technical Knowledge)Procrastination (Individual Attributes)ScoresGrades
61 Academic Success Rates Less than 10% of students with low scores experienced academic success.
62 Five SchoolsWhat is the relationship between measures of online student readiness and measures of online student satisfaction?
63 Methodology Incoming vs Outgoing Data from 1,611 students who completed both the SmarterMeasure Learning Readiness Indicator and the Priority Survey for Online Learners were analyzed.
64 FindingsThere were statistically significant relationships between factors of readiness and satisfaction.
65 Comparison to Compass Scores North Central Michigan College - Petoskey, MI
66 National Data 2013 Student Readiness Report Data from 639,324 students from 275 colleges and universities
67 Online Learner Demographics 69% were female54% were Caucasian/White54% had never taken an online course before40% were traditional aged college students53% were students at an associate’s level institution
68 Online Learner Demographics Dominant Social learning styleHighly motivatedModerate reading skillsPressed for timeIncreasing technical skills
69 Profile of a Successful Online Student Four demographic variables have had a statistically significant higher mean for five years in a row.Females higher in Individual Attributes, Academic Attributes, and Time Management.Males higher in Technical Knowledge.
70 Profile of a Successful Online Student Caucasians have had the highest means for five years in Technical Knowledge.Students who have taken five or more online courses have had the highest means for five years in Individual Attributes, and Technical Knowledge.
71 Readiness Impacts Satisfaction ConclusionStatistically significant relationships exist between measures of online student readiness and measures of academic success, engagement, satisfaction and retention.Readiness Impacts Satisfaction
72 Schools and Students Benefit ConclusionStudents individually benefit and schools collectively benefit from measuring learner readiness and appropriately responding.Schools and Students Benefit