Presentation is loading. Please wait.

Presentation is loading. Please wait.

Student Goals, Engagement and Outcomes: The Path to Success Mark Troy Yunhee Bae Texas A&M University AIR Forum, May 21, 2013.

Similar presentations


Presentation on theme: "Student Goals, Engagement and Outcomes: The Path to Success Mark Troy Yunhee Bae Texas A&M University AIR Forum, May 21, 2013."— Presentation transcript:

1 Student Goals, Engagement and Outcomes: The Path to Success Mark Troy Yunhee Bae Texas A&M University AIR Forum, May 21, 2013

2 Goals and engagement are dimensions that facilitate student learning and lead to student success How are these dimensions related? Are some more effective than others at facilitating learning? What combinations of goals and engagement lead to what types of learning?

3 IEO Assessment Model InputEngagementOutcomes

4

5 Instrument Student Experience in the Research University (SERU) Administered Spring students in Six universities – Minnesota – Pittsburgh – Rutgers – South Carolina – Texas A&M – Virginia 7603 responses from TAMU

6 Methods Exploratory factor analysis and confirmatory factor analysis to specify: – three goal factors (epistemic goals, intrinsic goals, and instrumental goals), – four engagement factors (attention and interest, class preparation, consolidated learning activities, and extra interaction with faculty), – three outcomes factors (knowledge/comprehension, research skills, and communication skills). Multiple Indicators and Multiple Causes (MIMIC) CFA to validate criterion validities through interrelationships between academic engagement constructs and academic outcomes including the three outcome factors and GPA Structural Equation Modeling (SEM) to identify relationships between factors

7 I: Inputs Knowledge and skill upon entering the university Student goals – Epistemic goal – Instrumental goal – Intrinsic goal Background information – Parents’ education – Parents’ household income

8 Measurement Constructs: Students’ College Goal Orientations (n=6,782) Reliability (Alpha) =.64 Confirmatory Factor Analysis Factor Loadings Be in a position to give something back to my community after finishing my education..55 Integrate spirituality into my life..41 Develop a personal code of values and ethics..71 Discover what kind of person I really want to be..65 Reliability (Alpha) =.55 Prepare for graduate or professional school..57 Achieve a high GPA..65 Be in a position to make a lot of money after finishing my education.45 Reliability (Alpha) =.62 Develop an in-depth understanding of a specific field of study..52 Explore new ideas..58 Acquire a well-rounded general education..55 Obtain the skills I need to function in the international arena..50 8

9 E: Engagement Academic Engagement – Attention and interest in class – Class responsibilities – Consolidated learning activities – Extra interactions with faculty Cognitive Strategies – Elaboration – Critical Thinking – Rehearsal

10 Measurement Constructs: Academic Engagement (n=33,880) Reliability (Alpha) =.85 Confirmatory Factor Analysis Factor Loadings Found a course so interesting that you did more work than was required..57 Chosen challenging courses, when possible, even though you might lower your GPA by doing so..36 Contributed to a class discussion..85 Brought up ideas or concepts from different courses during class Discussions..85 Asked an insightful question in class..86 Interacted with faculty during lecture class sessions..75 Made a class presentation..50 Class Preparation Reliability (Alpha) =.73 Not Gone to class without completing assigned reading..62 Not Gone to class unprepared..76 On average, how much of your assigned course reading have you completed this academic year?.50 Not Skipped class..66 Not Turned in a course assignment late..50 Time Use Reliability (Alpha) =.53 Studying and other academic activities outside of class..77 Attending classes, discussion sections or labs

11 Measurement Constructs: Academic Engagement (Continued) Reliability (Alpha) =.73 Confirmatory Factor Analysis Factor Loadings Worked on class projects or studied as a group with other classmates outside of class..50 Helped a classmate better understand the course material when studying together..57 Sought academic help from instructor or tutor when needed..62 Extensively revised a paper at least once before submitting it to be graded..61 Raised your standard for acceptable effort due to the high standards of a faculty member..53 Reliability (Alpha) =.75 Had a class in which the professor knew or learned your name..67 Talked with the instructor outside of class about issues and concepts derived from a course..77 Communicated with a faculty member by or in person..72 Worked with a faculty member on an activity other than coursework (e.g., student organization, campus committee, cultural activity)

12 Measurement Constructs: Emphasis on Cognitive Strategies in the Major (n = 32,034) 12 Reliability (Alpha) =.84 Confirmatory Factor Analysis Factor Loadings Examined how others gathered and interpreted data and assessed the soundness of their conclusions..84 Reconsidered your own position on a topic after assessing the arguments of others..75 Incorporated ideas or concepts from different courses when completing assignments..75 Used facts and examples to support your viewpoint..68 Reliability (Alpha) =.83 Judge the value of information, ideas, actions, and conclusions based on the soundness of sources, methods and reasoning..86 Break down material into component parts or arguments into assumptions to see the basis for different outcomes and conclusions..82 Create or generate new ideas, products, or ways of understanding..71 Reliability (Alpha) =.73 Explain methods, ideas, or concepts and use them to solve problems..94 Recognize or recall specific facts, terms, and concepts..62

13 O: Outcomes Knowledge and Comprehension Skills Research Skills Communication Skills GPA

14 Measurement Constructs: Students’ Perceived Learning Outcomes (n=28,681) Reliability (Alpha) =.82,.80 Confirmatory Factor Analysis Factor Loadings At StartingAt present Analytical and critical thinking skills Ability to read and comprehend academic material Ability to be clear and effective when writing Understanding of a specific field of study Ability to understand international perspectives (economic, political, social, cultural).5854 Quantitative (mathematical and statistical) skills Reliability (Alpha) =.81,.78 Internet Skills Computer Skills Library Research Skills.55 Other Research Skills.48 Reliability (Alpha) =.75,.77 Interpersonal (social) skills Ability to lead Ability to prepare and make a presentation Ability to speak clearly and effectively in English

15 Criterion Validation MIMIC (Multiple Indicators Multiple Causes) Correlations between factor scores and criteria variables – Parents’ education – Parents’ household income – GPA

16 Correlations between the Factors and Criterion Variables Path Coefficients from Criterion Variables to the Factors through MIMIC GPA Parent's Annual Income Parent Education GPA Parent's Annual Income Parent Education Multiple R AE Factors (n=33,880) Attention & Interest.12**.04*.07**.12**.00.05**.13 Class Responsibilities.24**.01.05**.24**-.03*.03*.24 Consolidated Learning.07**.03**.02*.07**.03* Extra Interaction with Faculty.13**.04**.07**.13**.00.05**.14 SCG Factors (n=6,782) Epistemic Goal **-.03* * Intrinsic Goal ** ** Instrumental Goal.19** **-.04* Note. ** : p <.001; * p <.01

17 SPLO-S Factors (n=28,681) Knowledge & Comprehension Skills.09**.02**.13**.07**.08**.16 Research Skills -.02*.04**.05**-.03**.03**.04**.05 Communication Skills.00.12**.06**-.02*.12** SPLO-P Factors(n=28,681) Knowledge & Comprehension Skills.20**.15**.11**.18**.12**.02**.24 Research Skills.02**.03*.02*.03** Communication Skills.04**.13**.05**.02**.14**-.02*.14 ECSM Critical Thinking *.00 Elaboration.05**.02*.00.05**.02*-.02*.05 Rehearsal.04**.08**.03**.08** Correlations between the Factors and Criterion Variables Path Coefficients from Criterion Variables to the Factors through MIMIC GPA Parent's Annual Income Parent Education GPA Parent's Annual Income Parent Education Multiple R Note. ** : p <.001; * p <.01

18 Criterion Variables GPA Annual Parents’ Household Income.13** Parent Education.16**.46** Correlations between the Factors and Criterion Variables Path Coefficients from Criterion Variables to the Factors through MIMIC GPA Parent's Annual Income Parent Education GPA Parent's Annual Income Parent Education Multiple R Note. ** : p <.001; * p <.01

19 Summary Cronbach’s alphas (.53 ≤α ≤.85; mean =.72) were acceptable for all constructs indicating good internal consistency. CFAs on each measurement model of AE, SCG, SRLO, ECSM yielded relevant factor loadings (.32 ≤ λ ≤.94; mean =.64) with good or acceptable model fits, supporting the construct validity of AE, SCG, and SPLO. The relationship of GPA with “Class Responsibilities”,”Instrumental Goal”, “Knowledge and Comprehension” at the present, and “Elaboration” were the strongest in AE, SCG, SPLO, and ECSM respectively. Parents’ annual income is most likely to influence “Knowledge and Comprehension” at present, while, Parent Education is most likely to be related with students’ “Knowledge and Comprehension” at starting indicating a greater impact of parent income than parent education on students’ college learning outcomes Necessary for the investigation of the relationships among AE, SCG, SPLO, ECSM and GPA to understand the paths to the learning outcomes. 19

20 Paths to Student Success in the SERU : The Relationships among 5 domains of students’ Self-Reported Constructs and their cumulative GPA through Structural Equation Modeling (SEM) 20 To investigate the relationships among Academic Engagement, Students’ College Goal, Learning Outcomes, and Cognitive Strategies. --> How do those constructs lead to each learning outcome including student’s perceived learning outcomes (Knowledge & Comprehension, Research Skills, and communication Skills) and GPA ? To better understand students’ college experiences and to develop productive policies and programs to heighten students’ competencies for their paths to success.

21 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major

22 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major

23 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

24 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

25 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

26 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

27 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

28 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

29 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.41 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

30 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) (ns) (ns) (ns) (ns).06.00(ns) (ns) Post- Research Skills.64 Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.50 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

31 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal Post- Communication Skills -.01(ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.49 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 = Students’ Perceived Learning Outcomes Now

32 Pre-Knowledge & Comprehension Pre- Research Skills Pre- Communication Skills Epistemic Goal Intrinsic goal Instrumental Goal Attention & Interest Class Responsibilities Consolidated Learning Extra- Interaction with Faculty Elaboration Critical Thinking Rehearsal (ns) Cumulative GPA Post- Research Skills Post- Knowledge & Comprehension Post- Knowledge & Comprehension Post- Communication Skills (ns).00(ns) -.01(ns).00(ns).01(ns) Students’ Perceived Learning Outcomes at Starting Students’ Perceived Learning Outcomes Now Students’ College Goal Orientations Academic Engagement Emphasis on Cognitive Strategies in the Major R2 =.07 R2 =.01 R2 =.02 R2 =.21 R2 =.08 R2 =.18 R2 =

33 NOTE: Model Fit - Χ2/DF: /39; CFI:.99; RMSEA:.04 Each construct value (factor score) was composed of 1) standardized item scores to address the difference in item-response scale, 2) the mean of item scores weighted by factor loadings, and 3) the resulting scores standardized into a mean of 5 and a standard deviation of 2. The SEM includes the error-correlations, but not shown in the figure. The color of each construct box is correspondent with that of each path coefficient box The Italic numbers followed by “(ns)” indicate non-significant path coefficients The numbers following “ “ indicate the strongest impact of each domain on each construct. 33

34 Conclusion Structural equation modeling (SEM) yielded several insights into the dimensions that have the greatest impacts on student learning. Not surprisingly, for any learning outcome, the student’s incoming ability has the greatest effect on current ability. Following that, the greatest effect on “knowledge and comprehension” is found on the path from “pre-communication skills”, to “intrinsic goal”, to “attention and interest in class”, and from “elaboration”. The most important factors leading to GPA were found to be the SPLO of “knowledge and comprehension,” and the AE of “class responsibilities”. “Elaboration” considered a high level of cognitive strategy did not contribute to GPA. The effect of “critical thinking” on GPA was significant, but negative. “Elaboration” is the strongest factor of ECSM on all of SPLO.

35 Conclusion, continued There are multiple paths to any desired outcome. Not all engagement is equal. Engagement activities need to be appropriate to the desired outcome. Goals are important

36 Contact Us Mark Troy – Yunhee Bae – To download this presentation, go to: –


Download ppt "Student Goals, Engagement and Outcomes: The Path to Success Mark Troy Yunhee Bae Texas A&M University AIR Forum, May 21, 2013."

Similar presentations


Ads by Google