1. Evaluation of Multimedia Software and a Workbook Designed to Improve 3-D Spatial Skills of Engineering Students Sheryl A. Sorby & Thomas Drummer Michigan Technological University

2. Background  Spatial skills are important for success in technical fields Especially important for success in engineering graphics Especially important for success in engineering graphics  Significant gender differences in 3-D spatial skills, favoring males Gender differences cross international boundaries Gender differences cross international boundaries  Poorly developed spatial skills may lead women to drop out of engineering at a disproportionate rate

3. Assessing Spatial Skills-Purdue Spatial Visualization Test: Rotations (PSVT:R)

4. Assessing Spatial Skills-Mental Cutting Test (MCT)

5. Assessing Spatial Skills-Differential Aptitude Test: Space Relations (DAT:SR)

6. Spatial Skills Class at Michigan Tech  Initially developed in 1993 with funding from the National Science Foundation  Additional funding in 1998 led to development of multimedia software and workbook  Class modified significantly in 2000 to include software and workbook

7. Multimedia Software and Workbook  Developed in 1998-99 with funding from the National Science Foundation  Consist of 9 separate modules designed to enhance 3-D spatial skills Each module consists of Background and Exercises sections Each module consists of Background and Exercises sections  Software is stand-alone and works on either a Mac or PC  Published by Delmar Learning in 2002

8. Assessment of Multimedia Software  Students given attitudinal surveys rating various aspects of the software Ratings given by module and overall Ratings given by module and overall  Ratings were on a 1-4 scale  Students asked to rate such things as ease of understanding, ease of use, etc.

9. Results from Attitudinal Surveys of Individual Modules ModuleBackgroundExercises Isometric Sketches 3.523.35 Orthographic Projection 3.583.44 Flat Pattern Folding 3.523.66 Rotation about One Axis 3.473.49 Rotation about Two Axes 3.623.79 Reflections and Symmetry 3.523.36 Cross Sections of Solids 3.633.58 Surfaces & Solids of Revolution 3.653.54 Combining Solids 3.503.53

10. Student Overall Ratings StatementAgreeDisagreeUncertain The modules helped me learn the course material 3212 The multimedia modules were fun to use 3122 The topics covered in the modules were appropriate to the course 3500 The multimedia modules were easy to use 2843 I would rather learn using the multimedia materials than using written materials 2159

11. Student Overall Ratings (cont.) StatementAgreeDisagreeUncertain The exercises in the multimedia modules were at the right difficulty level for me 18170 There were enough exercises at the end of each module to find out whether or not I understood the material 2285 It took me the entire hour of lab time to complete each module 1313 This is my first time using multimedia educational software 24101 Multimedia software is an effective tool for helping students develop their 3-D spatial ability 3212

12. Assessment of Gains on Spatial Skills Test-PSVT:R n Pre- Test Post- Test Gain Significance of Gain Original Course (93-99) 18650.576.926.4p<0.0001 Modified Course (00-02) 15748.373.725.4p<0.0001

13. Assessment of Gains on Spatial Skills Test-MCT n Pre- Test Post- Test Gain Significance of Gain Original Course (93-99) 9937.951.413.5p<0.0001 Modified Course (00-02) 10934.852.617.8p<0.0001

14. Assessment of Gains on Spatial Skills Test-DAT:SR n Pre- Test Post- Test Gain Significance of Gain Original Course (93-99) 9962.378.316.0p<0.0001 Modified Course (00-02) 11062.177.915.8p<0.0001

15. Improvement in Retention Rates WomenMen EGCGEGCG Original Course 88.9%68.3%75.3%69.0% Modified Course 87.4%71.7%76.8%70.0% Differences in Retention Rates for Women were Statistically Significant but those for Men were not

16. Current Project  New course using multimedia software and workbook seems to be effective Similar or better gains on spatial skills tests Similar or better gains on spatial skills tests Similar retention rates for men and women in engineering Similar retention rates for men and women in engineering  Research Questions: Is the software alone as effective as when it is used in conjunction with the workbook in developing spatial skills? and Can the materials be used effectively with non- engineering students?

17. Study Groups  170 non-engineering students participated in the study Most from either Biology or Computer Science Most from either Biology or Computer Science  Divided into four groups Software alone, Workbook alone, Software and Workbook, and Comparison Software alone, Workbook alone, Software and Workbook, and Comparison  Treatment groups participated in ten weeks of training in 2-hour sessions each week  Tried to balance groups for gender, major, and PSVT:R pre-test score  Comparison group invited back for post-test at end of 10- week period

18. Gains on Spatial Skills Tests- PSVT:R Pre-testPost-testGainSignificance EG-Software69.579.47.5p<0.005 EG-Workbook75.986.611.6p<0.0005 EG-Software & Workbook 74.385.611.2p<0.0005 Comparison Group 72.677.95.3p<0.03

19. Gains on Spatial Skills Tests-MCT Pre-testPost-testGainSignificance EG-Software44.551.06.5p<0.01 EG-Workbook53.962.38.4p<0.001 EG-Software & Workbook 46.460.213.8p<0.0005

20. Testing Data  Each group had one or two data points that appeared to be outliers  For example in SW group, one foreign-born student had pre-test of 17% and post-test of 92%. Her English skills improved dramatically over 10-week period Her English skills improved dramatically over 10-week period Large gain could be due to improvements in language skills more than to improvements in spatial skills Large gain could be due to improvements in language skills more than to improvements in spatial skills  Data analyzed using a nonparametric logistic regression model

21. Logistic Regression Results  Both groups that used the workbook were significantly better than the CG  Software only group was not better than CG

22. Attitudinal Survey and Results  Students in EG were administered an attitudinal survey at end of training period  62% would have preferred using the software alone for training even though it was least effective Less time involved Less time involved  Female students spent more time completing the training materials for all groups

23. Attitudinal Survey and Results  Male students expressed greater confidence in their spatial skills at beginning of study Question asked retrospectively Question asked retrospectively Level of Confidence Males (n=66) Females (n=29) Not confident 1.5%27.6% Somewhat Confident 28.8%51.7% Confident37.9%20.7% Most Confident 31.8%0%

24. Attitudinal Survey and Results  Women preferred the use of manipulatives and working in groups  Female students expressed higher level of learning Level of Learning Males (n=66) Females (n=29) Did not learn anything new 6%3% Learning limited to a few aspects 47%17% Learned something on most days 26%45% Learned something every session 21%35%

25. Conclusions  Multimedia software and workbook developed for use with engineering students is suitable for use with non-engineers  Software alone does not seem to be as effective as when used in conjunction with the workbook  There were gender differences in the type of training preferred and in the perceived level of learning

26. Acknowledgement  The authors gratefully acknowledge the support of this project from the National Science Foundation through grant number HRD-0429020