1. Developing into an engineer: the Academic Pathways Study*
December 6, 2007
Sheri Sheppard
* Project within CAEE, an NSF funded Engineering Education Center, 1/1/03-12/31/08

2. Acknowledgement
This material is based on work supported by the National Science Foundation under Grant No. ESI , which funds the Center for the Advancement of Engineering Education (CAEE). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
CAEE is a collaboration of five partner universities: Colorado School of Mines, Howard University, Stanford University, University of Minnesota, and University of Washington.
For further information see the CAEE Web site at or contact Cindy Atman at

3. Outline of Presentation
Framing of the APS
B. "What have we learned about engineering students and their transitions to the world of work? And what are the implications for educational practice?"
C. What is next…

4. The Pipeline Metaphor 1
1. Expresses the typical process students take in preparation to become engineers
2. Expresses the activity of students who prematurely exit the preparation process
3. Expresses the activities being done to repair the preparation process 2 3
We chose the pipeline metaphor because it was the most familiar to engineers and was adequate for capturing our intent, which was to examine and present engineering trends in preparation of women and minority students. Various aspects of the concept of the pipeline has been used to express salient aspects about the preparation process.

5. The Pipeline Metaphor 1
1. Expresses the typical process students take in preparation to become engineers
2. Expresses the activity of students who prematurely exit the preparation process
3. Expresses the activities being done to repair the preparation process
4. Expresses the successful exit as an engineer 2 4 3
We chose the pipeline metaphor because it was the most familiar to engineers and was adequate for capturing our intent, which was to examine and present engineering trends in preparation of women and minority students. Various aspects of the concept of the pipeline has been used to express salient aspects about the preparation process.

6. Subset of Pipeline
We worked with the concept that the entire pipeline included students K-6 and ended with several successful exits, with Baccalaurate, Masters or Doctorate in Engineering

7. detached…
attached…

8. APS Research Questions
Skills
How do students’ engineering skills and knowledge develop and/or change over time?
Identity
How do pre/engineering students identify themselves?
How do these students come to identify themselves as engineers?
How do student appreciation, confidence, and commitment to engineering change as they navigate their education?
What communities do engineering students belong to?
How does belonging to a community contribute to their identity?
Education
How do pre/engineering students navigate their educations?
What elements of students’ engineering educations contribute to changes observed in their skills and identity?
What do students find difficult and how do they deal with the difficulties they face?

9. Classification of Student Outcomes by Type of Outcome & Type of Data (from A. Astin *)
…as affected by “E” environmental factors (institutional characteristics, curricular measures,
Faculty environment, peer environment, individual involvement measures).
* A. Astin, What Matters in College? Four Critical Years Revisited, 1993

10. Classification of Student Outcomes being studied in APS*
…as affected by “E” factors: gender, race & ethnicity, institutional structures, etc.
* Based on A. Astin, What Matters in College? Four Critical Years Revisited, 1993

11. Academic Pathways Study (APS) Sheppard (Lead), Atman, Fleming, Miller, Smith, Stevens, Streveler
Large scale, multi-method, longitudinal study of undergraduate engineering students, class of 2007, UW, Howard, CSM, Stanford (160 students)
Cross-sectional study at 24 institutions, small-large,
(>5000 students)
From the student’s perspective…

12. The research team…
I just inserted the names of all the people listed in the document Dennis sent out ( CAEE Team contact list sb) except for the Evaluation Team names and the names in the categories after that one.
I added in Claire Rosenbaum although she wasn’t in the contact list document.
I didn’t know Mia’s last name to add her in.

13. APS’s Activities May 2006 – Sept. 2007

14. APS Research Methods Surveys Structured interviews
Semi-structured interviews, ethnographic observations
Engineering “thinking and doing” tasks
Academic transcript evaluation
Exit interviews

15. Assessment of Research Questions By Methodology
Ethnography
w/Semi-structured Interviews
Structured Interviews
Engineering Design Tasks
Surveys
(PIE, APPLE)
Cognitive (Skills) 
 
  
Affective (Identity)
“E” Factors:
Laura, please remove the last row totally.

16. What have we learned: selected findings from the APS team
Cognitive--Persistence (Sheppard)
Decision Making & “persistence”
Persistence in a field
Cognitive--Design Skills (Atman)
“E” Factors & Institutional Distinctions (Fleming)
Affective--Identity (Stevens)

17. Should I Stay or Should I Go
Should I Stay or Should I Go? Undergraduates’ Prior Exposure to Engineering and Their Intentions to Major
INTENTION
positive
unsure
low
high
EXPOSURE
For those with an identified interest in engineering,
even most students who are unsure of majoring
in engineering and who have little prior exposure
to engineering choose to major in engineering

18. Engr. (59%)
Other STEM (44%)
MATRICULATION MAJOR

19. 72% with preliminary interest in engineering

20. What are the gender and race & ethnicity make-up
52% with preliminary interest in engineering
88% with preliminary interest in engineering
What are the gender and race & ethnicity make-up
in these various engineering fields?

21. As of June 2007, complex and still forming lives…
How typical is this of the 400?
How typical is this of Stanford students?
How typical is this of engineering majors nationally?
Laura, can you work on labeling….hard to read.

22. APPLES with Cross-sectional Sample, sample findings…
(Cohort 3 schools)

23. What have we learned: selected findings from the APS team
Cognitive--Persistence (Sheppard)
Decision Making & “persistence”
Persistence in a field
Cognitive--Design Skills (Atman)
“E” Factors & Institutional Distinctions (Fleming)
Affective--Identity (Stevens)

24. Design Skills & Perceptions
Year 1
Women more context-oriented than men ٭
Women equally capable with design details ٭
• Year 4
Conceptions of engineering practice
Preparedness for engineering practice
Year 1 vs. Year 4, longitudinal
Changes in conceptions of design

25. Engineering Design: Freshmen Take 1
Survey Question:
You have been asked to design a playground. You have a limited amount of time and resources to gather information for your design. From the following list, please put a check mark next to the five kinds of information you would MOST LIKELY NEED as you work on your design…

26. Information Categories by Gender
*p < 0.10 or **p < 0.05, Fisher exact

27. Engineering Design: Freshmen Take 2
Ten Minute Paper and Pencil Engineering Task:
Over the summer the Midwest experienced massive flooding of the Mississippi River. What factors would you take into account in designing a retaining wall system for the Mississippi?

28. Examples of Detail and Context
Design detail
“cost of materials”
“check the budget available for the operation”
“how to contain the river water that has flooded out”
Design context
“aesthetic appeal – is it going to draw local complaint?”
“the surrounding habitat – make sure little or no damage is done to the environment”
“would wall impact use of the river by industry?”

29. Detail vs. Context Factors by GenderM
stat. sig. finding from which was the attachment for analytic memo
Factors by category and by gender
(all APS, N=51 F + 92 M)
significant difference, p < 0.02

30. Engineering Design.. an emerging picture from the first year…
Considering context – gender differences
men: emphasis on details of solution such as material, financial...
women: emphasis on contextual factors such as social, natural...
Conceptualizing design – gender differences
men: emphasis on building, prototyping...
women: emphasis on gathering information, planning...

31. ETD Selected findings Year 1 Year 4 Year 1 vs. Year 4, longitudinal
Women more context-oriented than men ٭
Women equally capable with design details ٭
Year 4
Low importance, preparation for contextual issues ٭
High importance, preparation for “people” skills
Year 1 vs. Year 4, longitudinal
Changes in conceptions of design
Finally, we return to a focus on design, a core element in engineering. We look at longitudinal analysis of how students conceive of design in engineering.

32. Conceptions of design: Important design activities
Of the twenty-three design activities below, please put a check mark next to the SIX MOST IMPORTANT...

33. Important design activities, by year
Comparison was on paired data, on only those students who responded in both years.
Design activities are sorted by Year 4 frequency. Four of top five match top five from expert data (shown in backup slide): Understanding the problem, Communicating, Identifying constraints, Brainstorming. Experts also had Seeking information in their top five, while APS seniors had Making decisions.
Note that Identifying constraints went up between Years 1 and 4. Let’s take a look how the design activities split between those that went down and up between Years 1 and 4.
side note: Although students were less likely to choose Communicating as seniors than as freshmen, still about half of seniors included it among their 5 most important.
statistical tests: McNemar for paired data for each item by year (Communicating p = 0.022, Planning p = 0.010, Visualizing p = 0.001, Identifying constraints p = 0.000, Iterating p = 0.006)
data source: \\zhora\celt_stu\APS Engineering Thinking and Doing\Year 4, \surveys\analysis\JEE_Analysis_MIDA_InfoGather_Data_AM_v5_wInst_NEW.xls
Year 1 vs. 4
Significant changes asterisked (**p  0.01, *p  0.05).

34. Important design activities, changes
Communicating* Planning** Using creativity Building Visualizing** Imagining Abstracting
down in Year 4
Identifying constraints** Evaluating Modeling Generating alternatives Prototyping Making trade-offs Iterating** Decomposing Synthesizing Sketching
up in Year 4
This slide shows the items that students were less likely to choose as important as seniors than as freshmen (left), and vice versa (right).
Note that the activities less frequently chosen by seniors (left) are terms that are commonly used in non-engineering senses (e.g., “planning”, “building” and “imagining”). In contrast, the activities more frequently chosen by the students as seniors (right) are more precise and engineering-specific.
Minnesota (Cohort 1’) data also shows shift toward more precise, engineering-specific vocabulary (vs. common English words like).
We expect conceptions of design to be affected by institution-specific factors, especially curriculum. Indeed, first analyses of institutional variation in Year 4 responses show that Stanford seniors are significantly more likely to include Prototyping (p < 0.01) and Visualizing (p < 0.05) in their set of most important skills.
statistical tests: McNemar for paired data for each item by year (Communicating p = 0.022, Planning p = 0.010, Visualizing p = 0.001, Identifying constraints p = 0.000, Iterating p = 0.006)
data source: \\zhora\celt_stu\APS Engineering Thinking and Doing\Year 4, \surveys\analysis\JEE_Analysis_MIDA_InfoGather_Data_AM_v5_wInst_NEW.xls
Significant Year 1–4 changes asterisked (**p  0.01, *p  0.05).

35. ETD Summary Complex gender differences
First-year women more context-oriented, but not at the expense of focus on design details.
Implications on continuing challenge to recruit, retain women.
Meeting the ABET a–k and 2020 challenges
Seniors value and have learned traditional “core” engineering, as well as some “people” skills (teamwork, communication).
...but not issues of societal/global context, contemporary issues.
The general understanding is that women differ from men in their interest in society, service, and people. ETD design task data shows gender differences in performance that correspond to this difference in interest or focus.

36. What have we learned: selected findings from the APS team
Cognitive--Persistence (Sheppard)
Decision Making & “persistence”
Persistence in a field
Cognitive--Design Skills (Atman)
“E” Factors & Institutional Distinctions (Fleming)
Affective--Identity (Stevens)

37. Institutional Factors Fleming, Lead; Ledbetter, McCain, Williams
Admission Policy
Access to Resources
Experiences Within University Environment
Diversity Issues

38. To What Extent Do You Consider Your School to be Diverse?

39. Does Your Gender Affect Your Views of Becoming an Engineer?

40. Female Student’s View Does your gender affect your views of becoming an engineer?
“… there are societies, like Society for Women Engineers…that does help change our perspective on being an engineer
…it’s ‘cause I’m female, because I’m a minority and I’m not used to being like that because I’m a white middle class individual
… it’s hard to become an engineer, it’s real intimidating to be ahm, working for… predominantly all males…it’s kind of a challenge to me,
…I can do this, I can pioneer this and be a female engineer, be just as good as a male engineer” Mountain Tech
This student from CSM represent many womens views across all schools. Some of the women are more encouraged than others. Many feel as though they have a gender advantage because there are so many male s in the field that they are guaranteed a job upon graduations. Others feel that in the male dominated profession that they are going to be overlooked or treated with less respect because of their gender.
Howard female students were less likely than all groups to say that their gender affects their views of becoming an engineer.
Stanford women like the idea of feeling elite among the men. One woman mentioned that she really liked being one of three girls in her math class.
A female student from UW offered another interesting perspective. She noted “…I am thinking well, as an engineer am I still going to be able to take time off to have a family…leave for eight years and raise my kids and still get back in? Is this a career that I can do part-time on, while raising my family…?”

41. Male Students’ View Does your gender affect your views of becoming an engineer?
“…if the females... have an advantage, just because [of] things like affirmative action … where they give certain advantages to some minorities, I wonder if it is a disadvantage being the majority?” University of West State-M
Like the females, male engineering students are aware of the dynamics in the field. The know that the field is heavily male dominated. However some men believe that that give them an advantage. Others believe that it keeps the competition tight. Some have even thought that females are directed from one type of engineering and steered into others (the more feminine fields).
Another UW males stated, “Yes, I think that engineering I predominatintly male at this point and I feel like the competition among males is higher because everybody focuses so mucho diversity that if you are a female entrusted in an engineering degree, you have people who are willing to go out of their way to try and encourage you to be in engineering. I feel like it’s a little unfair at times ahm and I’m one of the people who feels like the best candidate for the job is the best candidate. Ahm but of course that’s biased, because our current system favors someone else as opposed to me naturally I’m gonna favor a system which favors myself.
“It’s more natural for males to be engineers.” Coleman University-M

42. Diversity Summary
To most engineering students, diversity means difference (school, gender, race , geographical, major field, politics, religion).
Diversity can be an uncomfortable topic for students to discuss.
Students recognize the impact of diversity on their careers.

43. What’s Next? Mountains of data to be analyzed . . . . stay tuned
Possible Relationships to WASC:
• Study of knowledge evolution
• Curricular Flow-educational careers study (Dan McFarland)
• Model for study in other fields

44. Extra slides

45. APPLES with Cohort 3 Schools, sample findings…

46. What have we learned: selected findings from the APS team
Decision Making & “persistence”; (Sheppard)
• Identity (Stevens)
Design Skills and Perceptions (Atman)
Institutional Factors (Fleming)

47. Identity٭: Becoming an Engineer Stevens, Lead; Amos, Garrison, Jocuns
Identification
The practices by which an individual becomes identified with engineering (by her/himself and by others)
Navigation
How individuals navigate a pathway to becoming an engineer
Accountable Disciplinary Knowledge
Actions when performed are counted by someone as engineering knowledge
Our ethnographic perspective has enable us to take an holistic approach to the study of identity development among engineering students. This approach we refer to as becoming an engineer and it enables us to approach this ethnographic problem by observing three dimensions that change over time. While these are presented as three separate dimensions, it should become apparent during this discussion they work together in forging a trajectory of becoming an engineering.

48. Identification (Changes over time)
Increasing solidarity with other engineering students
We/they language, “Techies” and “Fuzzies”
Identity displays – websites, clothing, social activities
Increasing expression of views that they are different from non-engineering students
Engineering work is harder
Harder work justifies future lifestyle

49. Navigation at UWest
Different navigational pathways have a clear effect on identification of students as engineers
Students not admitted during their first year can be lost during this pre-engineering phase.
Once admitted to engineering majors, students are granted literal “keys to the clubhouse” — a critical rite of passage that changes how students identify themselves as engineers.
Students also took a more agentive stance to coursework and learning once admitted.
I think Uwest is a special case but I think the issue of the tension between weed-out and include is a productive one to consider on all the campuses. The opposite end is Coleman, where they effectively choose Engineering. Engineering does not choose or exclude them.

50. Gender Identity/Navigation at UWest
Women and men both form identities as engineers that seem quite similar across the genders and draw on stereotypical engineering image (problem solver, good at math, etc.)
However, in competitive admission practices at UWest women students are believed to have an advantage over men; presence of organizations supporting women also seen by men as evidence that women need help.
Men use this explanation of women having an advantage in admission to set up a rationale (that involves no fault of their own) for their potential failure to get into the major.
It is suggested that some believe that women who get into the major may be less qualified than men who do not. This leads to women working to “prove themselves” or working to appear deserving of being in engineering. This seems cultivate some stereotype threat (Steele).
Women go ‘underground’—seeking help from other women as a first resort, makes them sensitive to criticism of their male peers
Forming an identity as an engineer is important for both men and women. Identification for women engineering students is pretty similar to that of men students. Both men and women seem to pick and choose pieces of the “stereotypical engineer” identity to attribute to themselves. Like their male counterparts, women do not necessarily embrace all aspects of a “stereotypical” engineering identity. Some identify with (or find sponsorship in) the math-heaviness of the major, for others identification may be signaled by joining an engineering sorority or being awarded a special scholarship or office in a department club.
At Uwest the competitive nature of the admission process creates interesting dynamics, including a competitive climate in the first two years, pitting students against one another who are seeking a limited number of spots in the department. Related to this competitive climate is the way in which groups, such as women are seen as having an advantage over others in the admission process. Several men AND women expressed the belief that lower standards were applied to women applicants than were applied to men applicants. This belief has several consequences for both men and women.

51. Accountable Disciplinary Knowledge (Changes over time)
During first two years:
Technical subject matter prerequisites (mathematics, physics, chemistry) outside of engineering. Little exposure to engineering coursework.
Lecture-based teaching, individual-based problem sets and exams (except labs)
During latter two years:
Kinds of problems to be solved shifts to more open-ended problems
Students’ relationship to data changes. They go from mathematical puzzle solvers to data users to data collectors
Biggest changes in accountable disciplinary knowledge come through in experiences of Capstone project course
In becoming an engineer we noticed that what counts as accountable disciplinary knowledge for engineering students changes over time. During the first two years students have little exposure to engineering coursework or engineering problems because they are taking prerequisites in math and other sciences. Courses are presented in a sequential nature where one course was considered to be mastered prior to enrolling in the next course in the series. Our students noted that this was not a reality. For instance, Adam mentioned to us during his fourth year interview that he felt that the did not master a course until he had completed the course that followed it.
Early courses are also lecture based where individual students are responsible for completing assignments usually in the form of problem sets. The exception was labs, but these were often performed as a pair or small group and they were still working with problem sets.
During this phase many students at UWest mentioned that there was a considerable amount of competition where they were competing with classmates for spots in the department to which they will be applying.
There is also the thing about Adam mentioning that he felt he understood a course after he had taken one that followed it.

52. Accountable Disciplinary Knowledge (Changes over time)
Examples of two UWest students handling this change over time:
Adam struggled as problem-set based mathematics (school math) was displaced by group work and open-ended problems
Simon came into his element with the AA capstone; he drew on his wind tunnel experience (he ran the wind tunnel at UWest) and was the expert in some of the tests (even in relation to the professors/instructors)
The capstone was considering to be the rite of passage between the world of being an engineering student that of a practicing engineer. One of the reasons for this was the long hours put in where students had even less free-time than they normally did. One student mentioned to us that he has only seen his family for 30 minutes over the past 3 weeks and he lived at home. These courses provided the most clear glimpse at an actual engineering workplace than even internships and/or co-ops; where students did not always receive actual engineering practice. Students were presented with open-ended problems that could solve with a variety of means. They worked in groups or teams. The instructors (some of whom actually were in engineering firms) acted as consultants and in some cases clients. In a few of the courses students had real-worlds clients, e.g. the mechanical engineering capstone worked with dental professionals, the chemical engineering capstone was working with a hydrogen-based economy that were planning to submit the various results to the local transportation authority.

53. Identity Summary Complex relationships between
Identification
Navigation
Accountable Disciplinary Knowledge
Pathways for individuals vary greatly

54. What have we learned: selected findings from the APS team
Decision Making & “persistence”; (Sheppard)
• Identity (Stevens)
Design Skills and Perceptions (Atman)
Institutional Factors (Fleming)