Teaching Basic Skills Mathematics: Outcome Assessment Jenny Kimm, Associate Professor, City College Presented at the: 2009 Strengthening Student Success Conference San Francisco, CA: October 7, 2008 Xi Zhang, Campus Based Researcher, City College San Diego Community College District

I NTRODUCTION OF THE D ISTRICT San Diego Community College District 2 nd largest district in the state Three 2-year colleges and eleven Continuing Education campuses Serves approx. 100,000 students each semester

I NTRODUCTION OF THE S AN D IEGO C ITY C OLLEGE San Diego City College is the first established community college in San Diego. San Diego City College is a public, two-year community college administered by the San Diego Community College District. Serving as the educational cornerstone of downtown San Diego, the college offers more than 100 majors, 100 certificate programs and 1,500 classes each semester to 16,000 students.

P URPOSE OF THE P RESENTATION The primary purpose of this presentation is to share the research methodology and innovative techniques for data analysis and instrument refinement for SLO assessment rather than to disseminate results of the study. A second purpose is to support the teaching of Basic Skills Math in community colleges.

P URPOSE OF THE S TUDY Demonstrate SLO assessment cycle in the Math Department at San Diego City College Draw attention to measurement issues in assessing SLOs. Demonstrate methods of item analysis that has multiple advantages.

R ESEARCH D ESIGN SLO assessment cycle A repeated measure design: pre and post design

SLO A SSESSMENT IN THE M ATH D EPARTMENT City College Student Learning Outcomes Assessment Cycle 6 Column Form For Developmental Math Program- Math 35, 95, 96 Year: 2008/2009

R ESEARCH M ETHODOLOGY Rasch Model Estimate both item difficulty and person ability Map both parameters on the same scale Inform instrument refinement Paired Sample t-test Statistically significant improvement from pretest to posttest

T ARGET P OPULATION AND S AMPLE S IZE Developmental math courses: Pre-Algebra, Beginning Algebra, and Intermediate Algebra. This totals to over 1000 students taking the developmental math courses In Fall 2008, we collected pre/post paired data from about 250 students

D ATA C OLLECTION : I NSTRUMENT Developed one pre test and a similar post test TestGen testbank software 8 multiple choice questions Topics of the questions A Sample Instrument

D ATA C OLLECTION : T EST A DMINISTRATION Pre-test administration Pre-test grading Post-test administration Post-test grading

D ATA M ANAGEMENT For each student, itemized response per question and a total score were entered and paired in Excel Data then were exported to Winsteps for model fitting Estimates of item difficulty and student ability were analyzed in SPSS to compare pre test results to the post.

D ATA A NALYSIS Fit itemized responses rather than the total scores with the Rasch Model to obtain estimates of item difficulty and student ability.

D ATA A NALYSIS Map both item difficulties and person abilities on the same scale to produce the Item-Person Map. Compare Pre test and Post test Paired sample t-test Anchoring item difficulties

M EASURES Item difficulty Student ability

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Pre-test Item difficulty Student ability

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Pre-test Person-item map

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Post-test Item difficulty Student ability

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Post-test Person-item map

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Pre and post comparison Anchoring item difficulties to produce a new set of student ability estimates

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Pre and post comparison Anchoring item difficulties to produce a new set of student ability estimates

R ESULTS (P RE - ALGEBRA FALL 2008 DATA) Pre and post comparison Paired sample t test

F INDINGS Results revealed that students scored statistically significantly higher in the post test compared to their performance in the pretest. Content areas that the instructors need to emphasize for teaching. Information for instrument refinement.

INSTRUMENT REFINEMENT

U SE OF R ESULTS FOR P ROGRAMMATIC I MPROVEMENT Imbed the post test into the final exam to increase sample size. Also provide online version of pre test to collect data from online developmental classes. Rewrite or revise test questions based on results of item analysis. Identify difficult topics and disseminate the information to developmental math instructors for future teaching. Also disseminate the information to instructors of higher level math course for their preparation and planning.

D ISCUSSION Advantages of item analysis Solve measurement issues Conduct meaningful comparisons Bank good test items for constructing future tests Diagnostic function provides insight of the strength and weakness of student content knowledge.

L IMITATIONS OF THE R ESEARCH Small sample size Small number of test items Item stability Generalizability of the results

Q UESTIONS ?