Presentation on theme: "CALICO 2004 “CALL: Focusing on the Learner” Carnegie Mellon University Bridging Theory and Practice: Research-Based Listening Tasks for Video Comprehension."— Presentation transcript:
CALICO 2004 “CALL: Focusing on the Learner” Carnegie Mellon University Bridging Theory and Practice: Research-Based Listening Tasks for Video Comprehension Luba Iskold, Ed. D. Statistical analysis Greg Cicconetti, Ph.D. Muhlenberg College Allentown, PA
Statement of the Problem How do students develop listening skills by using video materials? Do they learn best by mere “exposure to …comprehensible input?” Should we assist students in comprehending a videotext? Do listening tasks performed by students while they are watching a video episode help them concentrate on important information in that video? Which tasks leave the learner with higher levels of video comprehension?
The Purpose of the Study To investigate if activities performed by learners during the listening stage enhance comprehension of a video episode. Two conditions of video viewing: ‘exposure only ‘and using Video Guides specifically designed for the present investigation. Thus, the purpose of this empirical study was to examine: 1. the effect of listening tasks performed during video viewing on the comprehension of second-semester Russian undergraduate college students of video episodes, as measured by Immediate Recall Protocols written in English, and 2. the effect of a Video Guide condition on students’ recall, recognition, and application of the content, vocabulary, structures, and cultural information from “Начало” video, as measured by immediate and delayed tests completed in Russian.
Previous Research Theoretical perspectives on language learning –Comprehension-based approaches –Cognitive-theoretical views –Sociocultural perspectives Research Related to Listening –Research on listening and reading comprehension –Factors that affect listening comprehension –Research on listener characteristics –Authentic materials in listening research –Video in listening research
How does one teach listening? Constructing tasks for listening comprehension Assessment of listening
Constructing Tasks for Listening Richards (1983) suggested manipulation of two variables as a means to develop listening micro-skills: the input and the task INPUT MICRO-SKILLS TASKS His taxonomy includes 33 micro-skills for listening to conversations 18 additional skills for academic listening: Joiner (1990) recommended listening/viewing guides which students use while watching the video episode
Assessment of Listening Comprehension How does one determine which listening tasks work best for learners? Recall protocols, frequently used in L1 research. In 1986-87, Bernhardt and James applied recall protocols to assess reading and listening comprehension in L2. Glisan (1988) advocated using comprehension checks in the native language, therefore avoiding the mixture of skills.
Materials designed for this study: Interactive Viewing Guides Recall Protocols Scoring Sheets Immediate and Delayed Quizzes (recall, recognition, application) Exit Survey Participant Information Survey Baseline Test
The Viewing Guides Constructed for the Study The purpose: To facilitate understanding of the video episodes. To help students focus on specific important aspects of each videotext, NOT to assess their comprehension of the video Features: The tasks were explained before students start watching the video. The reading load of the tasks was kept to a minimum. The items were presented in the true/false format Sections: Instructions Main Ideas: characters, places and events Details: vocabulary, phrases, cognates, idioms Cultural similarities/differences Space for note taking
The Study Setting An undergraduate, four-year college with 1, 800 students. Maintains a foreign language requirement. Languages: French, German, Hebrew, Italian, Russian, Spanish, Greek, and Latin. An interdisciplinary major/minor in Russian Studies. Elementary Language Level I and II are first and second semester courses of a target language sequence offered by the college. Placement is dependent upon experience or a placement test.
Population and Sample The study took place in the spring of 2004. Sixteen students enrolled in one section of a fifteen-week, second semester Elementary Russian II participated in the study: 12 freshman, 4 sophomores; 7 men, 9 women; all students took Elementary Russian I; 13 students never took Russian in high school; 3 participants had 2 years of Russian; 11 were planning to major/minor; 5 were fulfilling the College’s FL requirement; 14 students were native speakers of English; 2 were heritage speakers of Polish.
Treatments An intact class of sixteen Elementary Russian II Taught by one faculty member participated The class was divided into two groups Group homogeneity in composition: Students were ranked by prior achievement in Elementary Russian I and then randomly assigned to Group 1 (G1, n1 = 8) and Group 2 (G2, n2 = 8).
Baseline Test on listening comprehension. Both groups followed the same syllabus: »received the same instruction »completed the same assignments »took the same quizzes »on the same dates »in the same setting.
Watching the Video The participants watched four «Начало» video episodes on CD-ROM at the Language Learning Center, under two different conditions: A control group: ‘exposure only’ to the videotext in the commercially available format. An experimental group: the same videos in conjunction with the on-line Video Guides designed for the present investigation. Group One (G1) served as the control while Group Two (G2) served as the experimental group for video episodes one and three. Group One (G1) served as the experimental group, while Group Two (G2) served as the control for video episodes two and four. Table 1 Allocation of Groups to Video Formats VideoEpisode 1Episode 2Episode 3Episode 4 Group IControlExperimentalControlExperimental Group IIExperimentalControlExperimentalControl
Variables: Controlled variables: Groups (control and experimental) Treatments, conditions for video viewing. Dependent variables: 1.the amount of text material (or the number of scored points) recalled in four Immediate Recall Protocols; 2.the scores in listening comprehension achievement, as measured by four sets of immediate and delayed quizzes, and 3.participants’ opinions regarding the effectiveness of treatments, as measured by the Exit Survey.
Materials Texts The videotexts employed in the study were taken from «Начало» (Lubensky et. al)| –a video-driven package for beginning students of Russian –integrates the teaching of language and culture –provides a balance between structure and communicative activities. –sixteen 2.5-3.5- video episodes, connected by a story line; filmed in Moscow Simulated Authentic discourse
Testing and Scoring Procedures| Sample Recall Protocol Scoring Sheet Начало # 5: Бизнес по-московски 4 --most important event1 --least important event Idea Units/Sentences: 15 Total Points: 48 Points PossibleIdea UnitsPoints Scored 2What a nightmare! How terrible! 4The house is new, but there is no asphalt. 4Lena cannot find rubber boots. They don’t have her size. 4All neighbors have a big problem. 4Viktor understands their problem and is ready to help. 4Viktor has a business. He has rubber boots of all sizes, but he does not sell them. 3Every morning he waits for the neighbors in the hallway and hands out rubber boots to them. 3Viktor’s friend is at the bus stop. He waits for the neighbors and takes the boots. 3The neighbors pay 500 rubles. 3They think it’s expensive. 4Viktor says that 500 is not expensive. For this money they get clean boots and good mood. 2Viktor says that pretty girls get the boots for free. 2Sasha thinks, its discrimination. 2Victor says it’s a joke. 4Viktor thinks that Lena is a very pretty girl.
Findings Table 2 Summary of Nested Factorial Statistical Analysis of the Immediate Tests The results presented in Table 2 suggest that: There were no significant differences among scores from Recall (1) that may be attributed to either episode version, or treatment; Although the scores from Recognition (2), Application (3) and Total scores do exhibit differences, the treatment participation of a student (Control or Experimental) does not appear to affect his/ her scores. QuizGroupN Recall (1)P = 0.087P = 0.37062 Recognition (2)P < 0.0005P = 0.58562 Application (3)P = 0.002P = 0.44462 Total Score (4)P = 0.002P = 0.35462 Immediate Recall Protocols (5)P < 0.0005P = 0.02258
Statistically significant differences were found among the treatment groups and video episodes on the Immediate Recall Protocol: The Experimental group consistently performed better on Immediate Recall Protocols. The findings strongly suggest that the Video Guides were effective. Table 3 Mean Scores and Standard Deviations on Immediate Recall Protocols Video EpisodeTreatment GroupMeanStandard DeviationN 1Control83.6718.827 Experimental88.6920.988 2Control82.7421.847 Experimental98.622.888 3Control78.2920.517 Experimental96.574.867 4Control52.7510.357 Experimental71.9820.327
Table 4 Mean Scores and Standard Deviations on the Exit Survey QuestionMean Standard Deviation Using Video Guides helped me to stay focused during video viewing.2.050.42 Using Video Guides helped me to remember the video episodes better.2.000.61 In my opinion, using an interactive Video Guide may help me learn more from the video episodes. 2.000.50 I would like to continue using a Viewing Guide while watching «Начало» episodes next year. 1.820.52 I liked the design of the current Video Guide.2.170.80 The Video Guide distracted me from watching the video.3.820.95
Limitations The study was conducted at a small liberal arts undergraduate institution. The nature of the population may present questions about the role of attitude, interest, and motivation in FL video viewing. Because all students were in the same class, there was no random selection of participants. The sample size was limited to 16 students. Randomizing the sample and broadening its size might enhance the validity of the study. In the future, incentives may be incorporated to decrease the number of missing observations. The findings are based on a particular kind of videotext, a simulated authentic discourse based on a fictional narrative, that might not be applicable to other types of videotext (e.g., news broadcasts or interviews). The results may not generalize to studies that employ different assessment instruments. A longitudinal study over several semesters may be better suited for the purpose.
Contact Information: Dr. Luba Iskold 2400 Chew Street Muhlenberg College, Languages, Literatures and Cultures, Allentown, PA 18104 Phone: 484-664-3516 Fax: 484-664-3722 E-mail: firstname.lastname@example.org@muhlenberg.edu http://www.muhlenberg.edu/depts/forlang/LLC/iskold_home/index.htm