1. Multimodality and Activity Theory Methodological issues in their combination Dr. Mohammed Alhuthali Taif University, Saudi Arabia Email: mohuthali@gmail.com

2. Multimodality and Activity Theory These share a number of core assumptions: –Both share the assumption that a particular task or information is given meaning in a given setting by a particular group of people; –Both acknowledge the importance of context in creating this meaning making; –Both acknowledge the importance of the interaction of modes of meaning making (speech, gesture, tool use) in creating an overall understanding; –One issue is how to combine as there are differences?

3. Jewitt’s methodology Jewitt (2005, 2008) combined the two approaches; Used Activity Theory to understand the context in which the learning took place. In effect, the multimodal concept of resemiosis was argued not to be sufficient in this respect; Used Activity Theory to gain an overview of the flow of an entire learning process; However, only in terms of semiotics and multimodality, she only concentrated on shifts of speech

4. Adapting Jewitt’s Approach Student learning during a PBL class spread over a number of weeks was video-taped (4 hours in total); This was first analysed using the concepts of Activity Theory; From this, seven session were selected that typified key stages in the meaning making process. These were analysed in a full multimodal approach, combining speech, gesture, tool use and context.

5. Context: Problem-Based Learning (PBL) PBL is a student-centred learning technique where they are given a problem and expected to solve it with minimal intervention; It is conventional for a tutor to work with the group to facilitate this learning process (sometimes this role is now taken by IT); In reality, it is still task based. There are rules, some provided by the discipline (in this case Engineering), some by the academic setting (a fixed period of time, a particular approach to assessment); As such the tutor has a complex role of enabling student centred learning but also ensuring they operate to the requirements of the task.

6. An Activity Theory Representation

7. Activity Theory Using the concepts of activity theory it was possible to distinguish between the meaning making by the student group and that by the facilitator (students are in red in the diagram) In some respects these were similar, so the object ‘to complete the model bridge’ was shared; In others there were significant differences, in particular in terms of the rules and the expected division of labour

8. Multimodality The strength of bringing in a multimodal analysis is it allows a more detailed exploration of how these key differences occurred and were represented; Various techniques were adopted including counting of instances – such as how many times techniques to encourage student learning such as ‘scaffolding’ were adopted; Most of the analysis was the detailed interplay of speech, gesture and tool use to explore the different ways in which meaning making was carried out.

9. Multimodality – detailed analysis

10. Multimodal Approach Each phase of meaning making (verbal and non-verbal) was coded mostly in terms of the categories developed by Hmelo-Silver; This allows a detailed comparison of how different modes of meaning making varied according to shifting nature of the task or in terms of who was leading the meaning making (facilitator or students).

11. Multimodal Analysis

12. This shows a typical comparison, in this case between the use of verbal semiotics by the students and the tutor. Key is the students relied much more on brief and simple agreement (BA and AM) in the tables; The tutor tended to rely on the use of longer explanations (ET) in the table. In combination with other analyses, this stressed that the tutor had a different role (as identified in the Activity Theory analysis) and that this different role was marked by the use of different semiotic tools and techniques

13. Wider Findings Student meaning making varied as they progressed from understanding the theory behind the bridge design and the expectations of the test that would be applied; When they were building the bridge, they tended to rely on gesture and tool use, when they were discussing the theory, they tended to rely on verbal meaning making

14. Advantages Activity Theory made it easier to understand the importance of context. Despite the claims of PBL, this was not open-ended student problem solving, it was directed problem solving in the artificial confines of an academic task with a timescale and a set assessment process; AT helped to identify key stages where different modes of meaning making were used, or where the task focus shifted; Multimodality allowed a detailed analysis not just that there were differences at this stages but in terms of how meaning making differed.

15. Disadvantages Despite the advantages of combining the two, we still can only observe what is done. Interviews, or some form of reflective reports by the participants, would allow an understanding of how they understood the task and the logic for their actions; AT has a clear reporting structure, multimodality tends to adapt to the particular research being undertaken. As such, it is not always feasible to map the findings between the two methods.

16. Thanks for Giving Attention Questions?