1. A Study Into The Effectiveness of Web-based Reading LI Tianjiao FENG Chenyue ZHAO Fei QU Qu YAN Li

2. Agenda Motivation and Background Design of Experiment Data Analysis and Conclusion

3. The rapid development of computer and Internet technologies has made e-Learning become an important learning method. However, when we surfing on the internet, we usually fail to obtain information in an effective or efficient way. So we are desired to know how to obtain information effectively, accurately, and efficiently. 3 Motivation

4. 4 As the increasing number of tests (IBT, new CET 4 &6)converted from paper-based to internet-based, more attention is paid or will be paid to obtaining information, ideas, key points from reading materials. There are so many factors influencing on obtaining information or the leaning through web-pages, such as: The layout of web-pages; The amount & format of assistant information on web-pages; The amount & format of irrelative information on web-pages; The length & difficulty of reading materials themselves on web-pages; The English skill levels of different individuals; So on…… Motivation

5. 5 Investigate the factors that contribute to the effectiveness of learning through web-pages; Figure out the way to build a easy-to-learn web learning system; Enhance the effectiveness and efficiency of web learning. Objectives

6. 6 Expert Knowledge/Experience One of the key characteristics of e-Learning is its capability to integrate different media, such as text, picture, audio, animation and video to create a multimedia instructional material, promoting the reading interests and willingness of the learner. ( Gillani & Relan, 1997; Vichuda, Ramamurthy, & Haseman, 2001) Some research shows that too much unnecessary multimedia elements in instructional material may distract learners and actually decrease learning performance. (Bartscha & Cobern, 2003; Mayer et al., 2001; Park & Hopkins, 1993; Rieber, 1996;Sweller, van Merrienboer, & Paas, 1998) Relevant Background

7. 7 Ho, J. & Ko, L. (2003), Design for Experimenting in E-Learning: The Effects of Web Page Design on Students' Test Performance. In G. Richards (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2003 (pp. 1610-1613). Chesapeake, VA: AACE Previous Experiments Response variable: Test scores Control variable: 1. Message area location 2. Function area location Relevant Background

8. 8 Response variable (units) Normal operating level & range Meas. precision, accuracy How known? Relationship of response variable to objective Time used to complete reading the corresponding passage 7 minutes; From 7-3 min to 7+3 min per passage considering different individuals. By timer; Unit: second The shorter the required time, the higher learning effectiveness on web- pages. Correct rate Correct rate around 0.6 to 1.0, in common per passage. Correct rate(the ratio of the number of correct answers to all the answers) The higher the correct rate, the higher learning effectiveness on web- pages. Response Variables

9. 9 The layout of reading materials: The number of columns of the reading materials; One column with passage and questions; Two columns, one column with passage only and the other with questions only. The existence of assistant information: Whether there is an outline in front of reading materials; Yes No The existence of irrelative information: Whether there are some irrelative pictures in reading materials; Yes No Control Variables

10. 10 The length of reading materials: Around 270 words; 5 choice questions each with 4 choices. The difficulty of reading materials: CET 4 passages; Arguments with 5-6 paragraphs Environment factors: Noise, illumination, temperature; Others: Possible factors related to computers; The time of experiment. Can they be controlled constantly with deliberately selecting? Held constant Variables

11. 11 The difference between different individuals: The learning capability; The English skill level; The reading habit; The gender, sight; The mood. Block: Different individuals can be treated as a set of blocks. Nuisance Factors

12. Design of Experiment

13. Define Control Variables Deal with Nuisance Variables Verify Response Variables 13

14. Control and Response Variables Reading time Correct rate Number of columns Outline Irrelevant information

15. Define Control Variables Control V: Column, irrelevant information, outline Full Factorial Design

16. Deal with Nuisance Factors Possible nuisance How to eliminate the effects of these variables? Environment?  Held constant Passage?  Held constant. If can’t, block People?  Hard to be held constant. Block First, we want to know the effects of these variables People Passage Environment

17. Trial Runs Objective: effects of PEOPLE, PASSAGES Controlled factors  held constant (One column; No outline; No irrelevant Information) Environment  held constant (No. 4 Teaching Building, 9:00-10:00PM) 8 people 4 passages per person We tried to find passages of the same level

18. 双因子方差分析 : time(min) 与 people, Passage 来源 自由度 SS MS F P people 7 9.7587 1.39410 3.20 0.018 Passage 3 2.2559 0.75197 1.73 0.192 误差 21 9.1346 0.43498 合计 31 21.1492 S = 0.6595 R-Sq = 56.81% R-Sq （调整） = 36.24% People are very different (block) Passage are the same (held constant)

19. Response: Correct rate 双因子方差分析 : Correct rate 与 people, Passage 来源 自由度 SS MS F P people 7 0.10875 0.0155357 0.69 0.683 Passage 3 0.03375 0.0112500 0.50 0.689 误差 21 0.47625 0.0226786 合计 31 0.61875 S = 0.1506 R-Sq = 23.03% R-Sq （调整） = 0.00% People are the same(held constant) Passages are the same(held constant)

20. Response Variables: time and correct rate If they are related, correct rate may result from time, rather other control variables. We need to fix the response variable Longer reading time  reading is difficult  lower correct rate Longer reading time  more effort devoted  higher correct rate

21. 相关 : Correct rate, time(min) Correct rate 和 time(min) 的 Pearson 相关系数 = -0.020 P 值 = 0.914 Not correlation No need to fix the response Correlation between Time and Correct rate? Yes Find another response variable that eliminates the correlation. X=Correct rate/ Time No Analysis two distinct response variables with control variables.

22. Design of Experiment (1) ac ab bc Block 1 abc a b c Block 2 (1) ac ab bc Block 3 abc a b c Block 4 Factorial design with ABC confounded 4 treatment per block, 8 blocks, 32 run totally Time (1) ac ab bc Block 5 abc a b c Block 6 (1) ac ab bc Block 7 abc a b c Block 8

23. Factorial design with ABC confounded With no block, 32 runs totally Correct Rate Design of Experiment

24. Experiment Size The experiment is quite expensive ▫ The subjects can experiment with one passage only once ▫ The reading of one passage takes 7±3 minutes, 4 passages will make the total time around 30 minutes ▫ As the time increase, subjects may become impatient and tired, causing bias

25. Data Analysis and Conclusion

26. Analysis of Factorial Design 拟合因子 : Time 与 Blocks, Column, Outline, Irrelevant * 注 * 此设计具有一些修补运行。将使用回归方法进行分析。 Time 的效应和系数的估计（已编码单位） 项 效应 系数 系数标准误 T P 常量 5.9198 0.2378 24.89 0.000 Blocks -1.2111 -0.6056 0.3633 -1.67 0.110 Column 1.0396 0.5198 0.2378 2.19 0.040 Outline 0.0279 0.0140 0.2378 0.06 0.954 Irrelevant 0.1383 0.0692 0.2378 0.29 0.774 Blocks*Column -0.1514 -0.0757 0.3723 -0.20 0.841 Blocks*Outline -0.3718 -0.1859 0.3723 -0.50 0.623 Blocks*Irrelevant 0.0157 0.0079 0.3723 0.02 0.983 Column*Outline 0.5007 0.2503 0.2437 1.03 0.316 Column*Irrelevant -0.6411 -0.3205 0.2437 -1.32 0.203 Outline*Irrelevant 0.2233 0.1117 0.2437 0.46 0.65 When column= -1, reading time is less than column =1. Two column layout is efficient in receiving information from reading. Time

27. Analysis of Factorial Design(cont’d) Column*Outline = -1 format pattern is effective in receiving information from reading. 拟合因子 : Correctness 与 Column, Outline, Irrelevant Correctness 的效应和系数的估计（已编码单位） 项 效应 系数 系数标准误 T P 常量 0.81875 0.02453 33.38 0.000 Column 0.06250 0.03125 0.02453 1.27 0.214 Outline -0.03750 -0.01875 0.02453 -0.76 0.452 Irrelevant 0.03750 0.01875 0.02453 0.76 0.452 Column*Outline -0.11250 -0.05625 0.02453 -2.29 0.031 Column*Irrelevant 0.06250 0.03125 0.02453 1.27 0.214 Outline*Irrelevant 0.01250 0.00625 0.02453 0.25 0.801 Correct Rate

28. Design Projection Discard D (Irrelevant information) unchanged results Residual Checking: Conclusion is reasonable; Underlying assumptions are satisfied.

29. Discussion Q1: Why format with outline could not increase the efficiency of reading, that is, reduce the time on reading? A: 1. Outline lies on the top of web page. 2. People don’t have habit of reading web pages with outline given. Q2:Why irrelevant information isn’t significant in response time and correct rate? A: 1. The effect of irrelevant information is less than system error. This problem could be solved through large amount of runs. 2. The effect of variable is actually slight and ignorable.

30. Conclusion TimeCorrect Rate Better ConditionTwo columnTwo column, with outline ApplicationPassage is long; Retral part is relevant to the preceding information. Passage is long; Passage with many paragraphs; Quick reading is required. Conclusion and advice on designing the web page: 1.Outline could be designed as a moving bar, so that people could see it anywhere. 2.Irrelevant information such as advertisement at normal level won’t bother people and interfere the process of reading.