Presentation on theme: "Learning analytics. what are learning analytics? related fields of study processesresources implementation tips a model for learning analytics references."— Presentation transcript:
what are learning analytics? related fields of study processesresources implementation tips a model for learning analytics references where are we now? literature review
learning analytics are: the ability to “scale the real-time use of learning analytics by students, instructors and academic advisors to improve student success” - Next Generation: Learning Challenges main page next page: learning analytics involves
learning analytics involves: main page 1.the development of new processes and tools aimed at improving learning and teaching for individual students and instructors 2. the integration of these tools and processes into the practice of teaching and learning related links next page: related fields of study
related fields of study action analytics academic analytics web analytics business intelligence main page
business intelligence: a well-established process in the business world whereby decision makers integrate strategic thinking with information technology to be able to synthesize “vast amounts of data into powerful, decision making capabilities” - Baker, 2007 main page next page: web analytics
web analytics: “the collection, analysis and reporting of Web site usage by visitors and customers of a web site” in order to “better understand the effectiveness of online initiatives and other changes to the web site in an objective, scientific way through experimentation, testing, and measurement” - McFadden, 2005 main pagerelated links next page: academic analytics
academic analytics: the application of the principles and tools of business intelligence to how institutions gather, analyze, and use data to improve student success -Campbell and Oblinger, 2007 & Goldstein and Katz, 2005 main page next page: action analytics related links
action analytics: involves deploying academic analytics “ to provide actionable intelligence, service-oriented architectures, mash-ups of information/content and services. proven models of course/curriculum reinvention, and changes in faculty practice that improve performance and reduce costs - Norris et al, 2008 main page next page: learning analytics processes
learning analytics processes data gathering data gathering sharing knowledge application knowledge application information processing information processing aggregate capture select predict use refine main page
Large store of data already exist and computer-mediated distance education increasingly creates student data trails. Most often exists in disjointed and meaningless forms. main page next page: information processing data gathering data gathering capture select There are so many metrics that could be tracked, it is essential to define goals and identify relevant data. What do we want to achieve? Are we measuring what we should be? How can we create innovative metrics?
To be usable, we must be able to aggregate that data into a meaningful form. Dashboards and social network analysis are two promising tools. main page next page: information processing information processing aggregate predict Data is useful when it can be used to predict future events. To date, however, no guidance it available to educators to indicate which captured variables are pedagogically meaningful. Outside of education, search engines and recommenders sites are examples of aggregating information and using it to predict user needs.
In order to be a knowledge discovery cycle, data and actions must be re-presented to users. Otherwise, it is just data mining. main page next page: analytics tools knowledge application use refine Analytics are a self-improvement project. Monitoring impact must be a continual effort, the results of which are used to update the models and improve predictions.
main page next page: analytics tools When institutions work together and share, duplication is reduced and improvements are increased. Sharing data, models and innovations, therefore, has the potential to improve learning for everyone. sharing
learning analytics resources main page There are four types tools that must interact for learning analytics to be successful. Organizations Computers PeopleTheory...a single amalgam of human and machine processing which is instantiated through an interface that both drives and is driven by the whole system, human and Machine - Dron and Anderson, 2009
Sophisticated computers already collect data. They also facilitate data processing with visualization tools because we can process an incredible amount of information if it is packaged and presented correctly. Two promising visualization tools for learning analytics are dashboards and social networks maps. main pagerelated links next page: dashboards Organizations Computers PeopleTheory computers
main page Organizations Computers PeopleTheory dashboards Meaningful information can be can be extracted from CMS/LMS and be made available to students and instructors. next page: social network analysis related links
main page social network maps next page: theory Automates the process of extraction, collation, evaluation and visualisation of student network data into a form quickly usable by instructors. Organizations Computers PeopleTheory related links
Computer hardware and software are only useful if they are based on sound theory. main page next page: people theory Organizations Computers People Theory Social networks maps, for example, are only useful because of sound research-based theory that demonstrates we learn better when we interact with others.
There are still a significant aspects of an analytics system that require human knowledge, skills and abilities to operate. main pagemore information next page: organizations people Organizations Computers People Theory Developing effective learning interventions remains highly dependent on human cognitive problem-solving and decision-making skills.
Social networks maps, for example, are only useful because of sound research-based theory that shows peer networks play an important role in student persistence and overall success. main page next page: organizations organizations Organizations Computers PeopleTheory
main page next page: where are we now? Organizations Computers People Theory sharing knowledge application knowledge application information processing information processing aggregate predict use data gathering data gathering a model for learning analytics capture select refine
where are we now? Learning analytics is an emerging field. Analytics is other fields is already well established. Tools and lessons learned from other fields can be used to support the introduction of learning analytics to the majority. main page next page: tips for analytics more information
implementation tips 1.Learn from others disciplines in which analytics is an established field 2.Find out what you are already measuring 3.Combine web-based data with traditional evaluation, assessment and demographic data 4.Good communication skills are essential 5. Change is hard for everyone and rarely welcome - tread lightly and offer support main page next page: references
references Arnold, K. E. (2010). Signals: Applying Academic Analytics, EDUCAUSE Quarterly 33(1). Retrieved October 1, 2010 from http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/Si gnalsApplyingAcademicAnalyti/199385 http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/Si gnalsApplyingAcademicAnalyti/199385 Astin, A. (1993). What Matters in College? Four Critical Years Revisited. San Francisco: Jossey-Bass. Baker, B. (2007). A conceptual framework for making knowledge actionable through capital formation. D.Mgt. dissertation, University of Maryland University College, United States -- Maryland. Retrieved October 19, 2010, from ABI/INFORM Global.(Publication No. AAT 3254328). Dron, J. and Anderson, T. (2009). On the design of collective applications, Proceedings of the 2009 International Conference on Computational Science and Engineering, Volume 04, pp. 368-374. Goldstein, P. J. and Katz, R. N. (2005). Academic Analytics: The Uses of Management Information and Technology in Higher Education, ECAR Research Study Volume 8. Retrieved October 1, 2010 from http://www.educause.edu/ers0508http://www.educause.edu/ers0508 next page: references (cont’d)
references (continued) McFadden, C. (2005). Optimizing the Online Business Channel with Web Analytics [blog post]. Retrieved October 5, 2010 from http://www.webanalyticsassociation.org/members/blog_view.asp?id=533997&post=8932 8&hhSearchTerms=definition+and+of+and+web+and+analytics http://www.webanalyticsassociation.org/members/blog_view.asp?id=533997&post=8932 8&hhSearchTerms=definition+and+of+and+web+and+analytics NextGeneration: Learning Challenges (n.d.). Learning Analytics [website]. Retrieved October 12, 2010 from http://nextgenlearning.com/the-challenges/learning-analyticshttp://nextgenlearning.com/the-challenges/learning-analytics Norris, D., Baer, L., Leonard, J., Pugliese, L. and Lefrere, P. (2008). Action Analytics: Measuring and Improving Performance That Matters in Higher Education, EDUCAUSE Review 43(1). Retrieved October 1, 2010 from http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume4 3/ActionAnalyticsMeasuringandImp/162422http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume4 3/ActionAnalyticsMeasuringandImp/162422 Zhang, H. and Almeroth, K. (2010). Moodog: Tracking Student Activity in Online Course Management Systems. Journal of Interactive Learning Research, 21(3), 407-429. Chesapeake, VA: AACE. Retrieved October 5, 2010 from http://0- www.editlib.org.aupac.lib.athabascau.ca/p/32307.http://0- www.editlib.org.aupac.lib.athabascau.ca/p/32307
what are learning analytics? links What are Learning Analytics: elearn Space George Siemens's preliminary thoughts on learning analytics. He tends to focus on the social/ connected aspects of learning analytics. This view leads to the questions: Are the fields of learning analytics and academic analytics the same or different? At what levels can/ should we be applying analytics? How much of analytics can be system-driven? User-driven? Instructor/facilitator/institution driven? Learning analytics Google group Interesting discussion and good bibliography on the topic. Next Generation Learning Challenges: Learning Analytics A good list of resources and examples of current projects underway. backmain page
web analytics links Beyond Clickstream: Keeping Pace With Web Analytics An interesting article on the field of web analytics and future directions in the field. Supporting Smart Interactions with Predictive Analytics Although not aimed at the field of education, this article speaks about a future in which online interactions are "user-centric" and "responsive to user needs." It also talks about the development of systems that are able to "provide cognitive support that helps a user to think about and solve their problems." Learning Web Analytics – The Top 10 Things I Wish I Knew When I Started I great list of lessons learned related to web analytics that is also very applicable to learning analytics. backmain page
academic analytics links Academic Analytics - 27 Resources A good list of Academic articles, presentations and blogs related to academic analytics. This is still a good place to start identifying what has been done so far in the field, what challenges the implementation of analytics in an educational institution is likely to encounter, and a starting point for solutions. Academic Analytics and Data Mining in Higher Education A good overview of a series of academic analytics projects underway. Perspectives - Academic Analytics A 10-minute video that describes the experiences of Purdue University during the implementation of their Signals learning analytics project. Administrator's and instructors speak of some of the benefits and issues that emerged as a part of the project. backmain page
computer tools links Information visualization Information visualization is widely thought to be one of the keys to the development of "actionable knowledge" - the key to successful analytics. It overviews the field of information visualization for those with little knowledge of the field. Of particular interest with respect to learning analytics, it speaks of the new field of "visualization analytics" and the emerging trend of social network analysis. It also offers concrete examples and images of a variety of information visualization currently in use. 2010 Educational Data Mining Conference Proceedings A comprehensive collection of papers. They are also interesting in that many of them are looking at how concepts from the worlds of math and business do have relevant applications in education. Three interesting developments to watch: EDM Visualization Tool, A Data Model to Ease Analysis and Mining of Educational Data (looking at ways to more easily mine Moodle data), and Data Ming for Individualized Hints in elearning (using the questions others have got wrong to provide timely hints and helps for students). Tools for doing learning analytics in open education A list of tools for collecting and analysing data in open education. backmain page
dashboard links Using Analytics to Intervene with Underperforming College Students (Innovative Practice) A 1-hour panel presentation on analytics and how it is being used at three different universities. Provides a number of examples of what is currently being done. Power point slides and full recording of panel available (although you need to download Silverlight to view it). backmain page
social networks analysis links Learning Networks at the University of Wollongong Describes UOW work on learning networks and social network analysis. From the site you can also learn about and download their visualization tool SNAPP (Social Networks Adapting Pedagogical Practice) which can be used with an LMS to visualize social networks. Handbook of Social Network Technologies and Applications A comprehensive resource on the study of social networks. It is divided into five parts parts: (1) Social Media Analysis and Organization, (2) Social Media Mining and Search, (3) Social Network infrastructures and communities, (4) Privacy in online Social Networks, (5) Visualization and applications of Social networks. Thinking out loud: Generating social graphs from RSS in an Open Course In this blog post Leal talks out his process to "see" what is happening in an open course. Ultimately he concludes: “And if we want to get more serious about being open we will need, eventually, to provide alternative solutions easy to use for administrators, teachers and students, and analytical tools focused on the kind of things we'd like to observe and foster in our students.” backmain page
people links Evidence for a Collective Intelligence Factor in the Performance of Human Groups An interesting article that finds evidence of collective intelligence in groups. That is, groups do well on certain tasks will tend to also do well on other unrelated tasks. Interestingly they also found however that collective intelligence was not correlated to the individual intelligence of group members, but instead to the social sensitivity of the group members and the equality of the turn-taking in the group. These finding will have relevance when considering the creation and support of teams seeking to consider complex analytics issues related to education and learning. Creating and leading analytics teams Although not specifically related to education, this articles looks at the similarities of analytics across disciplines including finance, medicine, and intelligence. It then looks at the "hows" of setting up a successful analytic group. Findings in this paper may be a good starting point for getting away from talking about analytics theory and getting learning analytics projects mobilized. backmain page