Writing Analytics Clayton Clemens Vive Kumar

What are Writing Analytics? Analysis is the science of breaking a thing down into its constituent parts. The constituent parts discovered during an analysis are referred to as analytics. Analytics are usually quantitative. Writing analytics are the properties derived from analyzing a piece of writing.

History The field of writing analytics pre-dates computers. Is writing assessment the same as writing analytics? Early writing analytics involved manual counting of sentences, words, and even syllables. Computer programming began with very specific rigid command structures (grammars). English and other natural languages also have grammars, though they are far more complex. The inception of artificial intelligence represented the first foray into computer understanding of natural language.

Natural Language Processing NLP uses sophisticated software tools and AI techniques to analyze text. There are many different things that can be determined from natural language. NLP has provided us with the following tools: Tokenization Sentence Recognition Part of Speech Tagging Parsing (organizing words in a sentence into a syntactic structure) Named Entity Recognition Document Categorization Co-Reference Resolution Sentiment Analysis Rule-Based Grammar Checking

From NLP to Analytics Using the tools that NLP provides, we can determine a number of quantitative properties about a piece of writing: Number of simple/compound/complex sentences (parsing) Lexical diversity Readability Grammatical accuracy (Rule-Based Grammar Checking) Personal pronoun usage (NER) Connectivity (POS Tagging, Parsing) Modifier Complexity (POS Tagging) Noun-Phrase Complexity (Parsing) Tense Agreement (POS Tagging) Content vs. Function Words (POS Tagging)

From NLP to Analytics In fact, some of these tools, along with research in the field, can help us determine things that are considered to be subjective. Cohesion (POS Tagging, Parsing, Coreference Resolution) Imagery Familiarity Concreteness Positivity (Sentiment Analysis) These things represent a host of different data points by which to measure a piece of writing.

MI-Writer – One Step Further These analytics and more are used in many automated essay scoring (AES) systems. AES uses analytics to produce a grade similar to what an experienced marker would. AES assumes a finished composition. MI-Writer will track each stage of a student’s writing work via ‘snapshots’. Information about the student’s entire process is collected.

So What? MI-Writer will allow students and instructors to see an unprecedented level of detail when assessing and reviewing writing assignments. Students can see points where their composition became weaker or stronger in certain areas. They will be able to identify problem areas and target them for improvement. Instructors will be able to give detailed and specific feedback on pivotal points in the writing process. Structure and writing order can be reviewed, not just content. Growth of learner competencies can be measured over months or years. Analytics can be further processed to produce formal models of competency growth.

What’s Next? An early version of MI-Writer is currently in use in a study in India, in collaboration with VNR VJIET and Anna University. This study will track the progress of ESL students in an English composition course. New features for analytics visualization and feedback will be identified, and the scalability and efficiency of MI-Writer will be assessed through the study.

Thanks for Watching