By: Niraj Kumar Automatic Essay Grading Novelty Detection 1. 2.

Slides:

Advertisements

Similar presentations

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki

Advertisements

Multi-Document Person Name Resolution Michael Ben Fleischman (MIT), Eduard Hovy (USC) From Proceedings of ACL-42 Reference Resolution workshop 2004.

Introduction to: Automated Essay Scoring (AES) Anat Ben-Simon Introduction to: Automated Essay Scoring (AES) Anat Ben-Simon National Institute for Testing.

Proceedings of the Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2007) Learning for Semantic Parsing Advisor: Hsin-His.

Electronic Essay Graders Jay Lubomirski.  How electronic essay graders evaluate writing samples  Comparing the electronic graders to the human graders.

Net-centric User Knowledge Assessment Chris Isgrig CMSC 601_SP11.

Sentiment Analysis An Overview of Concepts and Selected Techniques.

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts 04 10, 2014 Hyun Geun Soo Bo Pang and Lillian Lee (2004)

Predicting Text Quality for Scientific Articles Annie Louis University of Pennsylvania Advisor: Ani Nenkova.

Predicting Text Quality for Scientific Articles AAAI/SIGART-11 Doctoral Consortium Annie Louis : Louis A. and Nenkova A Automatically.

Dissemination and Critical Evaluation of Published Research Peg Bottjen, MPA, MT(ASCP)SC.

Semantic text features from small world graphs Jure Leskovec, IJS + CMU John Shawe-Taylor, Southampton.

Detecting Near Duplicates for Web Crawling Authors : Gurmeet Singh Mank Arvind Jain Anish Das Sarma Presented by Chintan Udeshi 6/28/ Udeshi-CS572.

1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,

NICK PENDAR AND ELENA COTOS IOWA STATE UNIVERSITY THE 3RD WORKSHOP ON INNOVATIVE USE OF NLP FOR BUILDING EDUCATIONAL APPLICATIONS JUNE 19, 2008 Automatic.

Latent Semantic Analysis (LSA). Introduction to LSA Learning Model Uses Singular Value Decomposition (SVD) to simulate human learning of word and passage.

Named Entity Disambiguation Based on Explicit Semantics Martin Jačala and Jozef Tvarožek Špindlerův Mlýn, Czech Republic January 23, 2012 Slovak University.

Finding Advertising Keywords on Web Pages Scott Wen-tau YihJoshua Goodman Microsoft Research Vitor R. Carvalho Carnegie Mellon University.

Digging by Debating (DbyD):

GL12 Conf. Dec. 6-7, 2010NTL, Prague, Czech Republic Extending the “Facets” concept by applying NLP tools to catalog records of scientific literature *E.

Automated Essay Evaluation Martin Angert Rachel Drossman.

Some studies on Vietnamese multi-document summarization and semantic relation extraction Laboratory of Data Mining & Knowledge Science 9/4/20151 Laboratory.

CONTI’2008, 5-6 June 2008, TIMISOARA 1 Towards a digital content management system Gheorghe Sebestyen-Pal, Tünde Bálint, Bogdan Moscaliuc, Agnes Sebestyen-Pal.

Intrusion Detection Jie Lin. Outline Introduction A Frame for Intrusion Detection System Intrusion Detection Techniques Ideas for Improving Intrusion.

Slide Image Retrieval: A Preliminary Study Guo Min Liew and Min-Yen Kan National University of Singapore Web IR / NLP Group (WING)

Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification on Reviews Peter D. Turney Institute for Information Technology National.

Extracting Key Terms From Noisy and Multi-theme Documents Maria Grineva, Maxim Grinev and Dmitry Lizorkin Institute for System Programming of RAS.

Citation Recommendation 1 Web Technology Laboratory Ferdowsi University of Mashhad.

Towards Improving Classification of Real World Biomedical Articles Kostas Fragos TEI of Athens Christos Skourlas TEI of Athens

2007. Software Engineering Laboratory, School of Computer Science S E Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying.

Learning to Classify Short and Sparse Text & Web with Hidden Topics from Large- scale Data Collections Xuan-Hieu PhanLe-Minh NguyenSusumu Horiguchi GSIS,

Newsjunkie: Providing Personalized Newsfeeds via Analysis of Information Novelty Gabrilovich et.al WWW2004.

1 The Ferret Copy Detector Finding short passages of similar texts in large document collections Relevance to natural computing: System is based on processing.

Computing & Information Sciences Kansas State University Boulder, Colorado First International Conference on Weblogs And Social Media (ICWSM-2007) Structural.

Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA ImprovingWord.

Similar Document Search and Recommendation Vidhya Govindaraju, Krishnan Ramanathan HP Labs, Bangalore, India JOURNAL OF EMERGING TECHNOLOGIES IN WEB INTELLIGENCE.

Automated Writing Evaluation(AWE): Past, Present and Prospect Dr. Li Zhang ( 张荔） Shanghai Jiao Tong University Shanghai, China.

Relevance Detection Approach to Gene Annotation Aid to automatic annotation of databases Annotation flow –Extraction of molecular function of a gene from.

LATENT SEMANTIC INDEXING Hande Zırtıloğlu Levent Altunyurt.

Why Not Grab a Free Lunch? Mining Large Corpora for Parallel Sentences to Improve Translation Modeling Ferhan Ture and Jimmy Lin University of Maryland,

Learning Multilingual Subjective Language via Cross-Lingual Projections Mihalcea, Banea, and Wiebe ACL 2007 NLG Lab Seminar 4/11/2008.

Bootstrapping for Text Learning Tasks Ramya Nagarajan AIML Seminar March 6, 2001.

Automated Suggestions for Miscollocations the Fourth Workshop on Innovative Use of NLP for Building Educational Applications Authors:Anne Li-E Liu, David.

Exploring a topic in depth... From Reading to Writing Questioning the world around us is a natural instinct. We read articles in the paper. We read information.

Algorithmic Detection of Semantic Similarity WWW 2005.

1/21 Automatic Discovery of Intentions in Text and its Application to Question Answering (ACL 2005 Student Research Workshop )

WEB 2.0 PATTERNS Carolina Marin. Content  Introduction  The Participation-Collaboration Pattern  The Collaborative Tagging Pattern.

2015/12/121 Extracting Key Terms From Noisy and Multi-theme Documents Maria Grineva, Maxim Grinev and Dmitry Lizorkin Proceeding of the 18th International.

Creating Subjective and Objective Sentence Classifier from Unannotated Texts Janyce Wiebe and Ellen Riloff Department of Computer Science University of.

Presented By- Shahina Ferdous, Student ID – , Spring 2010.

Multi-level Bootstrapping for Extracting Parallel Sentence from a Quasi-Comparable Corpus Pascale Fung and Percy Cheung Human Language Technology Center,

Comparative Experiments on Sentiment Classification for Online Product Reviews Hang Cui, Vibhu Mittal, and Mayur Datar AAAI 2006.

Learning Subjective Nouns using Extraction Pattern Bootstrapping Ellen Riloff School of Computing University of Utah Janyce Wiebe, Theresa Wilson Computing.

Using Linguistic Analysis and Classification Techniques to Identify Ingroup and Outgroup Messages in the Enron Corpus.

Relevance Models and Answer Granularity for Question Answering W. Bruce Croft and James Allan CIIR University of Massachusetts, Amherst.

CONTEXTUAL SEARCH AND NAME DISAMBIGUATION IN USING GRAPHS EINAT MINKOV, WILLIAM W. COHEN, ANDREW Y. NG SIGIR’06 Date: 2008/7/17 Advisor: Dr. Koh,

Final Report (30% final score) Bin Liu, PhD, Associate Professor.

Using Bayesian Belief Networks in Assessing Software Architectures Jilles van Gurp & Jan Bosch.

1 ICASSP Paper Survey Presenter: Chen Yi-Ting. 2 Improved Spoken Document Retrieval With Dynamic Key Term Lexicon and Probabilistic Latent Semantic Analysis.

A Survey on Automatic Text Summarization Dipanjan Das André F. T. Martins Tolga Çekiç

GRAPH BASED MULTI-DOCUMENT SUMMARIZATION Canan BATUR

Sentiment analysis algorithms and applications: A survey

A Brief Introduction to Distant Supervision

Web News Sentence Searching Using Linguistic Graph Similarity

6th Grade Research Project Almeria Middle School 2013

Improving a Pipeline Architecture for Shallow Discourse Parsing

Applying Key Phrase Extraction to aid Invalidity Search

Introduction of KNS55 Platform

Information Retrieval

PURE Learning Plan Richard Lee, James Chen,.

Presentation transcript:

By: Niraj Kumar Automatic Essay Grading Novelty Detection 1. 2.

One or multiple model essays Test Essays Output: %score or Essay-class Essay grading by using “Model Essays(s)” Source document(s) Test essays Output: %score or Essay-class Essay Grading with “Source Document(s)” Some quality specific samples Test essays Output: Essay-class Essay Grading without “Model Essays” Automatic Essay Grading System

Our Contribution

Word Graph of Text TEXT: 1. “Fighters were developed in World-War-I to deny enemy aircraft and dirigibles the ability to gather information by reconnaissance. 2. Early fighters were very small and lightly armed by later standards, and most were biplanes built with a wooden frame, covered with fabric, and limited to about 100 mph. 3. As control of the airspace over armies became increasingly important all of the major powers developed fighters to support their military operations.”

Feature Extraction-1 : Relevance of information

Total Correlation

Feature Extraction-2 : Presence of sparsely connected words Topical Term - T Topical Term - B Topical Term - A Topical Term - E Topical Term - D Topical Term - C - Used Weakly connected components of EGO-Network to identify the weakly connected components

Feature Extraction-3: Statistical and semantic role of words

Feature Extraction-4: Presence of talkative terms

Essay Grading

Pseudo Code

Evaluation System% Accuracy High School essays (Two-groups) Grade 5 essays (Three-groups) According to dataset 10-Fold cross validation According to dataset 10-Fold cross validation Our devised system88.00%87.89%63.00%63.01% Betsy80.00%78.00%51.00%51.20%

Novelty Detection Document Similarity Analysis Plagiarized Text Non-Plagiarized Text May Contain Novel Information May contain Noisy Information

Novelty Detection (Current Status) Plagiarism Detection Identifying “Copy-Paste” based plagiarism 1.Hashing and String matching based techniques are available 2.Systems are effective. Identifying “Copy-Paste” based plagiarism 1.Hashing and String matching based techniques are available 2.Systems are effective. Identifying “Word-re-order” based plagiarism 1.Challenging problem 2.Scholary articles Identifying “Word-re-order” based plagiarism 1.Challenging problem 2.Scholary articles Current Status: 1.Successfully designed the approach to meet both requirements 2.Trying to extend the work towards: 1.Information reuse in scholary articles 2.Novelty Detection Current Status: 1.Successfully designed the approach to meet both requirements 2.Trying to extend the work towards: 1.Information reuse in scholary articles 2.Novelty Detection

Reference 1.Tuomo Kakkonen, Niko Myller, Jari Timonen, and Erkki Sutinen; Automatic Essay Grading with Probabilistic Latent Semantic Analysis; Proceedings of the 2nd Workshop on Building Educational Applications Using NLP, pages 29–36, Ann Arbor, June Jill Burstein, Martin Chodorow, Claudia Leacock: CriterionSM Online Essay Evaluation: An application for Automated Evaluation of Student Essays. IAAI 2003: Yigal Attali & Jill Burstein; Automated Essay Scoring With e-rater V.2; Journal of Technology, Learning, and Assessment (ISSN ). 4.Derrick Higgins, Jill Burstein, Y. Attali: Identifying off-topic student essays without topic-specific training data. Natural Language Engineering 12(2): (2006). 5.Jill Burstein, Martin Chodorow, Claudia Leacock: Automated Essay Evaluation: The Criterion Online Writing Service. AI Magazine 25(3): (2004). 6.Shermis, M. D. & Hamner, B. (2012) Contrasting State-of-the-Art Automated Scoring of Essays: analysis. 7.William, G.,Church, K., and Yarowsky, D., (1993). A method for disambiguating word senses in a large corpus. Computers and the Humanities, 26: Hanneman, Robert A. and Mark Riddle Introduction to social network methods. Riverside, CA: University of California, Riverside ( published in digital form at ). 9.Lawrence M. Rudner; Tahung Liang. Automated Essay Scoring Using Bayes’ Theorem. The Journal of Technology, Learning, and Assessment. Volume 1, Number 2 · June Rothstein J (1952). Organization and entropy, Journal of Applied Physics 23, 1281– Wiebe, J. and E. Riloff: 2005, ‘Creating Subjective and Objective Sentence Classifiers from Unannotated Texts’. In: Proceeding of CICLing-05, pp. 475– Jitesh Shetty,Jafar Adibi;Discovering important nodes through graph entropy the case of Enron database;KDD ’2005 Chicago, Illinois.