Deep Questions without Deep Understanding Paper Presentation

Abstract Develop an approach to generate deep comprehension question from novel text without creating a full semantic representation of text.

Deep questions Open ended problems that require deep thinking and recall It requires significant amounts of content rather than a single sentence. Why deep questions? Helps in understanding the text at its fullest and provides greatest educational value Key assessment mechanism for online educational options including MOOCs

Approach This paper introduced an ontology-crowd-relevance workflow to generate high level questions. This involves: Decomposing the original text into low dimensional ontology Obtaining high-level question templates from the crowd Retrieving subset of collected templates for a target text segment based on its ontological categories and ranking these questions

Example question template “Who were the key influences on <Person> in their childhood?” is a question template for category: Person and section: Early life

Category-section ontology Freebase “notable type” for each Wikipedia article is used to find the high-level categories. Took 300 most common categories across Wikipedia and then merged these categories into eight broad categories to reduce crowdsourcing effort: Person, Location, Event, Organization, Art, Science, Health, and Religion. These 8 categories cover 78% of Wikipedia articles. Category-section pairs for an article about Albert Einstein contains (Person, Early life), (Person, Awards), and (Person, Political views)

Crowdsourcing Methodology Designed a two-stage crowdsourcing pipeline Question generation task: to create question templates that are targeted to set of category-section pairs Question relevance rating task: to obtain binary relevance judgments for the generated question templates in relation to a set of article segments that match in category-section labels. Rating for each question is done on 3 dimensions: relevance, quality and scope.

Model Category/section inference Trained individual logistic regression classifiers for the eight categories and the 50 top section types using the default L2 regularization parameter in LIBLINEAR. Obtained an accuracy of 83% for category and 95% for section.

Model Relevance Classification Used a vector of the component-wise Euclidean distances between individual features of the question and article segment fi=(qi−ai)^2 where qi and ai are the components of the question and article feature vectors. The paper then augmented the vector by concatenating additional distance features between the target article segment and one specific instance of an entire article for which the question applied. Resulting feature vector include first k distances between question tem-plate and the target segment, and the next 𝑘 were between the augmenting article and the target segment. Trained the relevance classifier, a single logistic regression model using LIBLINEAR with default L2 regularization

Results

The slides are part of paper review for course CS671 This presentation, in no way, claims ownership of any contents in the slides