Quality-Aware Collaborative Question Answering: Methods and Evaluation Maggy Anastasia Suryanto, Ee-Peng Lim, Aixin Sun, and Roger H. L. Chiang. In Proceedings of the Second ACM International Conference on Web Search and Data Mining (Barcelona, Spain, February 9-12, 2009). Prepared and Presented by Baichuan Li August 17, 2015

Outline Introduction Quality-Aware Framework Expertise Based Methods Experiments Conclusion 17/8/2015 Paper Presentation 2/21

Introduction Community-Based Question-Answering (CQA) Services 17/8/2015 Paper Presentation 3/21

Diverse Answer Qualities 17/8/2015 Paper Presentation 4/21 good poor fair

Objective Automatically find good answers for a user given questions from a community QA portal ◦ answer features ◦ user expertise of answers 17/8/2015 5/21 Paper Presentation

Quality-Aware Framework 17/8/2015Paper Presentation 6/21

Expertise Based Methods 17/8/2015 Paper Presentation 7/21 Relevance score Quality score

Expertise Based Methods 17/8/2015 Paper Presentation 8/21

Question Independent Expertise 17/8/2015 Paper Presentation 9/21 EXHITS uses qscore_exhits(a) as the quality score of an answer a given in below equation: authorityhub

Question Dependent Expertise 17/8/2015Paper Presentation 10/21

Question Dependent Expertise 17/8/2015 Paper Presentation 11/21 EX_QD EX_QD’

Answer Relevance Models Answer ranking by Yahoo! Answers Query likelihood retrieval model 17/8/ /21 Paper Presentation all answers and questions in the dataset

Experiments 17/8/2015 Paper Presentation 13/21 Methods Compared ◦ BasicYA  BasicYA(subject + content)  BasicYA(subject + content + best answers) ◦ BasicQL  Adopts query likelihood retrieval model to score the relevance of an answer ◦ NT (classification based on non-textual answer features)  maximum entropy approach  9 features

Dataset 17/8/2015Paper Presentation 14/21

Evaluation 17/8/2015Paper Presentation 15/21 The top 20 of the ranked answers of each methods were manually judged in terms of their relevance and quality. The following evaluation metrics are used to evaluate the accuracy of the methods:

Results 17/8/2015 Paper Presentation 16/21

Results 17/8/2015 Paper Presentation 17/21

Results 17/8/2015 Paper Presentation 18/21

Conclusion Introduce a quality-aware QA framework that considers both answer relevance and quality in selecting answers to be returned. Develop several QA methods (namely, EXHITS, EXHITS QD, EX QD and EX QD') that consider answerer expertise to determine answer quality. Conducted extensive experiments and these experiments showed that quality-aware methods can improve both quality and overall performance. Among them, the methods EX QD and EX QD' using question dependent answerer expertise have the best performance. 17/8/2015 Paper Presentation 19/21

Ideas 17/8/2015 Paper Presentation 20/21

Q&A 17/8/2015 Paper Presentation 21/21