1. Designing Ranking Systems for Consumer Reviews: The Economic Impact of Customer Sentiment in Electronic Markets Anindya Ghose Panagiotis Ipeirotis Stern School, New York University ICEC 2007

2. Democratization of content creation O pinions in user-generated content –The Web has changed the way that people express their views and opinions –One can express opinions on almost anything at: review sites, forums, reputation profiles, blogs Opinions Matter Businesses: marketing intelligence, product design and service improvement. Firms invest a lot to find consumer opinions Consultants, Surveys and focus groups, etc Individuals: interested in other’s opinions on products, services, topics, events Online word-of-mouth

3. Selected Prior Research Analyze and classify sentiments in online opinions –Pang and Lee (2004), Hu and Liu (2004), Kim and Hovy (2004), Liu, Hu and Cheng (2005) Volume and valence in online reviews influences product sales –Dellarocas et al. (2005), Chevalier and Mayzlin (2006), Liu (2006), Clemons and Gao (2006) Reviewer’s social information influences product sales and review helpfulness over and above volume and valence –Forman, Ghose and Wiesenfeld (2006) What about the impact of textual sentiments on Product sales Review Helpfulness

4. Focus of this work Our Questions How sentiment in the text affects product demand?How sentiment in the text affects product demand? How sentiment in the text affects informativeness of reviews?How sentiment in the text affects informativeness of reviews? Searching for reviews is different from general Web search. –Reading only the review ranked at the top can be erroneous because it is only opinion of one person. Improve the design of electronic markets Propose two ranking mechanisms for ranking product reviews: –a consumer-oriented ranking mechanism ranks the reviews according to their expected helpfulness. –a manufacturer-oriented ranking mechanism ranks the reviews according to their expected effect on sales.

5. Data

6. Data Elements of a review and reviewer profile  Rating (number)->Moderate or Extreme Review  Number of helpful votes and total votes  Review posting date  Textual content-> ‘Subjective’ or ‘Objective’ Review and ‘Readability’  Reviewer information disclosure -> Real Name, Location, etc. Overview Mix of products: Audio-Video, Digital Cameras. Sample: March 2005 - April 2006 Summary of Amazon Data New: List Price, Amazon price, Sales rank, Release Date. Valence, Volume of reviews

7. Measuring Subjectivity Pang and Lee (2004)-technique that identifies which sentences in a text convey objective or subjective elements. Sentences in product description: “Objective” Sentences in reviews: “Subjective” A training set with two classes of documents: –A set of objective documents that contains the product descriptions of each of the 1,000 products in our data set. –A set of subjective documents that contains randomly retrieved reviews. Train a classifier (n-gram based) using Dynamic Language Model classifier from LingPipe toolkit to distinguish between subjective and objective sentences in each review. –‘Average Probability of Subjectivity’ for a review –‘Standard deviation of Subjectivity’ for each review In our context: “Subjectivity” measures deviation of review from manufacturer-provided product information

8. Measuring Readability “Readability” measures amount of information in the review, and difficulty of reading the provided information Current work: 1.Measured number of sentences 2.Measured length of review in words and characters 3.Ratio of length in characters (or words) to number of sentences. Ongoing work: Use ‘Readability’ statistics of the reviews –ColemanLiau –Flesh Kinkaid Level –Gunning –SMOG Index

9. Impact of Sentiments and Sales Product level fixed effects We also ran first differences model using variations in the unit of time analysis (weekly, biweekly and monthly) Qualitative nature of results are robust to these specifications Derived the stylistic characteristics of each review. Proceed to examine the economic impact of the subjectivity (or objectivity) of the review.

10. Estimates for effect on sales rank Independent Variable Audio-VideoDigital Camera Average Subjectivity -1.47 *** (.72) 1.27 (1.24) Deviation of Subjectivity -0.69 (1.06) -2.91 *** (1.7) Amazon Price1.59 *** (0.3) 6.2 *** (0.61) Elapsed Date0.12 (0.07) 0.28 ** (0.13) Average Rating-0.016 (0.02) -0.01 (0.03) Log (Num. Reviews) 0.6 *** (0.15) 1.08 *** (0.2) Readability0.06 (0.057) -0.15 * (0.08) R2R2 0.180.37

11. Effect of Sentiments on Informativeness

12. Validation with Content Analysis Validation with Content Analysis Asked two coders to classify 1,000 reviews (test set): –Is the review informative or not? –As influencing their purchasing decisions or not? Good inter-rater agreement (Kappa statistic of 0.73) For second question, using polychoric correlation agreement was strong (p < 0.05) Estimated regression coefficients using rest of reviews (training set) Predicted helpfulness and influence of the test reviews, using only text

13. F-measure F-measure combines precision and recall in a single metric

14. Findings and Conclusion Show that textual sentiment influences sales over and above the numeric and self-descriptive information that consumer reviews and reviewers display. –Increasingly subjective reviews can increase product sales. –A mixture of objective and subjective opinions with extreme subjective content can increase sales. –A mixture of objective and subjective opinions with extreme subjective content can increases informativeness of reviews –Increased `Readability’ can increases product sales and informativeness of reviews Our method can quickly identify reviews that are expected to be helpful to the users, and display them first. –Improving the usefulness of the reviewing mechanism to the consumer in an electronic market. –Manufacturers can understand which reviews are most impactful and use those product features for marketing promotions

15. Ongoing Work Combine the sentiments (positive, mixed or negative) with subjectivity analysis. –Negative reviews may increase sales if the reviews are informative Incorporate the impact of all the past reviews of a reviewer. Examine more text-based variables, in detail (e.g. readability metrics).