1. USING AARRR METRICS TO GUIDE PRODUCT DECISIONS @KNOWLEDGEVISION Matt Kaplan & Alex Kieft Wireless Access: Connect to Cambridge “st0126”

2. Lean Startup Circle Boston, Jan 2012 Wireless Access: Connect to Cambridge “st0126” Feb 17: Lean Startup Machine 10% off with code: “LeanCircle” Register Now: http://theleanstartupmachine.com/events/boston- february-17/

3. What is AARRR? AcquisitionUsers come to the site from various channels ActivationUsers enjoy 1st happy visit RetentionUsers come back, visit multiple times ReferralUsers like product enough to refer others RevenueUsers conduct some monetization behavior Dave McClure, creator of “Pirate Metrics” @davemcclure http://500hats.typepad.com

4. Why is AARRR important? Creates a Common Language

5. Why is AARRR important? Creates a Common Language Provides Meaningful Feedback

6. Why is AARRR important? Creates a Common Language Provides Meaningful Feedback Guides Decision Making

7. Why is AARRR important? Creates a Common Language Provides Meaningful Feedback Guides Decision Making Resolves Conflict

8. About KnowledgeVision

9. New Product Initiative  Objectives:  Easy enough for any user  Using only webcam and PowerPoint  Web only, no software to download  MVP  Register/Login  Upload PowerPoint  Record presentation  Save presentation  Email link

10. Knovio – Born 6/30/11, 11:55pm

11. Knovio – Upload Slides

12. Knovio – Record Video

13. Knovio Presentation

14. Launch Strategy  Private beta program  Friends & family seeding  Leverage educators and students for testing  Outreach to influential bloggers in Education Get email with activation code Sign-up Activate Account Check out

15. New User Growth in First Month

16. Knovio Traffic

17. Global Interest in Knovio

18. From zero to 29,400 results in 30 days

19. “…one of the best Web 2.0 applications of the year.”

20. Acquisition Metrics StageMetric Snapshot on July 7 AcquisitionVisits Knovio homepage 100% ActivationSigns up for Knovio private beta 40% ActivationActivates Knovio account (of signups) 58% ActivationCreates at least 1 Knovio (of signups) 11%

21. Acquisition Metrics StageMetric Snapshot on July 7 Snapshot on July 31 AcquisitionVisits Knovio homepage 100% ActivationSigns up for Knovio private beta 40% 20% ActivationActivates Knovio account (of signups) 58% 59% ActivationCreates at least 1 Knovio (of signups) 11%12% Initial sign-ups were biased by friends & family

22. Other Acquisition Strategies Initial Friends & Family Blogger Outreach Pay per click Organic

23. Acquisition Metrics StageMetricJulAug AcquisitionVisits Knovio homepage 100% ActivationSigns up for Knovio private beta 20% 6% ActivationActivates Knovio account (of signups) 59% 63% ActivationCreates at least 1 Knovio (of signups) 12%19% New acquisition strategies were not producing positive results Some positive signs of improvement

24. A/B Testing – Swap buttons 14% Check out

25. A/B Testing – One button 26%

26. A/B Testing – New Concept 27%

27. Stimulate User Activity  Fun, camp-themed email newsletter  Weekly, personalized activities  Specific recommendations and targets  Testing limits and game mechanics Check out 35% Open Rate

28. Acquisition Metrics MetricJulAugSepOct Visits Knovio homepage 100% Signs up for Knovio 20% 6%7%18% Activates Knovio account 59% 63%68%73% Creates at least 1 Knovio 12%19% 26%36% Improved activation

29. Product Improvements  Most requested features Delete presentation Embed code for sharing Audio-only presentations Shorter URLs  Re-recording slides  Editing presentations Check out

30. Involved users in feature design

31. Public Beta – Nov 15, 2011  Redesigned homepage  Removed extra step in registration process  Targeted business user  Improved messaging  Aligned with core product  200+ Bloggers

32. Acquisition Metrics MetricJulAugSepOctNovDec Visits Knovio homepage 100% Signs up for Knovio 20% 6%7%18%16%18% Activates Knovio account 59% 63%68%73%100% Creates at least 1 Knovio 12%19% 27%36%37%44% Much better activation metrics across the board

33. Knovio User Growth Avoid “feel-good” vanity Metrics

34. Initial Retention Metrics MetricJulAugSep Creates 2 or more presentations5%7%10% Returns to create presentation (following months)5%6%8% Logs in 3 times or more??? Hypothesis: If users could get analytics and feedback on presentations, they would use it more

35. New Feedback Features  Email alerts upon viewing  Built in feedback “A user from Concord, MA just watched your Knovio presentation” Mary M. said “This is a great presentation, please make more!”

36. Retention Metrics MetricJulAugSepOctNovDec Creates 2 or more presentations5%7%10%17%12%15% Returns to create presentation (following months) 5%6%8%7%14%TBD Logs in 3 times or more35%55% Rolled out email alerts upon viewing Began tracking login history

37. Cohort Report for Retention Analysis

38. Cohort Report – A better view

39. Knovio User Types 44% 34% 22% Note: Beware of other factors effecting metrics

40. Hypothesis: Viral features will fuel growth User creates 1 Knovio presentation Emails presentation Posts presentation to FB, Twitter, LinkedIn Invites friend to use Knovio 20 viewers x 2% =.4 50 viewers x 1% =.5 5 invites x 20% = 1 Viral Coefficient = 1.9 David Skok Matrix Partners ForEntrepreneurs.com

41. Referrals - Sharing built into app  Link  Embed  Email  Social Media

42. Email deliverability Check out 57% open rate 94% deliverability

43. Metrics for Sharing Options Get Embed Get Link Email Check out

44. Self Feedback Private Sharing & Feedback Public Sharing & Feedback Referrals - Sharing Hypothesis Requires Registration

45. New Feature: “Spaces”

46. Knovio “Spaces”

47. Spaces – Invite Users

48. Metrics for Sharing Options Get Embed Get Link Email Check out Spaces

49. Idea: Entice users to try Spaces

50. Referral Metrics MetricJulAugSepOctNovDec Share via Email7%11%14%23%16%21% Invite via Spaces1.8%2.9%2.4% Better, but not exactly what we were hoping for Good improvement but unpredictable

51. Revenue Hypothesis KVStudio Power Tools for rich content producers Lead generation

52. Revenue Metrics MetricJulAugSepOctNovDec Leads for KVStudio.2%.5%0%.6%1.4% Ugh

53. AARRR Lessons Learned  Create your AARRR dashboard before MVP  Build metrics capture into product  Measure only what you think you’ll need  Don’t change too many variables at once  Know who your target is and only listen to them  Get entire company on board (not easy!)

54. Too many metrics

55. Tools for capturing metrics/feedback  Prefinery – Private beta management  Mixpanel – Tracking user events  Unbounce – A/B testing  Wufoo – User feedback  MailChimp – Email engagement  SendGrid – Transactional email  Amazon SimpleDB – Error logs

56. Thank you! Matt Kaplan @mpkaplan matt@knowledgevision.com Alex Kieft alex@knowledgevision.com

57. Lean Startup Circle Boston, Jan 2012 Wireless Access: Connect to Cambridge “st0126” Feb 17: Lean Startup Machine 10% off with code: “LeanCircle” Register Now: http://theleanstartupmachine.com/events/boston- february-17/