1. TWITTER 3 DAY 33 - 11/12/14 LING 3820 & 6820 Natural Language Processing Harry Howard Tulane University

2. Course organization 10-Nov-2014NLP, Prof. Howard, Tulane University 2  http://www.tulane.edu/~howard/LING3820/ http://www.tulane.edu/~howard/LING3820/  The syllabus is under construction.  http://www.tulane.edu/~howard/CompCultEN/ http://www.tulane.edu/~howard/CompCultEN/  Chapter numbering  3.7. How to deal with non-English characters 3.7. How to deal with non-English characters  4.5. How to create a pattern with Unicode characters 4.5. How to create a pattern with Unicode characters  6. Control 6. Control

3. Open Spyder 10-Nov-2014 3 NLP, Prof. Howard, Tulane University

4. Twitter Review 10-Nov-2014 4 NLP, Prof. Howard, Tulane University

5. logon() 1. def logon(): 2. import tweepy 3. API_KEY = 'your_info_here' 4. API_SECRET = 'your_info_here' 5. ACCESS_TOKEN = 'your_info_here' 6. ACCESS_TOKEN_SECRET = 'your_info_here' 7. key = tweepy.OAuthHandler(API_KEY, API_SECRET) 8. key.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET) 9. return key 10-Nov-2014NLP, Prof. Howard, Tulane University 5

6. The other functions of tweepies.py 1. stream2screen(num, terms) 2. stream2var(num, terms) 3. stream2file(num, terms) 4. json2screen(num, terms) 5. json2screenpretty(num, terms) 6. dict2screen(num, terms) 7. dict2var(num, terms) 10-Nov-2014NLP, Prof. Howard, Tulane University 6

7. Quiz  Task: can you find a group of words that will distinguish two Twitter topics?  How to do it  Collect 500+ tweets from two trending topics into different variables.  Run each through a FreqDist to find frequent words that may be unique to each topic (filter out the stop words).  Use these key words in a ConditionalFreqDist to show how well they would work in identifying or classifying each topic. 10-Nov-2014NLP, Prof. Howard, Tulane University 7

8. tweepy's REST API 10-Nov-2014 8 NLP, Prof. Howard, Tulane University

9. How to access Twitter's APIs 07-Nov-2014NLP, Prof. Howard, Tulane University 9 streamingREST representational state transfer tweepy

10. New version of tweepies  New functions  timeline(num, userName)  trends()  localTrends(WOEID)  srch(num, query) 10-Nov-2014NLP, Prof. Howard, Tulane University 10

11. Usage 1. >>> from tweepies import timeline 2. >>> timeline(1,'JustinBieber') 3. >>> from tweepies import trends 4. >>> world = trends() 5. >>> for t in world: print t['name'], t['countryCode'], t['woeid'] 10-Nov-2014NLP, Prof. Howard, Tulane University 11

12. Usage, cont. 1. http://woeid.rosselliot.co.nz http://woeid.rosselliot.co.nz 2. >>> from tweepies import localTrends 3. >>> import pprint 4. >>> nola = localTrends('2458833') 5. >>> pprint.pprint(nola) 6. [{u'as_of': u'2014-11-12T19:36:01Z', 7. u'created_at': u'2014-11-12T19:29:32Z', 8. u'locations': [{u'name': u'New Orleans', u'woeid': 2458833}], 9. u'trends': [{u'name': u'Veterans Day', 10. u'promoted_content': None, 11. u'query': u'%22Veterans+Day%22', 12. u'url': u'http://twitter.com/search?q=%22Veterans+Day%22'}, 10-Nov-2014NLP, Prof. Howard, Tulane University 12

13. Usage, cont 1. >>> from tweepies import srch 2. >>> VD = srch(20, 'Verterans Day') 3. >>> pprint.pprint(VD) 10-Nov-2014NLP, Prof. Howard, Tulane University 13

14. something else Next time 10-Nov-2014NLP, Prof. Howard, Tulane University 14