Presentation is loading. Please wait.

Presentation is loading. Please wait.

Common Crawl : enabling machine-scale analysis of web data Lisa Green Kurt Bollacker Jordan Mendelson IIPC 2014-05-19.

Similar presentations


Presentation on theme: "Common Crawl : enabling machine-scale analysis of web data Lisa Green Kurt Bollacker Jordan Mendelson IIPC 2014-05-19."— Presentation transcript:

1 Common Crawl : enabling machine-scale analysis of web data Lisa Green Kurt Bollacker Jordan Mendelson IIPC 2014-05-19

2

3 Photo license: Public Domain Origin: http://en.wikipedia.org/wiki/File:Floppy_disk_2009_G1.jpghttp://en.wikipedia.org/wiki/File:Floppy_disk_2009_G1.jpg

4 Photo license: CC BY-SA http://commons.wikimedia.org/wiki/File:Img20050526_0007_at_tannheim_cumulus.jpghttp://commons.wikimedia.org/wiki/File:Img20050526_0007_at_tannheim_cumulus.jpg

5 Photo license: CC-BY-NC https://www.flickr.com/photos/malloreigh/5580160943

6 Photo license: CC-BY-SA Origin: http://en.wikipedia.org/wiki/File:Wikimedia_Foundation_Servers-8055_08.jpg

7 Enable machine scale access and analysis of web data for everyone “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

8 Web Data Commons: “Extracting Structured Data from the Common Crawl” “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

9 WikiEntities (Han Xiaogang) In What Context Is a Term Referenced? “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

10 WikiEntities Example: Discography Who are the most popular artists? “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

11 How Easily Can Google Analytics Track Our Browsing? (S. Merity, C. Hornbaker) “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

12 Data Publica: Finding French Open Data “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

13 Commercial Applications: Improved Spell Checking “Those who do not understand Unix are condemned to reinvent it, poorly.” - Henry Spencer

14 may be too domain specific Photo license: CC-BY-NC-ND https://www.flickr.com/photos/blueforce4116/1398245798

15 Photo license: CC BY-SA http://commons.wikimedia.org/wiki/File:Img20050526_0007_at_tannheim_cumulus.jpghttp://commons.wikimedia.org/wiki/File:Img20050526_0007_at_tannheim_cumulus.jpg

16 Photo license: CC-BY Origin: http://en.wikipedia.org/wiki/File:Internet_map_1024.http://en.wikipedia.org/wiki/File:Internet_map_1024

17

18

19

20

21

22 Image license: CC BY-SA https://www.flickr.com/photos/xdxd_vs_xdxd/6829447421https://www.flickr.com/photos/xdxd_vs_xdxd/6829447421

23 Photo license: CC-BY-SA https://www.flickr.com/photos/hackny/6202775045https://www.flickr.com/photos/hackny/6202775045

24

25

26 Thank You www.commoncrawl.org lisa@commoncrawl.org kurt@commoncrawl.org jordan@commoncrawl.org


Download ppt "Common Crawl : enabling machine-scale analysis of web data Lisa Green Kurt Bollacker Jordan Mendelson IIPC 2014-05-19."

Similar presentations


Ads by Google