1. Search Engine Optimization L. Grewe 1

2. Why Search Engines? An estimated 77 percent+ of Internet users employ search engines to find Web sites An estimated 77 percent+ of Internet users employ search engines to find Web sites If you are not listed on Google or Yahoo!, you are invisible If you are not listed on Google or Yahoo!, you are invisible It doesn’t matter how well known your brand is!It doesn’t matter how well known your brand is! Visitors will not guess your site name but will go through Google or Yahoo! just to be safeVisitors will not guess your site name but will go through Google or Yahoo! just to be safe 2

3. 3 1. How search engines handle your content 2. How to improve ranking of web pages on search results Objectives

4. 4 Search Engine Producers Google Google Yahoo! Yahoo! Bing Bing Ask.com Ask.com Major Search Engines and Directories

5. 5 Modified from Bruce Clay’s Search Engine Chart: http://www.bruceclay.com/searchenginechart.pdf http://www.bruceclay.com/searchenginechart.pdf

6. Search Engine Optimization Search Engine Optimization or SEO is the task of designing (or redesigning) you site (pages) to appear higher in the list of results from Search Engines. Search Engine Optimization or SEO is the task of designing (or redesigning) you site (pages) to appear higher in the list of results from Search Engines. 6

7. 7 Basics of Search Friendly Design

8. 8 Keywords: Use text that targets your audience; and attracts search engines and links from other sites Keywords: Use text that targets your audience; and attracts search engines and links from other sites Navigation: Give users and crawlers easy access to content Navigation: Give users and crawlers easy access to content Design Elements: Make sure bells and whistles don’t undermine SEO efforts Design Elements: Make sure bells and whistles don’t undermine SEO efforts Page Rank: Develop a linking strategy to improve your site’s “popularity” Page Rank: Develop a linking strategy to improve your site’s “popularity” Basic SEO Concepts

9. 9 Things Search Engines Like Text (Keyword) Text (Keyword) Words and phrases that match what your target audience types into search enginesWords and phrases that match what your target audience types into search engines Site Navigation Site Navigation Site navigation and URL structure that search crawlers can easily followSite navigation and URL structure that search crawlers can easily follow Links to Your Site Links to Your Site How “popular” are you?How “popular” are you?

10. Basic Idea of how Web Discovered How does a Search Engine discover the web…….. How does a Search Engine discover the web…….. Much is proprietary Much is proprietary Actual algorithms can be complex but, need to be fast! Actual algorithms can be complex but, need to be fast! Here is a peek into the concept…. Here is a peek into the concept…. 10

11. Web crawlers (and robots) Software that travel through a website gathering information. Software that travel through a website gathering information. Search engines use these automated SW robots to index a website. Search engines use these automated SW robots to index a website. Web crawlers are mainly used to copy all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code. Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam). Web crawlers are mainly used to copy all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code. Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam).indexHTMLspamindexHTMLspam 11

12. Other names for Web Crawlers Web crawlers are ants, automatic indexers, bots, [1] or Web spiders, [2] Web robots, [2] or— especially in the FOAF community—Web scutters [3]. Web crawlers are ants, automatic indexers, bots, [1] or Web spiders, [2] Web robots, [2] or— especially in the FOAF community—Web scutters [3]. [1] [2] FOAF [3] [1] [2] FOAF [3] How it works: How it works: it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies.it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies.URLs hyperlinksURLs hyperlinks 12

13. What Information Do Search Engines Gather No one know exactly –except each Search Engine Company No one know exactly –except each Search Engine Company People fear too much. People fear too much. Some known things: keywords, links to and from, content /word counts, images, contact info (email, etc.), media and ???? Some known things: keywords, links to and from, content /word counts, images, contact info (email, etc.), media and ???? 13

14. 14 Managing Page Content

15. 15  Search engines place “weight” on terms according to where they are used Keyword Prominence Primary Text Title tag Title tag Body text Body text Anchor text Anchor text Text near the top of the page Text near the top of the page Headings and Emphasized Text Headings and Emphasized Text Secondary Text  Alt text  Description tag  Domain name and URL elements

16. 16 What About Metadata? Metadata is not a factor in most web search engines Metadata is not a factor in most web search engines Except for Title and META description tags, crawlers ignore other META tags Except for Title and META description tags, crawlers ignore other META tags Use metadata if important for your agency’s mission or enterprise systems (e.g., CMS, site search engine) Use metadata if important for your agency’s mission or enterprise systems (e.g., CMS, site search engine)

17. 17 Keywords in Text Choose one or two keywords or phases to optimize for each page Choose one or two keywords or phases to optimize for each page Use language of your audience Use language of your audience Do not over use - avoid keyword stuffing Do not over use - avoid keyword stuffing Keywords in links influence relevancy of the pages your linking to Keywords in links influence relevancy of the pages your linking to No “click here” linksNo “click here” linksclick hereclick here

18. 18 Site’s Search Metrics Site’s Search Metrics Online Advertising Tools Online Advertising Tools Yahoo!/Overture: http://advertising.yahoo.com/Yahoo!/Overture: http://advertising.yahoo.com/http://advertising.yahoo.com/ (Yahoo! White paper, Case Study, Advertising Case Study ) (Yahoo! White paper, Case Study, Advertising Case Study )White paperCase StudyAdvertising Case Study White paperCase StudyAdvertising Case Study Google:Google: https://adwords.google.com/o/Targeting/Explorer?__u=100000000 0&__c=1000000000&ideaRequestType=KEYWORD_IDEAS#search.n one https://adwords.google.com/o/Targeting/Explorer?__u=100000000 0&__c=1000000000&ideaRequestType=KEYWORD_IDEAS#search.n one https://adwords.google.com/o/Targeting/Explorer?__u=100000000 0&__c=1000000000&ideaRequestType=KEYWORD_IDEAS#search.n one https://adwords.google.com/o/Targeting/Explorer?__u=100000000 0&__c=1000000000&ideaRequestType=KEYWORD_IDEAS#search.n one Keyword Density Keyword Density http://www.webjectives.com/keyword.htmhttp://www.webjectives.com/keyword.htmhttp://www.webjectives.com/keyword.htm http://www.keyworddensity.com/http://www.keyworddensity.com/http://www.keyworddensity.com/ Google Webmaster Tools Google Webmaster Tools Keyword Selection Tools

19. Other Keyword Selection tools http://www.wordtracker.com/ http://www.wordtracker.com/ http://www.wordtracker.com/ Not directly related to web and search engines Not directly related to web and search engines http://thesaurus.reference.com/thesaur us/http://thesaurus.reference.com/thesaur us/http://thesaurus.reference.com/thesaur us/http://thesaurus.reference.com/thesaur us/ 19

20. Keyword Selection Must relate to your business / site Must relate to your business / site Use Tools to help Use Tools to help DO you choose popular keywords or more specific less popular (but, very targeted keywords)? DO you choose popular keywords or more specific less popular (but, very targeted keywords)? Gardening or Butterfly Gardening ??? Gardening or Butterfly Gardening ??? Dig Deep and then select Dig Deep and then select 20

21. Chose your Keyword now what? Put keywords in your site prominently (think primary text) Put keywords in your site prominently (think primary text) Link to other pages that feature this keyword / related information. Link to other pages that feature this keyword / related information. Have annotated media related to this keyword(s). Have annotated media related to this keyword(s). 21

22. 22 Site Navigation

23. 23 Plan how pages link to each other Plan how pages link to each other Pages will not rank well if your site does not have a navigation schemePages will not rank well if your site does not have a navigation scheme Navigation scheme must please users and search engines Navigation scheme must please users and search engines Create a crawler-friendly site map Create a crawler-friendly site map Avoid dangling pages Avoid dangling pages Linking Basics

24. 24 Keywords in links tell crawlers about the pages to which you are linking Keywords in links tell crawlers about the pages to which you are linking Keywords in links influence relevancy of the page to which you are linking Keywords in links influence relevancy of the page to which you are linking Avoid “click here” links, instead create links like: Avoid “click here” links, instead create links like: Bad: Click here for more info on famous admirals. Bad: Click here for more info on famous admirals. Good: Visit our naval history site for more info on famous admirals. Good: Visit our naval history site for more info on famous admirals. Creating Links

25. 25 Make a site with a clear hierarchy Make a site with a clear hierarchy Text Links Text Links The most search engine friendlyThe most search engine friendly Use for primary or secondary navigationUse for primary or secondary navigation Every page should be reachable from a text linkEvery page should be reachable from a text link If using graphic navigation, use alt text If using graphic navigation, use alt text Avoid JavaScript, unless you provide a secondary text navigation scheme Avoid JavaScript, unless you provide a secondary text navigation scheme Site Navigation Schemes

26. 26 Poor HTML coding Poor HTML coding Image maps Image maps Frames Frames JavaScript JavaScript Dynamic Pages Dynamic Pages Flash Flash Problem Navigation Schemes

27. 27 Create a Site Map Create a Site Map HTML page on your site with text linksHTML page on your site with text links Create a Sitemap Create a Sitemap An XML file that lists your site’s URLsAn XML file that lists your site’s URLs Allows crawlers to analyzes information about your site’s architecture to improve crawlingAllows crawlers to analyzes information about your site’s architecture to improve crawling Google: http://www.google.com/webmasters/Google: http://www.google.com/webmasters/http://www.google.com/webmasters/ Yahoo: http://siteexplorer.search.yahoo.com/Yahoo: http://siteexplorer.search.yahoo.com/http://siteexplorer.search.yahoo.com/ Site Maps Protocol: http://www.sitemaps.org/Site Maps Protocol: http://www.sitemaps.org/http://www.sitemaps.org/ Help Crawlers Navigate

28. SiteMaps The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling.URLs is (usually) an XML file that lists the URLs for a site. is (usually) an XML file that lists the URLs for a site.XML This allows search engines to crawl the site more intelligently. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol and complement robots.txt, a URL exclusion protocol. Sitemaps are a URL inclusion protocol and complement robots.txt, a URL exclusion protocol. robots.txt Sitemaps are particularly beneficial on websites where: Sitemaps are particularly beneficial on websites where: some areas of the website are not available through the browsable interface, orsome areas of the website are not available through the browsable interface, or webmasters use rich Ajax, Silverlight, or Flash content that is not normally processed by search engines.webmasters use rich Ajax, Silverlight, or Flash content that is not normally processed by search engines.AjaxSilverlightFlashAjaxSilverlightFlash 28

29. Sitemaps Google Sitemaps Google Sitemaps http://www.electionsonline.us/sitemap.xml for example tells Google where to begin crawling even without a sitehttp://www.electionsonline.us/sitemap.xml for example tells Google where to begin crawling even without a sitehttp://www.electionsonline.us/sitemap.xml 29 http://www.electionsonline.us/ monthly http://www.electionsonline.us/sitemap.htm monthly 1.0 http://www.electionsonline.us/products-and-prices/electionServices.htm monthly …..MORE ‘s

30. Sitemap XML protocol Same for popular Search Engines: Google, Yahoo!, Bing Same for popular Search Engines: Google, Yahoo!, Bing 30

31. 31 Robots Exclusion robots.txt File robots.txt File Place in server’s root directoryPlace in server’s root directory Two elements: User-agent, DisallowTwo elements: User-agent, Disallow Example:Example: User-agent: * Disallow: /cgi-bin/ Disallow: /scripts/ Disallow: /images/ Meta-Tag Robots Exclusion Meta-Tag Robots Exclusion is a convention to prevent cooperating web spiders/crawlersweb spiders and other web robots fromweb robots accessing all or part of a website whichwebsite is otherwise publicly viewable. The standard is unrelated to, but can be used in conjunction with, Sitemaps, aSitemaps robot inclusion standard for websites.

32. 32 Managing Your Popularity

33. 33 A calculation assigned to a page based on the number of links into the page A calculation assigned to a page based on the number of links into the page The more popular pages that link to you, the higher your popularity rank The more popular pages that link to you, the higher your popularity rank An inbound link is a vote for a pageAn inbound link is a vote for a page An outbound link is a vote for a page you’re linking toAn outbound link is a vote for a page you’re linking to All search engines have different popularity algorithms All search engines have different popularity algorithms Google’s algorithm is called Page RankGoogle’s algorithm is called Page Rank Based on a scale from 0-10Based on a scale from 0-10 What is Popularity?

34. 34 Number and Popularity of Inbound Links Number and Popularity of Inbound Links Anchor Text Anchor Text Others’ use of keywords in a link to your siteOthers’ use of keywords in a link to your site Popularity is assigned per page, not for the entire site Popularity is assigned per page, not for the entire site Popularity is not inheritedPopularity is not inherited Need to deliberately link internal pages to pass on PRNeed to deliberately link internal pages to pass on PR Popularity Factors

35. 35 Get Listed in Yahoo! and DMOZ (www.dmoz.org – open directory project) Get Listed in Yahoo! and DMOZ (www.dmoz.org – open directory project)www.dmoz.org Impact on popularity may varyImpact on popularity may vary Helps to get noticed by crawlersHelps to get noticed by crawlers Establish reciprocal link arrangements with agencies covering similar topics Establish reciprocal link arrangements with agencies covering similar topics Reach out to state and local agencies Reach out to state and local agencies Syndicate your content Syndicate your content Reach out to professional communities of interest Reach out to professional communities of interest Getting Links to Your Site

36. 36 Links That Aren’t Links Link has been created in a manner that search engines can’t understand Link has been created in a manner that search engines can’t understand Link actually points elsewhere, and not directly to your site Link actually points elsewhere, and not directly to your site Example: Example: http://www.site.gov/ext/external.jsp?url=http://www.noaa.gov/ Link passes through a JSP program that logs the click and re-directs the browser to noaa.govLink passes through a JSP program that logs the click and re-directs the browser to noaa.gov Search engines read this as an internal link to Site.gov, not an outbound link to NOAASearch engines read this as an internal link to Site.gov, not an outbound link to NOAA Links from these sites will not help boost your search rankingLinks from these sites will not help boost your search ranking

37. 37 Design Considerations

38. 38 Implement bells and whistles carefully Implement bells and whistles carefully Use external Javascript and CSS to decrease page load timeUse external Javascript and CSS to decrease page load time Avoid using FramesAvoid using Frames Flash only pages – include skip link and title and META description tagFlash only pages – include skip link and title and META description tag Omit Session IDs when requestor is a search engine crawlerOmit Session IDs when requestor is a search engine crawler Ensure design team understands SEO concepts Ensure design team understands SEO concepts General Considerations

39. 39 Database-driven, created on the fly by asp, cfm, php, jsp or cgi scripts Database-driven, created on the fly by asp, cfm, php, jsp or cgi scripts Dynamic sites are comprised of templates, but usually without original content Dynamic sites are comprised of templates, but usually without original content When a page is viewed, the template loads the content from the database When a page is viewed, the template loads the content from the database Parameters are added to the URL, which tells the template to load specific content Parameters are added to the URL, which tells the template to load specific content Dynamic Pages

40. 40 Example: http://smithsonianstore.com/catalog/product.jsp?productId=1 4273&parentCategoryId=3151&categoryId=3152 http://smithsonianstore.com/catalog/product.jsp?productId=1 4273&parentCategoryId=3151&categoryId=3152 http://smithsonianstore.com/catalog/product.jsp?productId=1 4273&parentCategoryId=3151&categoryId=3152 URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page The more parameters, the less likely pages will be indexed The more parameters, the less likely pages will be indexed A database may continually feed data, crashing your server and scaring off the crawler A database may continually feed data, crashing your server and scaring off the crawler Dynamic Pages

41. 41 Modify URLs so they don’t look like dynamic pages, fewer parameters Modify URLs so they don’t look like dynamic pages, fewer parameters Use URL re-write trick using mod_rewrite (Apache) Use URL re-write trick using mod_rewrite (Apache) Create static HTML pages Create static HTML pages Search Friendly Dynamic Pages

42. 42 Make sure PDFs contain actual text, not images of text Make sure PDFs contain actual text, not images of text Same rules for use of keywords and phrases apply Same rules for use of keywords and phrases apply Put the most important text in the title, headlines Put the most important text in the title, headlines Minimize document size (> 100K) Minimize document size (> 100K) Create optimized HTML pages for PDFs Create optimized HTML pages for PDFs Optimizing PDFs

43. 43 Site Maintenance

44. 44 Plan BEFORE you embark on a re-design Plan BEFORE you embark on a re-design Will the site architecture change?Will the site architecture change? Static to Dynamic?Static to Dynamic? How is the content changing?How is the content changing? Add SEO review as an activity to your project plan Add SEO review as an activity to your project plan Make sure your contractor has SEO skills Make sure your contractor has SEO skills Poor planning and execution can kill your search rankings Poor planning and execution can kill your search rankings Do not wait until after you re-designed! Do not wait until after you re-designed! Re-Designing Your Site

45. 45 Try to keep the same filenames and directory structure when redesigning Try to keep the same filenames and directory structure when redesigning Recommended: Set up HTTP 301 redirects that point to the new site or pages Recommended: Set up HTTP 301 redirects that point to the new site or pages Not Recommended: Add a meta-refresh tag to your page header. This won’t remove your original page from the search index Not Recommended: Add a meta-refresh tag to your page header. This won’t remove your original page from the search index Changes to Site Architecture

46. 46 Avoid using default 404 pages Avoid using default 404 pages Custom 404 pages are more user- friendly Custom 404 pages are more user- friendly Submitting all 404 URLs to the search engine helpful, but not essential Submitting all 404 URLs to the search engine helpful, but not essential Remember to change the HTTP status code on custom error pages from 200 to 404 Remember to change the HTTP status code on custom error pages from 200 to 404 Removing Content

47. Updating Content Usually when you update a page on your web site it helps your site get a higher ranking Usually when you update a page on your web site it helps your site get a higher ranking Not only does frequent updating get you a higher ranking it will also assist search agents with finding your page as new or newly updated. Not only does frequent updating get you a higher ranking it will also assist search agents with finding your page as new or newly updated. So be sure to update your site as frequently as possible (but, only appropriately). So be sure to update your site as frequently as possible (but, only appropriately). 47

48. 48 Case Studies

49. 49

50. 50 Case Study: Business.gov Term: Advertising Regulations #6 on Google #6 on Google #8 on Yahoo #8 on Yahoo #2 on MSN #2 on MSN #1 on USA.gov #1 on USA.gov  Page created March ‘07  Submitted to crawlers, listed in DMOZ (www.dmoz.org – open directory project)www.dmoz.org  0-78 inbound links

51. 51 Case Study: Business.gov Text Component: page is well- optimized for “advertising” and “regulations” Text Component: page is well- optimized for “advertising” and “regulations” Link Component: All text links Link Component: All text links Popularity Component: Inbound links are using title submitted to DMOZ – “advertising” and “regulations” Popularity Component: Inbound links are using title submitted to DMOZ – “advertising” and “regulations”

52. 52 Case Study: NOAA Fisheries Term: Fisheries #1 on USA.gov #1 on USA.gov #2 on MSN #2 on MSN #2 on Yahoo #2 on Yahoo #3 on Google #3 on Google Term: Fish Doesn’t show up on the first page or subsequent pages Doesn’t show up on the first page or subsequent pages

53. 53

54. 54 Case Study: NOAA Fisheries Text Component: page is well- optimized for “fisheries” but not “fish” Text Component: page is well- optimized for “fisheries” but not “fish” Link Component: “pull-down” menu may be a problem for some crawlers. Add search friendly navigation (e.g., site map) Link Component: “pull-down” menu may be a problem for some crawlers. Add search friendly navigation (e.g., site map) Popularity Component: Lots of inbound links using “fisheries” (not “fish”) in anchor text Popularity Component: Lots of inbound links using “fisheries” (not “fish”) in anchor text

55. 55 Next Steps 1. Visit WebContent.gov Guide to Search Engines: http://www.usa.gov/webcontent/technology/search/search.shtml 2. Conduct an SEO best practices review 3. Conduct keyword analysis and determine your rank 4. Set success measures 5. Use Webmaster Tools offered by the search engines, (e.g. Google Webmaster Tools, etc.) Make SEO a routine in your web content management program

56. 56 Resources and Recommendations

57. 57 Battelle, John. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed our Culture. New York. Penguin, 2005. Battelle, John. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed our Culture. New York. Penguin, 2005. Kent, Peter. Search Engine Optimization for Dummies. Hoboken, NJ. Wiley, 2004. Kent, Peter. Search Engine Optimization for Dummies. Hoboken, NJ. Wiley, 2004. Moran, Mike, Hunt, Bill. Search Engine Marketing, Inc. Upper Saddle River, NJ. IBM Press, 2006. Moran, Mike, Hunt, Bill. Search Engine Marketing, Inc. Upper Saddle River, NJ. IBM Press, 2006. Thurow, Shari. Search Engine Visibility. Indianapolis, IN: New Riders, 2003. Thurow, Shari. Search Engine Visibility. Indianapolis, IN: New Riders, 2003. FIND YOU OWN MORE recent publications – search in library – ACM/ IEEE portals. FIND YOU OWN MORE recent publications – search in library – ACM/ IEEE portals. Recommended Reading

58. 58 SEO Forums & Events  Industry Forums Webmaster World: http://www.webmasterworld.com/http://www.webmasterworld.com/ Search Engine Watch: http://www.searchenginewatch.com/ http://www.searchenginewatch.com/ Digital Point Forums: http://forums.digitalpoint.com/http://forums.digitalpoint.com/ High Rankings: http://www.highrankings.com/forum/http://www.highrankings.com/forum/  Industry Events Search Engine Strategies: http://www.searchenginestrategies.com/ http://www.searchenginestrategies.com/

59. Now… Using tools monitor your site Using tools monitor your site Study your traffic reports to determine Study your traffic reports to determine those keywords which drive the most traffic to your site. Then edit your site to ensure they are near the top of the target page and the target page is not overwhelmed by other, less relevant, content. 59

60. Some Additional SOE tools/info http://www.seotoolset.com/tools/free_tools.html http://www.seotoolset.com/tools/free_tools.html http://www.seotoolset.com/tools/free_tools.html http://www.seomoz.org/tools http://www.seomoz.org/tools http://www.seomoz.org/tools http://seobook.com http://seobook.com http://seobook.com 60

61. The Next Step Pay for it - Search Engine Marketing (SEM) Pay for it - Search Engine Marketing (SEM) Can be a “sponsored link” Can be a “sponsored link” Data Asset Optimization (DAO) – Data Asset Optimization (DAO) – manage your media /data (images, video for Search Engines manage your media /data (images, video for Search Engines Youtube links or ?? Youtube links or ?? Social Media Social Media Twitter, Facebook, MySpace and more Twitter, Facebook, MySpace and more Mobile Mobile 61

62. The Next Step Continued Press Releases (paid for on external sites) Press Releases (paid for on external sites) PR web sites that specialize in online press releases can PR web sites that specialize in online press releases can help in taking the press releases outside of your own site and help in taking the press releases outside of your own site and onto various web sites of relevance. onto various web sites of relevance. i.e. http://prweb.com i.e. http://prweb.comhttp://prweb.com 62