1. Search Engine Marketing
Bob Keating
Search Program Manager
FirstGov Technologies

2. What is Search Engine Optimization (SEO)?
A set of web design and development methodologies aimed at improving the ranking of web pages in search engine results
What is Search Engine Marketing (SEM)?
Methodologies for submitting and positioning a website within proper search engines for maximum exposure. Often includes SEO.

3. Why is SEM Important?

4. People Use Search Engines
Search is the second most popular activity after
60 million Americans using search engines during a typical day, an increase of 55% from June ’04 to Sept ’05
Hundreds of millions of searches each day
Industry watchers estimate that Google gets 59 million unique visitors producing over 250 million search queries each day

5. Program Performance
Web metrics are used to report on program performance to senior management
Maximize web traffic via the features citizens use most
54% of Web users are looking for a government website – you want to rank high for your agency’s keywords

6. Government Policy OMB Memorandum, Dec. 2005
“When disseminating information to the public-at-large, publish your information directly to the Internet. This procedure exposes information to freely available and other search functions and adequately organizes and categorizes your information.”

7. OMB Memorandum Major Assumptions:
Publishing information to the Internet means search engines will find it
Publishing information to the Internet means the information is accessible
Webmasters will take any additional steps to ensure information is found and adequately organized by search engines
Webmasters understand how search engines work

8. GOAL: Help Webmasters Understand …
How all search engines handle their content
How to design, write and publish search engine friendly pages
How to maximize visibility of their pages for search engines and users
How to get the attention of search engines when publishing content

9. SEM Topics Overview of Search Engines and Web Directories
Search Engine Friendly Design
Managing Page Content
Managing Site Architecture
Design Considerations
Managing Link Popularity
Site Maintenance Issues
Recommended Resources

10. Search Engines and Directories

11. Major Search Engines and Directories
Search Engines Producers
Google
Yahoo!
MSN Search
Ask.com
Gigablast
WiseNut
Web Directories
Open Directory Project (DMOZ)
Yahoo! Directory

12. Supplies Primary Results Supplies Secondary Results or other info
Based on Bruce Clay’s chart:
Supplies Primary Results
Supplies Secondary Results or other info
Supplies Directory Results
Supplies Paid Results

13. Directories
Open Directory
Looksmart
Business.com
About.com
Indexes
Ask.com
Gigablast
Wisenut
Entireweb
Google
Yahoo
MSN
+ News Sources
Paid
Listings
FindWhat
Kanoodle

14. comScore Media Metrics Search Ratings
Source: Search Engine Watch, Aug. 23, 2005,

15. Request URL from your web server, which gives the page to the Search engine
Follow a link to URL
Spider starts at major ISPs and Directories
Web Page
Search Index
Crawler reads page & makes a list of words and phrases
User search index
Most are completely updated 4-6 wks
Search Engine applies ranking algorithms, factoring in text, link and popularity components

16. Text (Keyword) Component
Basic Page Components
Text (Keyword) Component
Words and phrases that match what your target audience types into search engines
Link Component
Site navigation and URL structure that search crawlers can easily follow
Popularity Component
Are sites link to you?

17. Text: Keyword Density
Choose one or two keywords or phases to optimize for each page
Do not over use - avoid keyword stuffing
Over use of keywords can result in being penalized or ignored
Incorporate other complimentary words and phrases
Check Keyword Density:

18. Text: Keyword Prominence
Search engines place “weight” on terms according to where they are used
Place Keywords in …
Title tags
Headings and emphasized text
Visible body text
Description meta tags
Alt text in images
Title and body tags are most important
Keyword in the URL is helpful, but not a significant factor in ranking

19. The Metadata Myth Quote: “Metadata improves search relevancy”
False. Except for title and description tags, Web search engines ignore other metadata. However, Metadata does matter to enterprise search.
Quote: “Using standard metadata is a best practice.”
False. It is only a best practice if the metadata is used by your agency for specific applications (e.g. enterprise search or content syndication). It is not a best practice among professional web designers.

20. The Metadata Myth
Quote: “If more agencies were diligent about adding metadata, Google would pay attention”
False. Industry is focused on developing algorithms that determine relevancy based on content rather than what an author or metadata creator says about the content
Key points
Metadata is not the key to high rankings
Only use metadata if it is important to your agency’s mission
Paying more attention to keyword usage in other areas will have a greater impact on relevancy

21. Link Component
Pages will not rank well if your site does not have a navigation scheme
Navigation scheme must please users and search engines
Create a site map, but also plan how pages link to each other
Avoid dangling pages

22. Problem Navigation Schemes
Poor HTML coding
Image maps
Frames
JavaScript
Dynamic Pages
Flash

23. Creating Links
Keywords in links tell crawlers about the pages to which you are linking
Keywords in links influence relevancy of the page to which you are linking
Avoid “click here” links, instead create links like:
Bad: Click here for more info on famous admirals.
Good: Visit our naval history site for more info on famous admirals.

24. Popularity Component Based on the number pages that link to you
The more popular pages that link to you, the higher your popularity
All search engines have different popularity algorithms
Google’s algorithm is called Page Rank
Every page on the Web is given a calculation of it is popularity based on inbound links

25. Popularity Factors Number and Popularity of Inbound Links Anchor Text
Get listed Yahoo!, DMOZ, and FirstGov.gov
Network with other agency and industry sites
Make your site a link magnet
Anchor Text
Others’ use of keywords in a link to your site
Popularity is assigned per page, not for the entire site
Popularity is not inherited
Need to deliberately link internal pages to pass on PR

26. Outbound Links
Your popularity is not determined by the sites you link to
Your outbound links affect the popularity of the sites you’re linking to
Internal links and inbound links have the most impact on your popularity
Outbound links help identify you with a hub

27. Robots Exclusion Meta-Tag Robots Exclusion Robots.txt File
<head><meta name=“robots” content=“no index, nofollow”></head>
Robots.txt File
Place in server’s root directory
Two elements: User-agent, Disallow
Example:
User-agent: *
Disallow: /cgi-bin/
Disallow: /scripts/
Disallow: /images/

28. Robots Exclusion
Not all search engines pay attention to robots.txt instructions
MSN and Yahoo! obey robots exclusion more often than Google
Never exclude msnbot … you won’t have site search
Blocking bots is contrary to OMB’s guidance

29. Case Study: NOAA Fisheries
Term: Fisheries
#1 on FirstGov
#1 on MSN
#2 on Google
#3 on Yahoo
Term: Fish
Doesn’t show up on the first page or subsequent pages

31. Case Study: NOAA Fisheries
Text Component: page is well-optimized for “fisheries” but not “fish”
Link Component: Fly-out navigation may be a problem for some crawlers. Add search friendly navigation (e.g., site map)
Popularity Component: Lots of inbound links using “fisheries” (not “fish”) in anchor text

32. Basics of Search Friendly Design

33. Basic Concepts
Page Content: using content and text that target your audiences; and attract search engines and links from other sites
Navigation: giving users and crawlers easy access to content
Design Considerations: make sure bells and whistles don’t undermine SEM efforts

34. Managing Page Content

35. Types of Pages Home Page About Contact Site Map News Forms Galleries
FAQs
Catalogs
Product pages
Shopping cart
Search Results
How you write, design and optimize a page depends on the type of page

36. Text to Include Keywords Page Content Make content appear focused
Use language of your audience
Use keyword selection tools
Yahoo!/Overture:
Google:
Page Content
Make content appear focused
Title tag, headings, contextual links, cross-links
Body text should be visible, i.e., should not have to do any action to view main page text

37. Primary vs. Secondary Text
Primary Text
Title tag
Body text
Text near the top of the page
Text in and around links (e.g. anchor text)
Secondary Text
Alt text
Description tag
Domain name and URL elements

38. What Kind of Content to Include?
Write your own
Pros: Original, unique content
Cons: Time consuming, bureaucratic process
Use someone else’s
Pros: Easy way to bulk up content
Cons: Still need to worry about how to add value in order to attract links

39. Use Syndicated Content
Tends to update frequently
Crawlers visit frequently-updated pages more often
Combine syndicated content to create a unique resource
Syndicate your own content (e.g. RSS)
Increase site visibility and attracts traffic
Another opportunity for inbound links

40. Managing Site Architecture

41. Site Navigation Scheme
Text Links
Very search engine friendly
Use for primary or secondary navigation
Problems with Text Links
Can negatively skew keyword density
Crawlers tend to read text links first

42. Site Navigation Scheme
Navigation buttons
Okay as long as you include alt text
Avoid JavaScript, unless you can provide navigation crawlers can follow
Recommendation: use alt text and text navigation at the bottom of the page – allows you put keywords in multiple places

43. Site Navigation Scheme
Image Maps
Crawlers ignore links inside image maps
User text links or navigation buttons elsewhere
Pull Down Menus
Generally not crawler friendly because they need JavaScript or a CGI program
Always provide two forms of navigation: one for your users, and one for your crawlers

44. Help Crawlers Navigate
Create a site map
Subscribe to Google Site Maps
A crawler enabling tool to assist the Google crawler
Analyzes information about your site’s architecture to improve crawling

45. Design Considerations

46. Design Considerations
Bells and whistles (Flash, JavaScript, animation) enhance user experience but can hurt search visibility
Implement features carefully in order to keep search engine ranking
Ensure design team understands SEM concepts

47. Use External JavaScript and CSS
Using JavaScript on site navigation can greatly decrease site crawlability
Crawlers do not follow links embedded inside JavaScript code; or they limit the types of embedded links crawled
JavaScript in <body> can decrease page load time
Long download times may indicate the site is spam, and crawlers could ignore

48. Use External JavaScript and CSS
External files decrease page load time for visitors
External files decrease download time for crawlers
Remember to disallow crawling scripts in the robots.txt
External scripts are easier to re-use

49. Frames Do not design in frames
Crawlers have trouble getting from the frameset page to the actual web page
Frameset does not provide crawler with keyword rich text and links
Each page is indexed separately, so pages that only make sense as a frameset will be indexed individually in search engines
<no frames> tag is ignored due to spam

50. Frames Workarounds Add Navigation
Give all pages unique title and description tags
Put navigation links on your pages
Add Java Script in your <head> tag
<script language=“javascript”>
<! –
if (top == self) self.location.href = “index.html”;
// 
</script>
This will force the browser to always load the frameset
However, browser will always load the home page, not the indexed page
Back button will be disabled

51. Flash
Few search engines crawl links embedded inside a Flash navigation scheme
Flash sites contain little text
If you include Flash …
Include a “Skip” link so both the user and crawler can go to the real homepage
Include title and description meta tags

52. Dynamic Pages
Database-driven, created on the fly by asp, cfm, php, jsp or cgi scripts
Dynamic sites are comprised of templates, but usually without original content
When a page is viewed, the template loads the content from the database
Parameters are added to the URL, which tells the template to load specific content

53. Dynamic Pages
Example:
URLs such as this are difficult for search engines to index because they do not know the parameters that define a unique page
The more parameters, the less likely pages will be indexed
A database may continually feed data, crashing your server and scaring off the crawler

54. Search Friendly Dynamic Pages
Create static HTML pages
Modify URLs so they don’t look like dynamic pages, fewer parameters
Use URL re-write trick using mod_rewrite

55. Session IDs The kiss of death if left unmanaged
Same content is delivered to the crawler but as unique URLs
Crawlers will ignore web pages with session IDs
Omit session IDs if the requestor is a crawler … but no cloaking!

56. Optimizing PDFs Make sure PDFs contain actual text, not images of text
Same rules for use of keywords and phrases apply
Put the most important text in the title, headlines
Minimize document size (> 100K)
Create optimized HTML pages for PDFs

57. Managing Page Rank

58. Understanding Page Rank
All search engines assign a value to your site based on inbound links
Google calls this relevancy factor “Page Rank” (PR) - synonymous with popularity ranking
An inbound link is a vote for your page
An outbound link is a vote for the page you’re linking to

59. Understanding Page Rank
A page is assigned PR as soon as it’s indexed
Make sure all your pages have at least one link back into your site
The page receiving the most inbound links gets the highest page rank
You can pass PR around to pages on your site

60. Can You Leak Page Rank? Controversial topic among SEOs
Supporting View: PR leaks in the sense that a page’s outbound links will decrease that page’s available PR for redistribution throughout the site
Opposing View: Search engines analyze inbound and outbound links to determine your authority as hub. No site is an island

61. Outbound Link Strategy
Do not create pages with mostly outbound links
Link to quality, related sites – helps to establish you as an authority or hub
If a page contains several outgoing links, also include links to other pages on your site
Don’t be afraid to link to sites with low page rank, but quality content

62. Getting Links to Your Site
Links from Yahoo!, DMOZ, and FirstGov
Impact on popularity may vary
Helps to get noticed by crawlers
Establish reciprocal link arrangements with agencies covering similar topics
Reach out to state and local agencies
Syndicate your content
Reach out to professional communities of interest

63. Site Maintenance

64. Re-Designing Your Site
Plan BEFORE you embark on a re-design
Will the site architecture change?
Static to Dynamic?
How is the content changing?
Add SEO review as an activity to your project plan
Make sure your contractor has SEO skills
Poor planning and execution can kill your search rankings
Do not wait until after you re-designed!
Do not ask the FirstGov Search Team to re-crawl your site!

65. Changes to Site Architecture
Try to keep the same filenames and directory structure when redesigning
Follow MSN’s “What to do when your site moves”:
Recommended: Set up HTTP 301 redirects that point to the new site or pages
Not Recommended: Add a meta-refresh tag to your page header. This won’t remove your original page from the MSN index, and thus your site’s search engine.

66. Removing Content Avoid using default 404 pages
Custom 404 pages are more user-friendly
Submit all 404 URLs to the search engines using their Add URL form – quickest way to get the 404 pages out of the index
Remember to change the HTTP status code on custom error pages from 200 to 404
Do not ask the FirstGov Search Team to re-crawl your site!

67. Resources and Recommendations

68. Recommendations Read Industry Forums Go to Industry Events
Webmaster World:
Search Engine Watch:
Digital Point Forums:
High Rankings:
Go to Industry Events
Search Engine Strategies:

69. Recommended Reading
Battelle, John. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed our Culture. New York. Penguin, 2005.
Kent, Peter. Search Engine Optimization for Dummies. Hoboken, NJ. Wiley, 2004.
Moran, Mike, Hunt, Bill. Search Engine Marketing, Inc. Upper Saddle River, NJ. IBM Press, 2006.
Thurow, Shari. Search Engine Visibility. Indianapolis, IN: New Riders, 2003.

70. Contact Information Bob Keating Search Program Manager (202)