Discussion:
To understand how Web pages can be made easier to find by search engines, one must know how search engines work. They index Web site content and maintain the results in databases. Although the indexing schemes used by search engines vary, they use programs called robots to build the index. The robot reads a Web site, follows any hyperlinks, and enters the HTML tag information and Web page text into its database. The results of indexing, and hence the relative ranking of a Web page or site, will vary with each search engine.
In general, a Web page is most likely to be found through a search when the following elements are included in the <HEAD> section of the page:
- <TITLE> tag: words describing the content of the Web page (since search results are often displayed based on contents of the <TITLE> tag).
- Description <META> tag: a brief description of the page or site (since search results may display this as a page summary).
- Keyword <META> tag: words people would associate with the content of the Web page and might use in a Web search.
- Robot <META> tag: allows HTML authors to indicate to visiting robots whether a document should be indexed or used to harvest more links.
Web developers and administrators should consider submitting their Web sites to the major search engines. Submission lets the search engine know the Web site exists and is ready to be indexed. Site administrators should review each search engine's Web site for information about effective submission and be aware that, due to demand on search engines, indexing may not occur until months after submission. When a DOC organization Web address changes, the Web site should be re-submitted to the major search engines.
In those instances where a page should not be indexed, use of the Robots Exclusion Protocol should be considered. |
