- Deep Web Video
- Federated Search at OSTI (video)
- Federated Search (Wikipedia)
[exit federal site] - OSTI Blog (Federated Search)
Federated search technology will take you to the deep web
If you’ve been searching for science information using popular search engines such as Google, Yahoo! and MSN, you may be missing out on the research you need.
That’s because popular search engines generally cannot search in the deep web where most research is found.
The deep web is huge – by some estimates, the deep web is more than 500 times the size of the surface web where popular search engines "crawl" [exit federal site].
To get to the deep web, you need federated search tools, such as Science Accelerator, Science.gov, and WorldWideScience.org.
With these featured search tools you can find the richest content – the results of billions of dollars worth of government-sponsored scientific research.
In one query, you can search multiple databases at one time, sort through the information in ways that are useful to you, and rapidly return relevant results to your desktop.
Here is how federated search works:
A web patron seeking science information comes to a gateway site, like Science.gov, and enters a query.
Systems that crawl the web do not typically reach below the surface.
Popular search engines "crawl" the surface web, but scientific databases are largely found in the deep web.
Scientific databases stump popular search engines.
The query is transmitted to the gateway server—in Oak Ridge, Tennessee, in the case of Science.gov—and then it is fanned out to a suite of databases across the US, or even the entire world.
At each database, the query launches a search and brings back a hit list. The list is then transmitted back to the gateway server, where the hits are relevancy ranked and presented to the web patron. So, in the span of about 20 seconds, the query is transmitted to numerous databases, searches are executed at numerous databases, and the results are brought back and ranked for the patron.
We need systems that probe the deep web.
Federated search drills down to the deep web where scientific databases reside.
Unlike the popular search engines, federated search places no burden on the database owners.
Federated search is cheap to implement, places no burdens on the database owner, and allows for fielded searching.
Federated search tools at your fingertips
If you haven't visited the deep web, you may not be getting the most authoritative, real-time, scientific information that you need. Try:
You can also visit many of the other web tools and key resources at OSTI.gov.
