Invisible Web vs Surface Web: The sites which we are searching on the web is known as a surface web [includes deep web links and dark web sites]. This surface web consists of indexed pages that a search engine’s web crawlers are programmed to retrieve.
Then the question comes Where are the rest? The major part of the web lies as invisible web, sometimes referred to as the Deep web. The actual size of the invisible web is impossible to measure.
The invisible sites are similar to our normal websites only. But it is not get crawled by any search engine or web crawlers. The recent famous news in the invisible web is SILK ROAD drug dealing.
Invisible Web Comprised of
- Private websites, such as VPN (Virtual Private Networks) and sites that require passwords and logins.
- Limited access content sites (which limit access in a technical way, such as using Captcha, Robots Exclusion Standard or no-cache HTTP headers that prevent search engines from browsing or caching them).
- Unlinked content, without hyperlinks to other pages, which prevents web crawlers from accessing information.
- Textual content, often encoded in image or video files or in specific file formats not handled by search engines.
- Dynamic content created for a single purpose and not part of a larger collection of items.
Why would you want to search the Invisible Web?
The quality of resources on the Invisible Web is often better than what you will find on the public web. Professionals with specific subject matter expertise maintain many database collections. Materials on the fee-accessed parts of the Invisible Web are likely to be more current and professionally prepared. Also, the Invisible Web is thought to be from 2 to 500 times larger than the public web. If you don’t want to miss what might be the very best sources of information on your topic, use the Invisible Web.