featured promotion

Deep Web: Understanding the Hidden Internet

The deep web comprises about 90% of the internet, containing unindexed content like private databases and intranets.

Posted: · Checked: November 18, 2026 · Editor: Rowan Nightshade

What is the Deep Web?

The deep web is the portion of the internet that traditional search engines like Google cannot index or access (2). It represents approximately 90% of the entire internet (1), making the visible "surface web" only a small fraction of what exists online. This vast hidden layer consists of web pages that are not linked from other sites, dynamically generated content, and resources requiring specific credentials or software to reach (2).

What the Deep Web Contains

The deep web includes a wide range of legitimate and everyday content. Private intranets used by companies and organizations form a significant part of this space (3). Commercial databases that require subscriptions or user fees also reside here, along with pages written in PDF format, Active Server Pages (ASPs), and other database-driven content (4). Academic journals, medical records, legal documents, and government databases typically sit behind authentication walls, making them inaccessible to standard search crawlers.

When you log into your email account, online banking portal, or university library system, you are accessing the deep web. These pages exist only after you submit login credentials or query a database. A library catalog search, for example, generates results dynamically based on your query; those result pages do not exist as static URLs that search engines can index (5).

Why Search Engines Cannot Reach It

Search engines rely on crawlers that follow links from one page to another. Deep web content lacks these connections (2). Pages may be generated only in response to specific form submissions, require authentication and permission to view (3), or sit behind paywalls and membership systems. Some content uses non-standard top-level domains or requires specific software like Tor to access (2), placing it further out of reach for conventional indexing.

For a clearer understanding of how to navigate this hidden layer, see How to Access the Deep Web: A Practical Guide.


Types of Deep Web Content

The deep web contains several distinct categories of content, each hidden from search engines for different technical or administrative reasons. Understanding these types helps clarify what lies beyond the reach of Google and other conventional search tools.

Pages Behind Authentication

A large portion of the deep web consists of content requiring login credentials. Accessing this material often requires authentication and permission (3). Corporate intranets, employee portals, online banking interfaces, and subscription-based academic journals all fall into this category. When you check your email or view your medical records through a patient portal, you are navigating deep web content. Search engines cannot index these pages because they cannot provide the username and password needed to view them.

Dynamically Generated Pages

Many deep web pages do not exist as static files on a server. Instead, they are created on demand in response to user queries. The deep web includes dynamically generated pages and databases with restricted access or user fees (4). When you search a library catalog or query a government database, the results page is built specifically for your search terms. A significant amount of information is believed to be hidden in the deep web, behind the query forms of searchable databases (5). These pages disappear after your session ends, leaving no permanent URL for search engines to crawl.

Limited-Access and Private Pages

The deep web contains unlinked pages, dynamic content pages, and limited-access or private pages (2). Some content sits on servers with no inbound links from other websites, making it invisible to crawlers that rely on following hyperlinks. Internal company documents, draft reports, and private file-sharing systems often exist in this state. Without a direct link or knowledge of the exact URL, these pages remain effectively hidden.

Pages Using Non-Common TLDs

Some deep web pages are not indexed because they do not use common top-level domains and require specific software like Tor to access (2). Sites using the .onion domain, for example, exist only within the Tor network and cannot be reached or indexed by standard browsers or search engines. This technical barrier places them firmly within the deep web, even if the content itself is publicly accessible to anyone with the right tools.


How to Access Deep Web Content

Accessing deep web content depends on the type of resource you want to reach. Most deep web material requires authentication and permission (3), meaning you need valid credentials to view it. Corporate intranets, subscription databases, and private email accounts all fall into this category. You cannot simply type a URL into your browser and expect to see the content; you must first prove you have the right to access it.

Authentication-Based Access

For content behind login walls, you need a registered account and password. University library systems, online banking portals, and professional databases grant access only after you authenticate. Once logged in, you can search and retrieve information that remains invisible to search engines. The process is straightforward: navigate to the site, enter your credentials, and use the internal search or navigation tools provided. No special software is required for this type of deep web content.

Accessing Pages with Specific Software

Some deep web pages are not indexed because they do not use common top-level domains and require specific software like Tor to access (2). Sites using the .onion domain exist only within the Tor network and cannot be reached with standard browsers. To access these pages, you must download and install Tor Browser, which routes your connection through multiple encrypted nodes. This software is necessary for any content hosted on non-standard domains that conventional browsers and search engines cannot resolve.

For detailed instructions on using Tor to reach these resources, see How to Access Deep Web Links with Tor Browser.

Direct URLs and Database Queries

If you know the exact URL of a deep web page, you can access it directly by typing the address into your browser. Many deep web pages are unlinked but not restricted; they simply lack inbound links from other sites. Similarly, querying a searchable database generates deep web content on demand. You submit a search term or form, and the system returns dynamically generated results. These pages exist only for your session and are not indexed by search engines.


Why Deep Web Content Isn't Indexed

Search engines like Google rely on automated crawlers that follow links from one page to another, building an index of discoverable content. The deep web is made up of a collection of web pages that cannot be, or are not, indexed by search engines like Google (2). Several technical and administrative barriers prevent these crawlers from reaching deep web content, leaving vast portions of the internet invisible to conventional search.

Lack of Inbound Links

The deep web contains unlinked pages, dynamic content pages, and limited-access or private pages (2). Pages without inbound links from other websites remain invisible to crawlers, which depend on following hyperlinks to discover new content. If no other page points to a resource, search engine bots cannot find it. Internal company documents, draft reports, and private file-sharing systems often exist in this state. Without a direct link or knowledge of the exact URL, these pages remain effectively hidden from indexing.

Dynamically Generated Content

Many deep web pages do not exist as static files. Instead, they are created on demand in response to user queries. A significant amount of information is believed to be hidden in the deep web, behind the query forms of searchable databases (5). When you search a library catalog or government database, the results page is built specifically for your search terms. These pages disappear after your session ends, leaving no permanent URL for crawlers to index. Search engines cannot predict every possible query or submit forms on behalf of users, so this content remains out of reach.

Restricted Access and Authentication

Pages requiring login credentials or permissions cannot be indexed because crawlers lack the necessary authentication. Corporate intranets, subscription databases, and private email accounts all sit behind these barriers. Search engines cannot provide usernames and passwords, so they cannot view or catalog the content. This administrative restriction keeps sensitive and proprietary information off public search results, even though the pages themselves are accessible to authorized users.

Non-Standard Domains and Software Requirements

Some deep web pages are not indexed because they do not use common top-level domains and require specific software like Tor to access (2). Sites using the .onion domain exist only within the Tor network and cannot be reached or resolved by standard browsers or search engines. This technical barrier places them firmly outside the scope of conventional indexing, regardless of whether the content is public or restricted.

Deep Web vs. Dark Web: Key Differences

Characteristic
Definition
Deep Web
Content not indexed by search engines
Dark Web
Encrypted networks requiring specific software
Characteristic
Size
Deep Web
Approximately 90% of the entire internet
Dark Web
Small subset of the deep web
Characteristic
Access Method
Deep Web
Standard browsers with authentication or direct URLs
Dark Web
Tor Browser or similar anonymity software
Characteristic
Typical Content
Deep Web
Intranets, databases, paywalled journals, email accounts
Dark Web
Anonymous forums, marketplaces, hidden services
Characteristic
Domain Types
Deep Web
Standard TLDs (.com, .org, .edu)
Dark Web
Non-standard TLDs (.onion, .i2p)
Characteristic
Indexing Barrier
Deep Web
Authentication, dynamic generation, lack of links
Dark Web
Encryption and non-standard protocols
Characteristic
Legality
Deep Web
Entirely legal and routine
Dark Web
Legal to access; content legality varies
Characteristic
Primary Purpose
Deep Web
Privacy, security, database functionality
Dark Web
Anonymity and censorship resistance

Common Misconceptions and Mistakes

The deep web is often misunderstood, leading to confusion and misinformation. We clarify some common misconceptions to help you better understand this hidden layer of the internet.

Confusing Deep Web with Dark Web

Many people mistakenly equate the deep web with the dark web. However, the deep web is a much broader term that encompasses any content not indexed by search engines, including legitimate and legal content like corporate intranets and academic databases (3). The dark web, on the other hand, is a small subset of the deep web that requires specific software like Tor to access and is often associated with anonymity and illicit activities. To avoid this mistake, remember that not all deep web content is hidden for nefarious reasons.

Assuming Deep Web Content is Illicit

Another common misconception is that the deep web contains primarily illicit or illegal content. In reality, the deep web is estimated to be approximately 90% of the entire internet, and most of it consists of benign content like private databases, email accounts, and subscription-based services (1). To put this into perspective, accessing deep web content often requires authentication and permission, which is typically used for legitimate purposes (3).

Believing Deep Web is Inaccessible

Some people believe that the deep web is entirely inaccessible. While it's true that some deep web content requires specific software or authentication to access, much of it can be reached using standard browsers and direct URLs. For example, you can access a library catalog or a government database by submitting a query through a publicly available form (5).

Overestimating Risks of Accessing Deep Web

There's a misconception that accessing the deep web is inherently risky. While it's true that some parts of the deep web, like the dark web, can pose risks, most deep web content is safe to access as long as you follow proper authentication and security protocols. To minimize risks, ensure you have valid credentials and use the right software for the type of content you're trying to access. For instance, accessing a corporate intranet requires a valid login, while accessing .onion sites requires Tor Browser (2).

Key Takeaways

To summarize, here are the main points to remember:

  • The deep web is approximately 90% of the entire internet and contains content not indexed by search engines (1).
  • Accessing deep web content often requires authentication, permission, or specific software like Tor (3) (2).
  • The deep web includes private databases, email accounts, and subscription-based services, which are mostly benign (3).
  • Not all deep web content is hidden for nefarious reasons; most is legitimate and legal.

To explore the deep web further, start by learning How to Access the Deep Web: A Practical Guide.

People also ask

What is the deep web?

The deep web is the portion of the internet that is not indexed by traditional search engines like Google. It is estimated to comprise approximately 90% of the entire internet (1). The deep web contains content that has not been indexed, including private intranets, commercial databases, and pages with restricted access or user fees.

Where can I find deep web sites?

Deep web sites can be found through various means. Some require direct URLs or authentication, while others need specific software like Tor to access. You can access deep web content by knowing the exact URL, using authentication for login-protected sites, or querying searchable databases.

Is it illegal to enter the dark web?

Accessing the dark web is not inherently illegal. The dark web is a subset of the deep web that requires specific software like Tor to access. While some content on the dark web may be illicit, the act of accessing it is not illegal; the legality depends on the content you access.

Can I browse the dark web?

Yes, you can browse the dark web using specific software like Tor Browser. Tor allows you to access .onion domains and other content not indexed by standard search engines. However, be cautious as the dark web contains a mix of legitimate and illicit content.

Is it illegal to browse Tor?

Browsing Tor is not illegal. Tor is a tool that provides anonymity and is used for various legitimate purposes, including privacy protection and censorship resistance. The legality of using Tor depends on how it is used; accessing illegal content is illegal, but using Tor itself is not.

  1. BSI - Dark net and deep web - Dark net and deep web: shedding light on the dark
  2. The Deep Web
  3. If12172.web.html
  4. The Deep Web – Favourite Articles – Writing Tools – Resources of the Language Portal of Canada – Canada.ca
  5. Accessing the Deep Web – Communications of the ACM