What Is URL Filtering? How It Works, Examples & DNS Comparison

URL filtering is a security and access-control technique that evaluates web destinations against an organization's rules. Depending on the system and what it can observe, a rule may apply to an entire domain, a URL category, or a specific path such as example.com/uploads.

That last distinction matters. Most web traffic uses HTTPS, so a network service does not automatically see every part of a URL. A filter may know that a device connected to example.com while being unable to see the requested path, query string, headers, or page content. Full URL filtering therefore depends on where the filter runs and whether the organization has configured permitted TLS inspection or endpoint-level visibility.

Quick Answer

URL filtering controls access to web destinations by comparing an observed domain or URL with security, reputation, category, and organization-specific rules. It can allow, block, warn, isolate, or log a request. Domain-level filtering can work without seeing the complete page address, but path-level filtering of HTTPS traffic normally requires an endpoint agent, browser integration, or authorized TLS inspection. URL filtering complements DNS filtering and firewalls; it does not replace them.

Key Takeaways

• URL filtering is a policy function, not one particular appliance. It can be implemented in endpoint software, browsers, firewalls, forward proxies, secure web gateways, or cloud security services.
• The word “URL” can overstate what a product actually sees. With HTTPS, a network intermediary may see a hostname but not the path, query, headers, or content unless it has appropriate application-layer visibility.
• DNS filtering works earlier and at a broader level. It controls how selected domain-name queries are answered, but it does not normally distinguish two pages on the same hostname.
• A firewall and a URL filter answer different questions. A traditional network firewall evaluates values such as IP addresses, ports, and protocols; a URL filter evaluates web destinations and related context. Modern security products may provide both.
• A proxy can enforce URL-filtering rules, but not every proxy does. Routing traffic through a proxy does not automatically create a filtering policy or give the proxy access to encrypted page paths.
• A block does not have one universal HTTP status. Users may receive a block page, warning, redirect, DNS response, connection failure, or certificate error depending on the enforcement point.
• Good policy design includes an exception process. False positives, newly registered domains, changing site categories, certificate pinning, and legitimate business requirements all need review paths.

Methodology: We reviewed the previous Proxidize article, six months of Search Console demand, RFC 3986 and RFC 9110, current Cloudflare Gateway and Microsoft Defender documentation, and CISA secure-network-access guidance. The examples below use reserved or fictional domains and illustrate behavior; they are not results from a tested filtering product.

What Is URL Filtering?

URL filtering is the process of applying an access policy to web requests based on the requested destination and other available context. A policy can use one or more of the following:

• an exact domain, hostname, or URL;
• a domain or URL category, such as known phishing or gambling sites;
• a reputation or threat-intelligence result;
• a path, query, filename, or file type when those values are visible;
• the user, group, device, network, or time of request;
• an organization-maintained allowlist or denylist;
• an application or requested action, when the platform supports application-aware controls.

A match can lead to several outcomes. The system may allow the request, block it, display a warning, require an exception, open the destination in an isolated browser, or record the event for review.

URL filtering is commonly included in secure web gateways, endpoint-security platforms, next-generation firewalls, and managed browser systems. A forward proxy can also act as the enforcement point when client web traffic is deliberately routed through it.

The exact capability matters more than the product label. One service may block only domains and broad categories. Another may inspect complete HTTP requests, identify individual application actions, scan downloads, and apply data-loss-prevention rules. Those are related capabilities, but they should not all be assumed from the phrase “URL filtering.”

Which Parts of a URL Can Be Filtered?

Consider this example:

RFC 3986 defines the generic URI components and explains that a fragment is separated before dereferencing. This is why a filtering product positioned on the network may receive https://docs.example.com/security/filtering?mode=strict but not #exceptions.

Do not infer path-level filtering from a marketing statement about “HTTPS support.” A service can block an HTTPS hostname without seeing the individual page requested on that hostname.

How Does URL Filtering Work?

Implementations differ, but a URL-filtering decision generally follows six stages.

1. The system observes or receives the request

Traffic might pass through an explicit proxy or secure web gateway. Alternatively, an endpoint agent, managed browser, browser extension, or firewall can supply the necessary request context.

The deployment determines which traffic is covered. A browser extension may see browser traffic but not a command-line tool. A forward proxy protects only clients and applications configured to use it unless network controls force the route. An endpoint agent may cover managed devices away from the office network.

2. It extracts the destination it can see

For plaintext HTTP, an application-layer filter can usually inspect the hostname, path, query, method, and headers. For HTTPS, its visibility depends on the architecture. It may see only an IP address and port, a hostname, or a decrypted request containing the full URL.

3. It normalizes and classifies the destination

The filter may normalize capitalization or encoding according to its matching rules, check an exact custom list, assign a category, and consult a reputation or threat-intelligence service. Categories can include known malware, phishing, newly observed domains, file sharing, social media, streaming, or organization-specific groups.

Categorization is not infallible. A new domain may be uncategorized, a legitimate site may be assigned the wrong category, and a previously safe site can be compromised. Some systems therefore combine categories with reputation, content inspection, and local overrides.

4. It adds identity and device context

The same destination may be allowed for one group and restricted for another. A marketing team may need social-media access that is unnecessary for other departments. A developer may need a package repository that should remain blocked for unmanaged devices.

Useful context can include:

• authenticated user or group;
• managed versus unmanaged device;
• operating system or browser;
• device security posture;
• network or location;
• time of day;
• requested application action.

5. It evaluates policy precedence

The engine compares the request with ordered rules. A specific allow exception may take precedence over a category block, or a “do not inspect” rule may be evaluated before an HTTPS inspection rule. The precedence model is product-specific and should be tested rather than assumed.

6. It enforces and records the outcome

Logs can support troubleshooting and security investigation, but browsing records can be sensitive. Organizations should limit collection and retention to what their operational, legal, and security requirements justify.

URL Filtering Example

Suppose a company wants to reduce phishing risk while preserving access to legitimate cloud-storage documentation.

Its policy could work like this:

1. Block domains categorized as phishing, malware, or command-and-control infrastructure.
2. Warn users before visiting newly observed or uncategorized domains.
3. Allow docs.storage-example.com for all managed employees.
4. Restrict a particular upload path to an approved group, but only if the filtering deployment can see HTTPS paths.
5. Exempt an internal application from TLS inspection because it uses mutual TLS, after a documented risk review.
6. Log the policy identifier, observed destination, user or device identifier, action, and time.
7. Provide a review form when a legitimate site is categorized incorrectly.

This example uses layers. DNS filtering can stop known malicious domains before a connection is made. URL filtering can make more specific web-request decisions. Endpoint security can still inspect files and behavior after a page is allowed. No single layer proves that every permitted page is safe.

Can URL Filtering Inspect HTTPS Traffic?

Sometimes—but not automatically.

HTTPS uses TLS to protect the HTTP request and response. Without endpoint visibility or TLS decryption, an intermediary generally cannot inspect the complete path, query string, request headers, form data, response body, or downloaded content.

What remains observable varies by protocol and deployment:

An explicit HTTP proxy normally receives a CONNECT host:port request before tunneling HTTPS. RFC 9110 defines CONNECT using the host and port of the tunnel destination. After the tunnel is established, the proxy forwards encrypted bytes unless it has been deliberately configured as an inspecting intermediary. The HTTP CONNECT guide explains that boundary in more detail.

For full HTTPS inspection, a secure web gateway commonly terminates the client's TLS connection, evaluates the decrypted request, and creates another TLS connection to the origin. That normally requires an organization-controlled certificate trusted by managed devices. Cloudflare's current TLS-decryption documentation similarly states that full URLs, headers, and request bodies are not visible to its gateway without TLS decryption.

This design has consequences:

• certificate pinning can cause applications to reject the inspecting certificate;
• mutual TLS may fail because the intermediary cannot transparently forward the client's certificate exchange;
• Encrypted Client Hello can reduce hostname visibility for controls relying on SNI;
• unsupported protocols or applications may need a documented no-inspection rule;
• inspection can add latency and operational complexity;
• decrypted traffic and logs create privacy, security, and governance obligations.

CISA's secure-network-access guidance describes secure web gateways as URL-filtering and web-access-control enforcement points while also noting encrypted-traffic analysis. An organization should define inspection scope with its security, privacy, legal, and employee-policy requirements—not enable broad decryption without review.

Browser and endpoint support also differs by product. For example, Microsoft's current Defender documentation says that its web-content-filtering implementation can block full HTTPS URL paths only in Microsoft Edge. Buyers should verify the exact browsers, applications, and protocols a proposed control covers instead of assuming that one product behaves identically everywhere.

What Types of URL-Filtering Rules Are Common?

Keyword-only matching is usually a weak primary control. A harmless URL may contain a blocked word, while an unsafe page may use an unrelated path. Categories, reputation, exact rules, and request context generally produce more defensible decisions.

DNS Filtering vs. URL Filtering

DNS filtering controls whether a resolver returns a usable answer for selected domain-name queries. URL filtering evaluates web destinations at the HTTP or browser layer. They overlap in purpose but have different visibility and enforcement points.

A DNS filter does not directly inspect a page. It can refuse a domain, return a sinkhole address, or apply another resolver policy. It also does not necessarily control requests made with a cached address, literal IP address, or an unapproved encrypted resolver unless the organization enforces the intended DNS path.

Our DNS server guide explains recursive and authoritative resolution. In practice, organizations often combine a controlled resolver with URL or HTTP policies: DNS rejects known-bad domains early, while the web layer handles permitted domains that need more specific rules.

URL Filtering vs. Web Filtering vs. Firewalls

These terms are related but not interchangeable.

“Web filtering” is often used as an umbrella term covering URL filtering, category filtering, content inspection, malware scanning, safe-search enforcement, and application controls. A modern firewall or secure web gateway may offer several rows from the table in one product.

The useful procurement question is not “Does this product have web filtering?” It is: Which traffic does it cover, which request fields can it inspect, what happens with HTTPS, and which policy actions can it enforce?

How Are Proxies Related to URL Filtering?

A proxy server is an intermediary that receives traffic from a client and communicates with a destination on the client's behalf. URL filtering is a policy decision about whether and how a destination may be reached.

The two can work together:

• An enterprise forward proxy can authenticate users, record requests, and enforce URL rules.
• A secure web gateway can combine proxying with TLS inspection, malware scanning, and data-loss prevention.
• A basic tunneling proxy may know only the destination host and port for HTTPS traffic.
• A SOCKS proxy transports connections and does not inherently apply HTTP-aware URL policy.

They are not synonyms. Using an egress proxy does not automatically create URL filtering, and adding a URL filter does not necessarily change the public exit IP a destination observes.

Proxidize provides managed residential and mobile proxy infrastructure for legitimate web access, data collection, monitoring, and testing workflows. It is not positioned as an employee-monitoring or corporate secure-web-gateway product. Proxidize changes the configured traffic route and exit IP; the customer's application and security stack remain responsible for request policy, browser state, authentication, data handling, and permitted use.

Likewise, an employee who encounters an organization-managed block should use the documented exception process. A third-party proxy should not be used to defeat an employer's, school's, or network owner's access policy.

What Happens When a URL Is Blocked?

There is no universal response.

HTTP 405 Method Not Allowed has a specific protocol meaning: the origin recognizes the request method but does not allow it for that resource. It is not a standard signal for URL filtering. Our proxy error-code guide explains why the status code and the component that generated it must be identified before troubleshooting.

How Do You Troubleshoot a Legitimate Site Blocked by a URL Filter?

Use a controlled diagnostic process instead of trying random workarounds.

1. Record the exact destination and time. Include the complete URL if company policy permits, the user or device, network, browser, and whether the problem affects one page or the entire site.
2. Read the block message. Save the policy name, category, request or incident ID, and named security product. Do not share credentials or sensitive query values in a public ticket.
3. Determine the failure layer. Check whether DNS resolution failed, a TCP/TLS connection failed, a proxy rejected authentication, a filtering product displayed a block page, or the origin returned an HTTP error.
4. Compare an approved control. Test another permitted page on the same domain or the same destination from another managed device, if your organization authorizes that comparison.
5. Review the relevant logs. DNS, firewall, proxy, endpoint, and HTTP logs answer different questions. Browser DevTools, an HTTP debugging proxy, and Wireshark also provide different levels of visibility.
6. Check policy precedence and categorization. Confirm which rule matched and whether an allow exception was placed below a broader block.
7. Check HTTPS-inspection compatibility. Certificate pinning, mutual TLS, self-signed certificates, and unsupported protocols may require a narrowly scoped no-inspection rule after review.
8. Submit a documented exception. Explain the business requirement, required scope, users or devices, duration, and any compensating control.
9. Retest and close the loop. Verify that the smallest approved exception works and that broader restricted access remains blocked.

Do not tell users to disable certificate validation, install an unknown certificate, change to an unauthorized resolver, or route around a managed policy. Those actions can create a larger security problem than the original false positive.

URL-Filtering Benefits and Limitations

Benefits

• Blocks known malicious destinations before or during a browser request.
• Applies acceptable-use rules consistently across managed users and devices.
• Supports more precise exceptions than an all-or-nothing domain block when full URLs are visible.
• Adds identity, device, location, and time context to access decisions.
• Creates reviewable records for incident response and policy improvement.
• Can reduce access to high-bandwidth or non-business categories when that is a documented organizational goal.

Limitations

• Categories and reputation data can be wrong or incomplete.
• Newly created or compromised legitimate sites may not be classified quickly.
• HTTPS limits page-level visibility unless the architecture provides permitted inspection.
• TLS decryption can break pinned or mutually authenticated applications and creates privacy obligations.
• Rules can be bypassed accidentally when applications use an uncontrolled DNS resolver or unsupported traffic path.
• Broad categories can block valid business activity.
• Filtering adds policy, certificate, exception, logging, and support overhead.
• A permitted destination can still contain unsafe content or become compromised later.

URL filtering reduces a class of risk. It does not replace endpoint protection, email security, identity controls, patching, backups, user training, DNS controls, network firewalls, or incident response.

URL-Filtering Policy Best Practices

Define the security outcome first

“Block more websites” is not a useful objective. State the risk or requirement: reduce access to known phishing infrastructure, prevent unmanaged devices from reaching an internal application, or restrict unapproved file uploads.

Start in audit mode where practical

Observe which rule would match before enforcing a broad category. Review business dependencies and false positives, then move high-confidence rules to block or warn.

Use layers at the appropriate granularity

Use DNS controls for broad domain-level protection, network policy for ports and protocols, URL filtering for web destinations, and endpoint or content controls for files and behavior. Avoid making one layer imitate every other layer.

Scope access by identity and device

Give users the access their roles require instead of creating an organization-wide exception for one team's workflow. Apply additional restrictions to unmanaged or unhealthy devices when the platform supports reliable posture checks.

Design TLS inspection deliberately

Document which traffic is decrypted, which destinations are exempt, how certificates are deployed, where processing occurs, which data is logged, and who can access those logs. Test applications that use pinning, mutual TLS, HTTP/3, or non-browser clients.

Maintain an exception and recategorization process

Every block page should tell legitimate users how to request review. Require a business reason, limit the exception's scope, assign an owner, and set a review date for temporary access.

Minimize and protect logs

Full URLs and query strings can contain personal, confidential, or authentication-related information. Avoid collecting unnecessary values, redact secrets, apply access controls, and define a justified retention period.

Measure policy quality

Useful measurements include confirmed malicious requests blocked, false-positive rate, time to resolve legitimate requests, uncategorized-domain volume, inspection failures, certificate-related incidents, and policy coverage by device and traffic type. Raw block count alone can reward a noisy policy.

Conclusion

URL filtering applies security and access rules to web destinations. It can operate at domain level or, when the deployment can observe the full request, at page and path level. HTTPS makes that visibility conditional: an ordinary network intermediary or CONNECT proxy does not automatically see the complete encrypted URL.

Use URL filtering alongside DNS, firewall, endpoint, identity, and content controls. Define what each layer can observe, test policy precedence and HTTPS compatibility, protect the resulting logs, and give legitimate users a clear exception process.