Most Common HTTP Errors and How to Fix Them

HTTP errors are the short status codes your server returns when a request cannot be completed, and they are often the fastest clue to what broke. Most problems fall into 4xx client-side issues like 404 Not Found or 401/403 access failures, and 5xx server-side failures such as 500 Internal Server Error, 502 Bad Gateway, or 503 Service Unavailable. Fixes usually start with confirming the URL and method, checking authentication and permissions, then tracing what the app and any proxy or upstream service logged at the same time. One easy-to-miss pitfall is masking a real failure with a friendly HTML page that still returns 200, which can hide the root cause and confuse monitoring.

HTTP status codes in browsers and API responses, explained

4xx vs 5xx status codes at a glance

HTTP status codes are the standardized way a server tells a client what happened to a request. In HTTP Semantics (RFC 9110), the broad rule is simple:

• 4xx (Client Error): the server received the request, but the request is wrong or not allowed. Think invalid syntax (400), missing or invalid login (401), blocked permissions (403), missing resource (404), or rate limiting (429).
• 5xx (Server Error): the request might be fine, but the server (or an upstream dependency) failed to complete it. Common examples include application crashes (500), bad upstream/proxy responses (502), planned or unplanned downtime (503), and upstream timeouts (504).

For SEO and monitoring, this split matters because it changes your “first move.” With 4xx errors, you usually start by validating the URL, headers, auth, and request shape. With 5xx errors, you start by checking server health, deploys, logs, and upstream services.

Browser error pages vs real HTTP status codes

A browser’s “error page” is not always an HTTP status code. Some messages happen before HTTP even exists, like DNS failures, TLS certificate problems, blocked connections, or CORS issues. These can look like server errors to a user, but they will not show up as a 4xx/5xx response in your server logs because the request never completed.

The opposite problem is also common: a page displays an error message but still returns 200 OK. In SEO, this often becomes a “soft 404,” where the content looks like a missing page even though the server says it succeeded. Search engines rely heavily on status codes during crawling, and Google notes that it generally indexes pages served with HTTP 200 (success). See Google Search technical requirements.

For APIs, treat the status code as the primary truth, and the JSON body as the detail. If an API returns 200 with an error payload, clients, caches, and AI agents that call your endpoints can mis-handle failures or store bad responses.

Confirming the real status code with DevTools, curl, and logs

Using the Network tab to find the failing request

In the browser, the quickest way to stop guessing is the Chrome DevTools Network panel. Open DevTools, go to Network, reload the page, and click the request that actually failed (often an API call, not the HTML document).

A few checks save a lot of time:

• Filter by status: look for 4xx/5xx, then confirm which request triggered the visible error.
• Disable cache (while DevTools is open) and consider Preserve log so you can see redirects and retries.
• Confirm the request details: URL, method (GET vs POST), query string, and request headers (Authorization, Cookie, Content-Type).
• Confirm the response details: status code, response headers, and whether you were redirected (301/302/307/308) before the final error.
• If the browser shows a generic error page, verify whether the failing request is the main document, an XHR/fetch call, or a blocked preflight (OPTIONS) request.

Reproducing errors with curl and HTTP headers

Once you have the exact URL and headers, reproduce it from the command line. This removes extensions, cached assets, and UI noise, and makes it easier to share a minimal test case with your team.

Useful patterns (adjust the URL and headers to match what you saw in DevTools):

# Show status line + response headers
curl -i https://example.com/path

# Follow redirects and show headers for the final response
curl -iL https://example.com/path

# Headers only (HEAD request)
curl -I https://example.com/path

# Verbose: shows request + response headers (great for 401/403/429 debugging)
curl -v https://example.com/path

If you are testing an API, include the same auth and content type:

curl -i https://api.example.com/v1/items \
 -H "Authorization: Bearer <token>" \
 -H "Accept: application/json"

For deeper curl options and header behavior, the official curl HTTP scripting guide is a reliable reference.

Checking access and error logs quickly

DevTools and curl tell you what the client saw. Logs tell you what the server did.

Start with access logs to confirm the status code, request path, user agent, and timing. Then check error logs for stack traces, upstream connection failures, timeouts, or permission denials. When possible, match events by timestamp and a request ID / correlation ID header.

Two practical tips:

• If you are behind a CDN or reverse proxy, also check the proxy or load balancer logs. A “502” at the edge can hide a different upstream status.
• Be careful not to paste secrets from logs (API keys, cookies, bearer tokens) into tickets or chat.

Most common 4xx client errors and the fastest fixes

400 Bad Request

A 400 Bad Request means the server could not understand the request. It is often triggered by invalid syntax, malformed JSON, missing required parameters, or a header the server rejects. In practice, 400s show up most during form submissions and API calls.

Fast fixes to try:

• Validate the request payload: confirm your JSON is valid, fields match the API schema, and data types are correct.
• Check Content-Type and encoding: many APIs require Content-Type: application/json for JSON bodies. If you send form data or an empty body, you can get a 400.
• Confirm URL encoding: unescaped characters in query parameters can break parsing.
• Look for oversized headers/cookies: some servers respond with 400 when headers are too large (often caused by bloated cookies). Testing in an incognito window or clearing cookies can quickly confirm this.
• Compare DevTools vs curl: reproduce the request with curl and strip it down until it works. The last change usually reveals the problem.

For SEO, a 400 on a public page is uncommon but serious. It can block crawlers from accessing content and may indicate broken routing, bad redirects, or a WAF rule misfiring.

401 Unauthorized and WWW-Authenticate header

A 401 Unauthorized response means authentication is missing or invalid. The key detail is the WWW-Authenticate header, which tells the client what authentication scheme to use (for example, Basic, Bearer, or other challenge types). This behavior is defined in the HTTP authentication framework and is worth checking when debugging clients and APIs.

Fast fixes to try:

• Confirm you are sending credentials: Authorization header, session cookie, or signed request, depending on the system.
• Check token validity: expired JWTs, revoked API keys, wrong audience/issuer, or clock skew can all cause 401s.
• Verify the auth scope: some APIs return 401 when the token exists but lacks required scopes.
• Check redirects: auth gateways sometimes redirect and drop headers, leading to a 401 on the final hop.
• Use the WWW-Authenticate header as a clue: it may indicate “invalid_token,” required realm, or the expected scheme.

In browser-based apps, a 401 often means “your session is gone.” In APIs, it usually means “your client is not authenticated correctly.”

403 Forbidden and permission issues

A 403 Forbidden means the server understood the request but refuses to authorize it. Unlike 401, adding credentials may not help if the authenticated user still lacks permission, or if access is blocked by policy.

Common causes and quick fixes:

• Authorization rules: the user is logged in but lacks role-based access to the resource. Verify roles, group membership, and object-level permissions.
• IP, geo, or network restrictions: corporate allowlists, admin-only networks, or region blocks.
• WAF/CDN bot protection: automated traffic patterns, missing headers, or high request rates can trip rules and return 403.
• File and directory permissions: on servers, incorrect filesystem permissions or ownership can prevent access (especially after deployments).
• Hotlink and referer rules: common with images and static assets.

For SEO, unintended 403s can silently remove key pages from search visibility. If a page should be indexable, make sure it returns 200 for normal users and for Googlebot-like clients, and that security rules are targeted rather than blanket-blocking.

404 Not Found errors and how to fix missing pages

Broken links vs moved URLs vs deleted content

A 404 Not Found means the server is reachable, but the specific URL does not map to a resource anymore. The fastest fix depends on why the URL is missing:

• Broken link (typo or outdated internal link): fix the link at the source. Update navigation, templates, in-content links, and any hard-coded URLs in your app.
• Moved URL (new slug, new folder, migration): add a 301 redirect from the old URL to the most relevant new URL. Then update internal links so you are not relying on redirects for normal navigation.
• Deleted content (no close replacement): return a real 404 (or 410 Gone if it is permanently removed). This is often the cleanest signal for search engines and avoids “soft 404” issues where a missing page looks like an error but returns 200.

For SEO, avoid “redirect everything to the homepage.” If the replacement is not a close match, search engines may treat it as a soft 404 and ignore the redirect signal.

Redirects, canonical URLs, and trailing slash mismatches

Many 404s are self-inflicted URL consistency problems. Common culprits include:

• Trailing slash vs no trailing slash (/page vs /page/)
• Uppercase vs lowercase (/Blog vs /blog)
• HTTP vs HTTPS and www vs non-www
• Query parameters used as if they were unique pages

Pick one canonical format and enforce it consistently. Use 301 redirects for the non-canonical versions, and ensure your canonical tags, sitemaps, and internal links all point to the same preferred URL. This matters even more when multiple crawlers are involved, including AI-driven bots and link preview systems that may surface stale URL variants.

Custom 404 pages that help users recover

A good custom 404 page improves UX without hiding the error. Key requirements:

• Return an actual 404 status code (do not return 200 with a “not found” template).
• Offer clear next steps: a site search, top categories, and a link back to the homepage.
• If you can reliably detect intent, show suggested matches (for example, similar products or a closest category), but avoid auto-redirecting people to irrelevant pages.

Keep the 404 page lightweight and fast. If your 404 template triggers extra API calls that fail, you can end up masking the original issue and creating noisy monitoring signals.

429 Too Many Requests errors from rate limits and WAFs

Retry-After header and backoff behavior

A 429 Too Many Requests response means the server (or an edge layer like a CDN/WAF) is throttling a client for sending requests too quickly. It is not a “broken server” in the 5xx sense. It is a protection mechanism.

When a 429 happens, check for the Retry-After response header. It tells the client how long to wait before trying again, either as a number of seconds or as an HTTP date. The safest client behavior is:

• Pause for the Retry-After duration (if present).
• Retry with exponential backoff plus a bit of random jitter.
• Reduce concurrency (too many parallel requests can trigger 429 even if your average rate looks fine).

If you own the API, returning 429 without guidance forces clients to guess, which usually makes bursts worse. The MDN reference for 429 Too Many Requests is a solid baseline for expected behavior.

API keys, quotas, and bot protection triggers

429s commonly come from API quotas (per key, per user, per IP, per route) and WAF/bot rules (login brute force protection, scraping defenses, or generic anomaly scoring). Look for patterns like one endpoint being hit in a tight loop, retries that ignore backoff, or a single shared IP (NAT, serverless egress) representing many users.

For SEO, 429s can also affect crawling. If Googlebot is consistently rate-limited, it can slow or pause crawling, which delays updates and can reduce discovery of new URLs. Google’s Crawl Stats documentation notes that extended periods of 429/5xx responses can lead to reduced crawling over time. See the Crawl Stats report.

Reducing bursts with caching and batching

The fastest way to eliminate 429s is to reduce unnecessary request volume:

Use caching for repeat reads (CDN caching for public pages, application caching for expensive API calls). Batch or debounce client requests (search-as-you-type, autosave, analytics beacons). For AI integrations and agent traffic, add clear rate limit headers, provide bulk endpoints where possible, and document safe retry rules so automated clients do not accidentally DDoS you with “helpful” retries.

Most common 5xx server errors and what usually causes them

500 Internal Server Error

A 500 Internal Server Error is the catch-all when your application or server fails in an unexpected way. The request reached your origin, but something broke while handling it.

Common causes include unhandled exceptions, bad deploys, misconfigured environment variables, database connection failures, template rendering errors, and permission issues on the server (for example, the app cannot read a required file).

Fast triage usually looks like this: confirm the exact failing URL and method, check the error log for a stack trace at the same timestamp, and compare against recent changes (deploys, config updates, feature flags). If only some pages fail, look for a shared dependency, like a specific database query or an external API call.

502 Bad Gateway

A 502 Bad Gateway usually means a gateway, reverse proxy, or CDN could not get a valid response from an upstream server. You will often see 502s in stacks with a load balancer in front of Nginx/Apache, or an API gateway in front of microservices.

Typical causes: the upstream process crashed, the upstream returned an invalid HTTP response, TLS handshake issues between proxy and origin, DNS problems, or the origin closing connections early.

Fast checks: confirm whether the 502 is generated by the edge (CDN) or your own proxy, then test the upstream directly (bypassing the proxy) to see if it returns a clean 200/4xx/5xx.

503 Service Unavailable

A 503 Service Unavailable means the server is temporarily unable to handle requests. It is commonly used for maintenance windows, restarts, and overload situations (CPU, memory, connection pool exhaustion).

If you expect a short outage, 503 is often a better signal than a “friendly” 200 error page. For SEO, persistent 503s can reduce crawl activity and, if they continue long enough, can lead to URLs being dropped from the index. Google documents how it slows crawling on 5xx responses and may eventually drop persistently failing URLs in its crawling guidance on how HTTP status codes affect Google’s crawlers.

504 Gateway Timeout

A 504 Gateway Timeout happens when a gateway or proxy did not receive a response from the upstream server in time. The upstream might be slow, stuck, or overloaded, even if it eventually finishes the work.

Common causes: slow database queries, long-running background work happening in the web request path, upstream thread starvation, or timeouts that are too aggressive at the proxy/CDN layer.

Fast checks: identify which timeout fired (CDN, load balancer, reverse proxy, app server, database). Then decide whether to speed up the work (indexes, caching, queueing) or adjust timeouts where it is safe.

Quick decision tree for 502 vs 503 vs 504

• Did the gateway get a response at all?
• No, it timed out waiting: 504
• Yes, but the response was invalid or the connection was broken: 502
• Is the origin intentionally refusing traffic right now (overload, restart, maintenance)?
• Yes: 503
• Does it only happen through a proxy/CDN, but not when hitting the origin directly?
• Often 502/504 at the edge, with the root cause in upstream health, networking, or timeouts.

In 2026, these distinctions also matter for AI-driven traffic. Automated agents tend to retry aggressively. Clear 503 maintenance behavior and predictable timeouts help avoid retry storms that turn a small incident into sustained 5xx errors.

Escalation paths and preventing repeat HTTP errors

Debugging through CDNs, reverse proxies, and load balancers

When a site sits behind a CDN, reverse proxy, or load balancer, the status code you see might be generated by the edge, not your app. Start by identifying the layer that produced the response:

• Check response headers that indicate an edge path (for example, a CDN header) and whether the request reached origin.
• Compare results from different vantage points (your browser, a server in the same region as the origin, and a direct origin test if it is safe).
• Correlate requests across layers using request IDs. If you use Cloudflare, the Cloudflare Ray ID is designed for this kind of tracing.

If the error happens only through the CDN, focus on WAF rules, caching configuration, origin health checks, TLS settings between edge and origin, and upstream timeouts. If it happens both through the CDN and directly on origin, focus on the application and its dependencies.

When to contact hosting or app support

Escalate when you have a reproducible request and a clear symptom, but the fix is outside your control or requires privileged access. Good triggers include:

• Infrastructure-level failures: load balancer target health flapping, repeated instance restarts, network errors, or storage saturation.
• Managed services: database timeouts, queue backlogs, cache cluster instability, or provider-side incidents.
• Edge security blocks: false positives from bot protection or DDoS mitigation that you cannot safely tune alone.

When you contact support, include exact timestamps (with timezone), failing URLs, client IP ranges (if relevant), request IDs, and a curl reproduction. This shortens back-and-forth and helps support find the same event in their logs.

Monitoring and alerting to catch errors early

Preventing repeat HTTP errors is mostly about fast detection and clean signals. Track error rate by endpoint, latency percentiles, and dependency health. Alert on sudden changes, not just absolute thresholds, and separate 4xx from 5xx so you do not page the on-call for user typos.

For SEO, monitor crawlability alongside uptime. A spike in 5xx or persistent 404s can slow discovery and indexing. The Crawl Stats report helps you spot sustained server errors and crawling drops that correlate with incidents.

In an AI-driven traffic world, also watch for automated clients that retry aggressively. Publish consistent error bodies, honor Retry-After on throttling, and rate-limit per token or agent identity where possible. This reduces “retry storms” that can turn a small outage into prolonged 5xx.