ScrepyDocs

Redirect Waste

Learn how to analyze repeated crawler requests to redirected URLs, remove chains and internal hops, and preserve necessary redirects safely.

Redirect Waste identifies URL paths repeatedly requested by crawlers that return a redirect. Screpy reports the path, redirect status, and redirected-request volume so you can find old URLs still being generated or referenced.

Redirects are necessary when content moves. Waste occurs when the site itself repeatedly sends users and crawlers through avoidable hops instead of linking to the final destination.

Understand redirect behavior

StatusTypical meaning
301 / 308The resource moved permanently
302 / 307The move is temporary
Meta refresh / JavaScript redirectClient-side or page-level redirection; usually less direct than server-side HTTP redirects

Use the status that matches the real change. Google's redirect guidance recommends permanent server-side redirects for permanent moves and temporary redirects for temporary changes.

Find the source of repeated redirects

Check:

  • navigation and footer links;
  • contextual links inside content;
  • XML sitemaps;
  • canonical and hreflang annotations;
  • structured data;
  • image, script, and stylesheet URLs;
  • campaign templates and email links;
  • HTTP-to-HTTPS and hostname normalization;
  • external backlinks and old bookmarks.

Internal sources should normally point directly to the final canonical URL. External sources may continue using old URLs, which is why necessary redirects should remain available.

Analyze redirect waste step by step

  1. Sort by redirected-request volume.
  2. Follow the complete redirect path to the final response.
  3. Confirm the final destination is relevant, successful, indexable, and canonical.
  4. Determine whether the redirect is permanent, temporary, or accidental.
  5. Find internal references to every intermediate URL.
  6. Update the source links and annotations to the final URL.
  7. Keep required redirects for external traffic and historical URLs.

Redirect chains and loops

A chain sends A → B → C when A → C would be sufficient. Chains add latency, increase failure points, and consume crawler requests. A loop never reaches a final page and must be repaired immediately.

During migrations, create a direct one-to-one mapping where possible. Avoid redirecting every removed page to the homepage; irrelevant destinations confuse users and can be treated as soft errors.

Example

An old blog path redirects from HTTP to HTTPS, then from www to the preferred hostname, then from an old slug to a new slug. The site navigation still uses the HTTP old slug.

Keep a direct permanent redirect from the historical URL to the final page, but update navigation, sitemaps, canonicals, and internal content links to the final HTTPS canonical URL. This preserves external value while removing internal waste.

When not to remove a redirect

  • The old URL has external links or direct traffic.
  • Users may have bookmarks.
  • A permanent migration still needs continuity.
  • A short-term campaign or test genuinely requires a temporary destination.
  • Hostname and protocol normalization protect one canonical URL.

The objective is not “zero redirects.” It is zero unnecessary internally generated redirects and no broken redirect paths.

Verify the cleanup

Run a new crawl and confirm internal links resolve directly, chains are removed, and all final destinations return the intended status. Compare fresh redirected-request volume over a similar period. Residual external requests can be normal even after internal cleanup.

On this page