ScrepyDocs

List crawled pages

Lists pages from a completed crawl. Send filters as indexed PHP-style query arrays, such as `filters[0][field]=status_code&filters[0][operator]=eq&filters[0][value]=440`.

GET/projects/{project_uid}/crawls/{crawl_uid}/pages

Lists pages from a completed crawl. Send filters as indexed PHP-style query arrays, such as filters[0][field]=status_code&filters[0][operator]=eq&filters[0][value]=440.

Authorization

bearerApiKey
AuthorizationBearer <token>

Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.

In: header

Path Parameters

project_uid*string

Accessible project UID. Discover it with GET /projects; do not guess it.

Match^[a-z0-9]{10}$
crawl_uid*string

Crawl UID belonging to the selected project. Discover it with GET /projects/{project_uid}/crawls.

Match^[a-z0-9]{16}$

Query Parameters

filters?

Up to 20 indexed PHP-style filter objects. Example: filters[0][field]=status_code&filters[0][operator]=eq&filters[0][value]=440.

Propertiesproperties <= 20
limit?integer

Maximum records to return. Defaults to 50.

Range1 <= value <= 100
Default50
cursor?string

Opaque cursor returned by the prior response. Do not construct or alter it.

Lengthlength <= 512

Response Body

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

curl -X GET "https://example.com/projects/string/crawls/string/pages"
{  "data": [    {      "url": "http://example.com",      "final_url": "http://example.com",      "domain": "string",      "status_code": 0,      "content_type": "string",      "title": "string",      "title_length": 0,      "description": "string",      "description_length": 0,      "h1": "string",      "h1_count": 0,      "h2_count": 0,      "words_count": 0,      "content_ratio": 0,      "depth": 0,      "response_time_ms": 0,      "response_bytes": 0,      "canonical": "string",      "canonical_present": true,      "canonical_matches_final_url": true,      "json_ld_count": 0,      "og_required_tags_present": true,      "twitter_card_present": true,      "total_links_count": 0,      "external_links_count": 0,      "internal_links_count": 0,      "total_images_count": 0,      "missing_alt_images_count": 0,      "fetched_at": "2019-08-24T14:15:22Z"    }  ],  "links": {    "next": "http://example.com"  },  "meta": {    "limit": 0,    "next_cursor": "string"  }}