Crawls
Start website crawls and retrieve crawl status, progress, and completed audit summaries with the Screpy REST API.
Treat crawl status as part of the workflow
Start a crawl only after you have selected the right project and checked the current account allowance. Then poll the crawl status until it is complete before relying on its summary or using that crawl in a comparison.
A crawl that is still running is not a final technical baseline. Keep the crawl UID returned by the API so later crawl-data requests read the matching result set.
/projects/{project_uid}/crawlsAuthorization
bearerApiKey Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.
In: header
Path Parameters
Accessible project UID. Discover it with GET /projects; do not guess it.
^[a-z0-9]{10}$Query Parameters
Maximum records to return. Defaults to 50.
1 <= value <= 10050Opaque cursor returned by the prior response. Do not construct or alter it.
length <= 512Response Body
application/json
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/projects/string/crawls"{ "data": [ { "uid": "string", "status": "string", "target_url": "http://example.com", "max_pages": 0, "depth": 1, "health_score": 0, "seo_score": 0, "pages_total": 0, "critical_issues": 0, "warnings": 0, "notices": 0, "queued_at": "2019-08-24T14:15:22Z", "started_at": "2019-08-24T14:15:22Z", "finished_at": "2019-08-24T14:15:22Z", "created_at": "2019-08-24T14:15:22Z" } ], "links": { "first": "http://example.com", "last": "http://example.com", "prev": "http://example.com", "next": "http://example.com" }, "meta": { "path": "http://example.com", "per_page": 0, "next_cursor": "string", "prev_cursor": "string" }}{ "message": "Unauthenticated."}{ "message": "string", "code": "string"}{ "message": "Project not found or not accessible."}{ "message": "string", "code": "string", "errors": { "property1": [ "string" ], "property2": [ "string" ] }}{ "message": "string", "code": "string"}/projects/{project_uid}/crawlsStarts a crawl with the project's current settings. It may consume the account's crawl allowance.
Authorization
bearerApiKey Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.
In: header
Path Parameters
Accessible project UID. Discover it with GET /projects; do not guess it.
^[a-z0-9]{10}$Response Body
application/json
application/json
application/json
application/json
application/json
application/json
application/json
curl -X POST "https://example.com/projects/string/crawls"{ "data": { "uid": "string", "status": "string", "target_url": "http://example.com", "max_pages": 0, "depth": 1, "health_score": 0, "seo_score": 0, "pages_total": 0, "critical_issues": 0, "warnings": 0, "notices": 0, "queued_at": "2019-08-24T14:15:22Z", "started_at": "2019-08-24T14:15:22Z", "finished_at": "2019-08-24T14:15:22Z", "created_at": "2019-08-24T14:15:22Z" }}{ "message": "Unauthenticated."}{ "message": "string", "code": "string"}{ "message": "Project not found or not accessible."}{ "message": "string", "code": "string"}{ "message": "string", "code": "string"}{ "message": "string", "code": "string"}/projects/{project_uid}/crawls/{crawl_uid}Authorization
bearerApiKey Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.
In: header
Path Parameters
Accessible project UID. Discover it with GET /projects; do not guess it.
^[a-z0-9]{10}$Crawl UID belonging to the selected project. Discover it with GET /projects/{project_uid}/crawls.
^[a-z0-9]{16}$Response Body
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/projects/string/crawls/string"{ "data": { "crawl_uid": "string", "status": "string", "state": "string", "status_label": "string", "started_at": "2019-08-24T14:15:22Z", "finished_at": "2019-08-24T14:15:22Z", "last_synced_at": "2019-08-24T14:15:22Z", "progress": { "pages_crawled": 0, "pages_total": 0, "max_pages": 0, "current_depth": 0 } }}{ "message": "Unauthenticated."}{ "message": "string", "code": "string"}{ "message": "Project not found or not accessible."}{ "message": "string", "code": "string"}/projects/{project_uid}/crawls/{crawl_uid}/summaryAuthorization
bearerApiKey Create a REST API key in Settings > Connected Apps. MCP credentials do not authenticate this API.
In: header
Path Parameters
Accessible project UID. Discover it with GET /projects; do not guess it.
^[a-z0-9]{10}$Crawl UID belonging to the selected project. Discover it with GET /projects/{project_uid}/crawls.
^[a-z0-9]{16}$Response Body
application/json
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/projects/string/crawls/string/summary"{ "data": { "crawl_uid": "string", "status": "string", "state": "string", "status_label": "string", "started_at": "2019-08-24T14:15:22Z", "finished_at": "2019-08-24T14:15:22Z", "last_synced_at": "2019-08-24T14:15:22Z", "progress": { "pages_crawled": 0, "pages_total": 0, "max_pages": 0, "current_depth": 0 }, "health": { "score": 0, "confidence": 0, "coverage": 0 }, "issues": { "critical": 0, "warnings": 0, "notices": 0, "total": 0 }, "totals": { "pages": 0, "indexable_pages": 0, "noindex_pages": 0, "links": 0, "images": 0, "unique_urls": 0 } }}{ "message": "Unauthenticated."}{ "message": "string", "code": "string"}{ "message": "Project not found or not accessible."}{ "message": "string", "code": "string"}{ "message": "string", "code": "string"}