Crawl

Crawl websites.

client.browserRendering.crawl.create(, ?): CrawlCreateResponse

POST/accounts/{account_id}/browser-rendering/crawl

Get crawl result.

client.browserRendering.crawl.get(, , ?): CrawlGetResponse { id, browserSecondsUsed, finished, 5 more }

GET/accounts/{account_id}/browser-rendering/crawl/{job_id}

Cancel a crawl job.

client.browserRendering.crawl.delete(, , ?): CrawlDeleteResponse { job_id, message }

DELETE/accounts/{account_id}/browser-rendering/crawl/{job_id}

ModelsExpand Collapse

CrawlCreateResponse = string

Crawl job ID.

CrawlGetResponse { id, browserSecondsUsed, finished, 5 more }

id: string

Crawl job ID.

browserSecondsUsed: number

Total seconds spent in browser so far.

finished: number

Total number of URLs that have been crawled so far.

records: Array<Record>

List of crawl job records.

metadata: Metadata { status, url, title }

status: number

HTTP status code of the crawled page.

url: string

Final URL of the crawled page.

title?: string

Title of the crawled page.

status: "queued" | "errored" | "completed" | 3 more

Current status of the crawled URL.

One of the following:

"queued"

"errored"

"completed"

"disallowed"

"skipped"

"cancelled"

url: string

Crawled URL.

html?: string

HTML content of the crawled URL.

json?: Record<string, unknown>

JSON of the content of the crawled URL.

markdown?: string

Markdown of the content of the crawled URL.

skipped: number

Total number of URLs that were skipped due to include/exclude/subdomain filters. Skipped URLs are included in records but are not counted toward total/finished.

status: string

Current crawl job status.

total: number

Total current number of URLs in the crawl job.

cursor?: string

Cursor for pagination.

CrawlDeleteResponse { job_id, message }

job_id: string

The ID of the cancelled job.

message: string

Cancellation confirmation message.