Skip to content
Cloudflare API
Start here
API Reference
Radar

Robots TXT

Robots TXTTop

Get top domain categories by robots.txt files parsed
radar.robots_txt.top.domain_categories(TopDomainCategoriesParams**kwargs) -> TopDomainCategoriesResponse
GET/radar/robots_txt/top/domain_categories
ModelsExpand Collapse
class TopDomainCategoriesResponse: …
meta: Meta

Metadata for the results.

confidence_info: Optional[MetaConfidenceInfo]
annotations: List[MetaConfidenceInfoAnnotation]
data_source: Literal["ALL", "AI_BOTS", "AI_GATEWAY", 22 more]

Data source for annotations.

One of the following:
"ALL"
"AI_BOTS"
"AI_GATEWAY"
"BGP"
"BOTS"
"CONNECTION_ANOMALY"
"CT"
"DNS"
"DNS_MAGNITUDE"
"DNS_AS112"
"DOS"
"EMAIL_ROUTING"
"EMAIL_SECURITY"
"FW"
"FW_PG"
"HTTP"
"HTTP_CONTROL"
"HTTP_CRAWLER_REFERER"
"HTTP_ORIGINS"
"IQI"
"LEAKED_CREDENTIALS"
"NET"
"ROBOTS_TXT"
"SPEED"
"WORKERS_AI"
description: str
end_date: datetime
formatdate-time
event_type: Literal["EVENT", "GENERAL", "OUTAGE", 3 more]

Event type for annotations.

One of the following:
"EVENT"
"GENERAL"
"OUTAGE"
"PARTIAL_PROJECTION"
"PIPELINE"
"TRAFFIC_ANOMALY"
is_instantaneous: bool

Whether event is a single point in time or a time range.

linked_url: str
formaturi
start_date: datetime
formatdate-time
level: int

Provides an indication of how much confidence Cloudflare has in the data.

date_range: List[MetaDateRange]
end_time: datetime

Adjusted end of date range.

formatdate-time
start_time: datetime

Adjusted start of date range.

formatdate-time
last_updated: datetime

Timestamp of the last dataset update.

formatdate-time
normalization: Literal["PERCENTAGE", "MIN0_MAX", "MIN_MAX", 5 more]

Normalization method applied to the results. Refer to Normalization methods.

One of the following:
"PERCENTAGE"
"MIN0_MAX"
"MIN_MAX"
"RAW_VALUES"
"PERCENTAGE_CHANGE"
"ROLLING_AVERAGE"
"OVERLAPPED_PERCENTAGE"
"RATIO"
units: List[MetaUnit]

Measurement units for the results.

name: str
value: str
top_0: List[Top0]
name: str
value: int

Robots TXTTopUser Agents

Get top user agents on robots.txt files
radar.robots_txt.top.user_agents.directive(UserAgentDirectiveParams**kwargs) -> UserAgentDirectiveResponse
GET/radar/robots_txt/top/user_agents/directive
ModelsExpand Collapse
class UserAgentDirectiveResponse: …
meta: Meta

Metadata for the results.

confidence_info: Optional[MetaConfidenceInfo]
annotations: List[MetaConfidenceInfoAnnotation]
data_source: Literal["ALL", "AI_BOTS", "AI_GATEWAY", 22 more]

Data source for annotations.

One of the following:
"ALL"
"AI_BOTS"
"AI_GATEWAY"
"BGP"
"BOTS"
"CONNECTION_ANOMALY"
"CT"
"DNS"
"DNS_MAGNITUDE"
"DNS_AS112"
"DOS"
"EMAIL_ROUTING"
"EMAIL_SECURITY"
"FW"
"FW_PG"
"HTTP"
"HTTP_CONTROL"
"HTTP_CRAWLER_REFERER"
"HTTP_ORIGINS"
"IQI"
"LEAKED_CREDENTIALS"
"NET"
"ROBOTS_TXT"
"SPEED"
"WORKERS_AI"
description: str
end_date: datetime
formatdate-time
event_type: Literal["EVENT", "GENERAL", "OUTAGE", 3 more]

Event type for annotations.

One of the following:
"EVENT"
"GENERAL"
"OUTAGE"
"PARTIAL_PROJECTION"
"PIPELINE"
"TRAFFIC_ANOMALY"
is_instantaneous: bool

Whether event is a single point in time or a time range.

linked_url: str
formaturi
start_date: datetime
formatdate-time
level: int

Provides an indication of how much confidence Cloudflare has in the data.

date_range: List[MetaDateRange]
end_time: datetime

Adjusted end of date range.

formatdate-time
start_time: datetime

Adjusted start of date range.

formatdate-time
last_updated: datetime

Timestamp of the last dataset update.

formatdate-time
normalization: Literal["PERCENTAGE", "MIN0_MAX", "MIN_MAX", 5 more]

Normalization method applied to the results. Refer to Normalization methods.

One of the following:
"PERCENTAGE"
"MIN0_MAX"
"MIN_MAX"
"RAW_VALUES"
"PERCENTAGE_CHANGE"
"ROLLING_AVERAGE"
"OVERLAPPED_PERCENTAGE"
"RATIO"
units: List[MetaUnit]

Measurement units for the results.

name: str
value: str
top_0: List[Top0]
name: str
value: int
fully: Optional[int]
partially: Optional[int]