Spider

aid: spider-cloud
name: Spider
description: Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back
  clean markdown, structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass,
  residential proxies, headless browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.
kind: contract
image: https://kinlane-productions2.s3.amazonaws.com/apis-json/apis-json-logo.jpg
access: 3rd-Party
created: '2026-05-25'
modified: '2026-05-25'
position: Consuming
tags:
  - Crawling
  - Scraping
  - Data Extraction
  - URLs
  - AI
  - Markdown
  - MCP
  - Rust
  - Headless Browser
  - Proxies
url: https://raw.githubusercontent.com/api-evangelist/spider-cloud/refs/heads/main/apis.yml
specificationVersion: '0.19'
apis:
  - aid: spider-cloud:spider-api
    name: Spider API
    description: The Spider Cloud REST API exposes core /crawl, /scrape, /search, /links, /screenshot, /unblocker,
      and /transform endpoints, plus AI-discovered per-domain /fetch endpoints, residential/ISP/mobile proxy
      routing via proxy.spider.cloud, CDP/WebDriver BiDi cloud browsers, scraper directory lookups, crawl logs,
      and a credits endpoint. All endpoints accept and return JSON (also XML, CSV, JSONL via content-type) and
      authenticate with Bearer API keys. Up to 10,000 RPM per account.
    humanURL: https://spider.cloud/docs/api
    baseURL: https://api.spider.cloud
    tags:
      - Crawling
      - Scraping
      - Data Extraction
      - URLs
      - AI
      - Markdown
      - Search
      - Screenshot
      - Headless Browser
      - Proxies
    properties:
      - type: Documentation
        url: https://spider.cloud/docs/api
      - type: Documentation
        url: https://spider.cloud/docs/overview
      - type: OpenAPI
        url: openapi/spider-cloud-openapi.yml
      - type: Authentication
        url: https://spider.cloud/docs/api
      - type: RateLimits
        url: https://spider.cloud/docs/api
      - type: NaftikoCapability
        url: capabilities/spider-cloud-crawling.yaml
      - type: NaftikoCapability
        url: capabilities/spider-cloud-scraping.yaml
      - type: NaftikoCapability
        url: capabilities/spider-cloud-search.yaml
      - type: NaftikoCapability
        url: capabilities/spider-cloud-browser.yaml
      - type: NaftikoCapability
        url: capabilities/spider-cloud-data.yaml
  - aid: spider-cloud:spider-mcp
    name: Spider Cloud MCP Server
    description: Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations
      (crawl, scrape, search, links, screenshot, unblocker, transform, get_credits), five AI-routed variants,
      and nine browser-automation tools (open, navigate, click, fill, screenshot, content, evaluate, wait_for,
      close) — to any MCP-compatible client. Authenticated with a Bearer API key. Also runnable locally via
      `npx spider-cloud-mcp`.
    humanURL: https://spider.cloud/docs/integrations/mcp
    baseURL: https://mcp.spider.cloud/mcp
    tags:
      - MCP
      - AI
      - Agents
      - Crawling
      - Scraping
      - Browser Automation
    properties:
      - type: Documentation
        url: https://spider.cloud/docs/integrations/mcp
      - type: SourceCode
        url: https://github.com/spider-rs/spider-cloud-mcp-v2
      - type: Authentication
        url: https://spider.cloud/docs/integrations/mcp
common:
  - type: Website
    url: https://spider.cloud
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: APIReference
    url: https://spider.cloud/docs/api
  - type: Guides
    url: https://spider.cloud/guides
  - type: Pricing
    url: https://spider.cloud/pricing
  - type: StatusPage
    url: https://spidercloud.statuspage.io
  - type: GitHubOrganization
    url: https://github.com/spider-rs
  - type: SourceCode
    url: https://github.com/spider-rs/spider
  - type: SDKs
    url: https://github.com/spider-rs/spider-clients
  - type: SDK
    url: https://github.com/spider-rs/spider-py
  - type: SDK
    url: https://github.com/spider-rs/spider-nodejs
  - type: SDK
    url: https://docs.rs/spider
  - type: MCP
    url: https://mcp.spider.cloud/mcp
  - type: Integrations
    url: https://spider.cloud/docs/integrations/mcp
  - type: Integration
    url: https://spider.cloud/docs/integrations/langchain
  - type: Integration
    url: https://spider.cloud/docs/integrations/llamaindex
  - type: Integration
    url: https://spider.cloud/docs/integrations/crewai
  - type: Integration
    url: https://spider.cloud/docs/integrations/flowiseai
  - type: Integration
    url: https://spider.cloud/docs/integrations/agno
  - type: Integration
    url: https://spider.cloud/docs/integrations/zapier
  - type: Integration
    url: https://spider.cloud/docs/integrations/x402
  - type: Support
    url: mailto:[email protected]
  - type: Plans
    url: plans/spider-cloud-plans-pricing.yml
  - type: RateLimits
    url: rate-limits/spider-cloud-rate-limits.yml
  - type: FinOps
    url: finops/spider-cloud-finops.yml
maintainers:
  - FN: Kin Lane
    email: [email protected]