> ## Documentation Index > Fetch the complete documentation index at: https://docs.hydrafetch.com/llms.txt > Use this file to discover all available pages before exploring further. # Search the web > Search the web for a query and get back ranked results. By default each result comes back with its page fetched and returned as clean data; set `scrapeResults` to false for just the ranked titles, urls, and snippets. ## OpenAPI ````yaml https://api.hydrafetch.com/openapi.json post /v1/web/search openapi: 3.0.0 info: title: Hydrafetch API description: >- Hydrafetch turns any URL into clean, LLM-ready data through one API. Give us a link and get back Markdown, the page's own structured data, extracted JSON, links, a summary, or a screenshot. Point us at a whole site and get every page. Ask a question and get answers with per-field confidence and the passage each value came from. You describe the outcome you want — the pipeline decides how to get it. ## Authentication Every request is authenticated with your API key in the `X-API-Key` header. Keys are scoped to a workspace and carry its credit balance. ## Credits Calls are billed in credits and charged only on success. A standard scrape is one credit; richer formats and the extraction tier cost more. Each response reports what it consumed. ## Conventions All timestamps are UTC ISO 8601. Long-running jobs (crawl, batch) return a job id you poll, or a webhook you register. Errors return a structured body with a stable `code` and an HTTP status. version: '1.0' contact: name: Hydrafetch url: https://hydrafetch.com email: team@hydrafetch.com servers: - url: https://api.hydrafetch.com description: Production security: - apiKey: [] tags: - name: Web Scraping description: >- Turn URLs into clean, LLM-ready content: scrape a single page, crawl or batch-scrape a whole site, map its URLs, search the web, and capture screenshots or images. - name: Web Extraction description: >- Pull schema-shaped JSON out of one or many pages, with optional per-field confidence and the source passage behind each value. paths: /v1/web/search: post: tags: - Web Scraping summary: Search the web description: >- Search the web for a query and get back ranked results. By default each result comes back with its page fetched and returned as clean data; set `scrapeResults` to false for just the ranked titles, urls, and snippets. operationId: run parameters: [] requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SearchRequestDto' responses: '200': description: The ranked search results. content: application/json: schema: $ref: '#/components/schemas/SearchResponseDto' components: schemas: SearchRequestDto: type: object properties: query: type: string maxLength: 500 example: best open source vector databases description: What to search the web for. limit: type: number minimum: 1 maximum: 15 example: 5 description: How many ranked results to return. Omit for the default. scrapeResults: type: boolean example: true description: >- When true (default), each result comes back with its page fetched and returned as clean data. When false, you get just the ranked title, url, and snippet. scrapeOptions: $ref: '#/components/schemas/ScrapeOptionsDto' engine: type: string enum: - duckduckgo - brave example: brave description: >- Pin the search engine to use. Omit to let us pick the best available one and fall back automatically if it is unavailable — recommended unless you need results from one specific index. timeRange: type: string enum: - day - week - month - year example: week description: Restrict results to a recency window relative to now. country: type: string example: us description: ISO 3166 alpha-2 country code to bias results toward a region. includeDomains: maxItems: 15 example: - github.com - arxiv.org description: Only return results from these domains. type: array items: type: string excludeDomains: maxItems: 15 example: - pinterest.com description: Drop results from these domains. type: array items: type: string required: - query SearchResponseDto: type: object properties: data: $ref: '#/components/schemas/SearchResultDto' required: - data ScrapeOptionsDto: type: object properties: formats: type: array example: - markdown description: What to return per page. Omit for Markdown only. items: type: string enum: - markdown - html - rawHtml - links - structured onlyMainContent: type: boolean description: Return only the main content, dropping boilerplate. Default true. includeTags: maxItems: 50 description: CSS selectors to keep. type: array items: type: string excludeTags: maxItems: 50 description: CSS selectors to strip before extraction. type: array items: type: string removeBase64Images: type: boolean description: Strip inline base64 images. Default true. blockAds: type: boolean description: Remove common ad and tracking elements. Default true. includeLinks: type: boolean description: >- Keep inline links in the markdown. Default false — dropping them is what keeps our output dense. Turn on for reference, API-docs and code pages where cross-references are content. renderJs: type: boolean description: Force full page rendering for JavaScript-heavy pages. waitFor: type: number minimum: 0 maximum: 30000 description: Extra milliseconds to let the page settle before capture. timeout: type: number minimum: 1000 maximum: 120000 description: Per-page time budget, in milliseconds. location: $ref: '#/components/schemas/OptLocationDto' headers: type: object additionalProperties: type: string description: Extra request headers to send when fetching each page. preferStructure: type: boolean description: Preserve document structure over prose density. Default off. maxAge: type: number minimum: 0 maximum: 604800000 description: Serve a page from cache if younger than this many milliseconds. SearchResultDto: type: object properties: query: type: string example: best open source vector databases description: The query you searched. engine: type: string enum: - duckduckgo - brave example: duckduckgo description: >- The search engine that answered, which may differ from the one you asked for. status: type: string enum: - ok - no_results example: ok description: >- Whether the engine found anything: `ok` when it returned results, `no_results` when it answered but the web had no match. An empty `results` with `no_results` is a real miss — if no engine could answer at all, the request fails with 503 rather than returning an empty list. results: description: The ranked results. type: array items: $ref: '#/components/schemas/SearchResultItemDto' required: - query - engine - status - results OptLocationDto: type: object properties: country: type: string example: us description: ISO 3166 alpha-2 country to fetch as if from. languages: example: - en-US - en description: Preferred content languages, most-preferred first. type: array items: type: string SearchResultItemDto: type: object properties: title: type: string example: Best open source vector databases description: The result title. url: type: string example: https://example.com/vector-databases description: The result URL. snippet: type: string example: >- A rundown of the leading open source vector databases and how they compare. description: A short snippet describing the result. rank: type: number example: 1 description: The result's position in the ranked list. data: nullable: true description: >- The scraped page for this result, or null when scraping is off or unavailable. type: object allOf: - $ref: '#/components/schemas/WebScrapeDataDto' required: - title - url - snippet - rank - data WebScrapeDataDto: type: object properties: url: type: string example: https://example.com description: The URL you requested. finalUrl: type: string example: https://example.com/ description: The final URL after any redirects. redirected: type: boolean example: false description: >- Whether the origin redirected: true when `finalUrl` differs from the URL you requested. Explicit so a page with no redirect is distinguishable from one whose redirect was not tracked. status: type: number example: 200 description: HTTP status of the fetched page. cached: type: boolean example: false description: Whether this result was served from cache. warning: type: string description: Set when the page was returned with a caveat (e.g. partial content). metadata: $ref: '#/components/schemas/WebScrapeMetadataDto' usage: $ref: '#/components/schemas/WebUsageDto' quality: description: Per-page extraction quality signals. allOf: - $ref: '#/components/schemas/WebQualityDto' markdown: type: string description: >- Clean Markdown of the main content. Returned when `markdown` is requested. html: type: string description: Cleaned main-content HTML. Returned when `html` is requested. rawHtml: type: string description: The unmodified page HTML. Returned when `rawHtml` is requested. links: description: Returned when `links` is requested. allOf: - $ref: '#/components/schemas/WebLinksDto' structured: description: Returned when `structured` is requested. allOf: - $ref: '#/components/schemas/WebStructuredDataDto' summary: type: string description: >- A concise factual summary. Returned when `summary` is requested (LLM-backed). json: type: object additionalProperties: true description: Schema-shaped JSON. Returned when `json` is requested (LLM-backed). required: - url - finalUrl - redirected - status - cached - metadata WebScrapeMetadataDto: type: object properties: title: type: object nullable: true example: Example Domain description: The page title. pageType: type: string example: article description: Coarse page classification (e.g. article, listing, forum, docs). wordCount: type: number example: 214 description: Word count of the extracted main content. description: type: object nullable: true example: A short summary of the page, as published by the page itself. description: The page's own description (meta description / og:description). language: type: object nullable: true example: en description: The language the page declares. author: type: object nullable: true example: Jane Doe description: The declared author. siteName: type: object nullable: true example: Example Blog description: The declared site name. publishedTime: type: object nullable: true example: '2026-01-05' description: When the page says it was published (ISO 8601). image: type: object nullable: true example: https://example.com/cover.png description: The page's lead image (og:image). required: - title - pageType - wordCount - description - language - author - siteName - publishedTime - image WebUsageDto: type: object properties: creditsUsed: type: number example: 1 description: Credits this call consumed. Charged only on success. creditsRemaining: type: number example: 4999 description: Credits left in your workspace's balance after this call. freshness: type: string enum: - cache - fresh example: fresh description: Whether the result was served from cache or freshly fetched. required: - creditsUsed - creditsRemaining - freshness WebQualityDto: type: object properties: confidence: type: number example: 0.94 minimum: 0 maximum: 1 description: >- How trustworthy the extraction is, from 0 to 1. High when independent checks corroborate substantial content; near zero when the page yielded almost nothing. complete: type: boolean example: true description: >- Whether the result captured the bulk of the content available on the page. False when the result looks thin or truncated. blocked: type: boolean example: false description: Whether the page appeared to be behind a challenge or bot wall. required: - confidence - complete - blocked WebLinksDto: type: object properties: internal: description: Links pointing to the same site. type: array items: type: string external: description: Links pointing to other sites. type: array items: type: string required: - internal - external WebStructuredDataDto: type: object properties: entities: description: >- The page's own structured data, normalised into one deduplicated list of typed entities. type: array items: $ref: '#/components/schemas/WebStructuredEntityDto' jsonLd: description: Raw JSON-LD blocks, as found on the page. type: array items: type: object microdata: description: Raw microdata items. type: array items: type: object opengraph: description: Raw OpenGraph/Twitter-card tags. type: array items: type: object rdfa: description: Raw RDFa items. type: array items: type: object appState: description: Names of embedded framework app-state blocks detected on the page. type: array items: type: string required: - entities - jsonLd - microdata - opengraph - rdfa - appState WebStructuredEntityDto: type: object properties: type: type: string example: Product description: The schema.org type of the entity. source: type: string enum: - json-ld - microdata - rdfa - opengraph description: Which structured-data syntax the entity came from. properties: type: object additionalProperties: true description: The entity's properties, as published. required: - type - source - properties securitySchemes: apiKey: type: apiKey in: header name: X-API-Key ````