> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hydrafetch.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Changelog
> What's new in the Hydrafetch API.
## Style guides that capture the look, not only the tokens
`/v1/web/styleguide` now describes how a site is drawn and what it looks like in dark mode.
* `components` describes the treatment most panels share (border, corners, background, padding, text colour, shadow and any decorative layer such as an offset dotted shadow), the top navigation bar, how the selected and other tabs are drawn, a full-width band painted far from the page colour such as a dark hero, and whether panel titles sit on the border line, inside the panel or above it
* `secondaryButton` is the second call to action, painted differently from the first or only outlined
* Every button, panel, header and tab carries a `css` string of declarations you can paste into a rule, and `css`, `tailwind` and `shadcn` include the panel, header, tab, band and secondary colours as variables
* `url` reads one page instead of the homepage, for a section styled differently from the rest of the site, and `maxAge` sets how old a stored result may be. `maxAge: 0` reads the page fresh
* `dark` holds the colours a site switches to for visitors who prefer a dark theme. It is `null` when we checked and the site does not change, so you know not to invent a dark palette
* `shadcn` is the design under shadcn/ui variable names (`--background`, `--foreground`, `--primary`, `--card`, `--border` and the rest), with a `.dark` block when the site has one. `css` and `tailwind` gain a `prefers-color-scheme: dark` block in the same case
* A typeface published on Google Fonts now comes with its Google `stylesheet` link, even when the site hosts its own copy, and the exports load it from there. `files` only lists a site's own font files when there is no public source
* The button is the site's call to action, not a dropdown or tab that happens to be the largest control. `minWidth` and `minHeight` are the size it rendered at
* Two colours never share a name. A second shade of the same colour is called, for example, "Pale Blue (darker)"
* Colours written in newer CSS formats such as `oklch()` are read as well. Sites built that way used to come back without text, border or button colours
* Body text size is the size most of the words are set in, not the size of the longest paragraph
Style guides saved before this change are read again the next time you ask for them.
## More of the page, more often
Pages that used to come back thin now come back whole.
* Pages built from cards and tiles, such as product hubs and homepages, keep their cards even when the top of the page reads like a short article
* Pages that draw their content in the browser after they load, such as a list filled in from a template or an app that keeps its text in page data, are now loaded the way a browser loads them, so that content comes back
* A page we already have is no longer thrown away because a later attempt ran into a block page
* A rate limit is no longer mistaken for a block, and a short JSON reply is returned as the answer it is rather than retried as a thin page
* A sign-in page with a "verify you are human" box is returned as the page it is, not reported as blocked
None of this changes the response shape. You will see higher word counts on the pages that were affected, and fewer `blocked` results on pages that were never blocked.
## A subdomain answers for its company
A brand or logo lookup for a subdomain with no identity of its own, such as `promotions.acme.com`, now returns the company's brand instead of nothing.
* `domain` is the company's domain, and `queriedDomain` is the host you asked for, so you can tell the two apart
* A subdomain with its own name, description or logo keeps its own answer
* A customer site on a hosting platform never inherits the platform's brand
Previously the lookup returned no data until you passed the bare domain yourself.
## The right dates, and when a page was last updated
`metadata.publishedTime` now comes from the article itself, and a new `metadata.modifiedTime` says when the page was last updated.
* The publication date is read from what the page declares, then from the byline beside the title. A date in a related-posts card, a sidebar or a list of other articles is no longer taken as the page's own
* When the dates beside the title disagree, or the only date is labelled as an update, `publishedTime` is `null` rather than a guess
* `modifiedTime` is the date the page gives for its last update, from its declared markup or a date it labels "Last updated". It is `null` when the page does not say
* With `onlyMainContent: true`, an article keeps the short summary paragraph directly under its title. On announcements that paragraph usually carries the key facts
A wrong date is worse than no date, because you cannot tell it apart from a real one. Fewer pages now come back with a date, and far fewer come back with the wrong one.
## Style guides you can paste
`/v1/web/styleguide` now returns the design it found as code as well as data.
* `css` is a stylesheet: an `@font-face` rule for each webfont and a `:root` block of CSS variables for colours, fonts, the type scale, radius, shadows and spacing
* `tailwind` is the same set of variables as a Tailwind CSS v4 theme, so classes such as `bg-accent`, `font-heading` and `text-h1` pick up the site's design
* Only plain colours, lengths, font names and https font files make it into either one, so a page cannot write anything else into your stylesheet
Webfont files are also more accurate. When a site served an italic and an upright file for the same weight, the italic one could be returned as the regular font; the upright file now wins. Style guides saved before this change are read again the next time you ask for them.
## Endpoints that only answer a POST
Some data never appears in a page. It sits behind a JSON or GraphQL endpoint that will not answer a `GET`, and until now the URL was the only thing you could describe.
* `method: "POST"` and `body` send the request the endpoint expects. The body goes verbatim, and you set your own content type through `headers`, so JSON, form-encoded and anything else are all just a body and a header
* `GET` stays the default. Every existing call behaves exactly as before
* Bodies are capped at 64KB. A request body selects what an endpoint returns; anything larger is a file upload, which is a different thing and not one we do
* A `POST` and a `GET` to the same URL are cached separately, and so are two `POST`s with different bodies
Two limits worth knowing. A `POST` target is fetched, never rendered, because a browser navigation is a `GET` and rendering one would quietly ask the origin a different question than you did. And `PUT`, `PATCH` and `DELETE` are deliberately not supported: `POST` is what makes unreachable data reachable, and a narrower surface is easier to reason about for a service that takes arbitrary URLs.
## Crawl and batch are tools now
Last time these were left off, on the grounds that they are jobs and a tool call is not. That was the wrong call for an agent that is perfectly happy to come back later.
* `crawl` and `batch` start the job and hand back an id; `crawl_status` and `batch_status` read it
* `logo` joins them, so an agent can fetch a company's mark without building a URL itself
* `screenshot` now takes the same sizing and timing options as the REST call
* Thirteen tools, all under the existing `web:scrape` scope, so a grant you have already approved needs no re-authorising
Costs are unchanged: a tool costs exactly what the equivalent REST call costs.
## Bring your team in
A workspace was effectively one person. It is now a team, with the awkward cases handled rather than left to support.
* Invite people, set their role, and remove them. A removed member's seat comes back to the plan immediately
* Sign up with an email at a company that already has a workspace and you are not turned away: you ask to join, and an owner approves or declines it from the team page
* An invitation now survives sign-in and sign-up, so following an invite link and then creating an account lands you in the right workspace instead of a new empty one
* One workspace per company domain, enforced everywhere a workspace can be created
We never show one member another's email address, including the owner's.
## Told in the product, not just by email
Things that only ever arrived in an inbox now appear in the app, as they happen.
* Credits running low, credits exhausted, a failed payment, a cancelled plan, your logo allowance, a publishable key being used off-domain, and join requests waiting on you
* They arrive live. No reload, no polling
* Email is now something you choose per category in settings, split into account alerts and product updates, so you can turn off the newsletter without also turning off "your payment failed"
Account alerts are the ones we would rather you did not miss, which is exactly why they are in the product now as well as your inbox.
## Colours and marks you can trust
A brand's palette is only useful if it is the brand's palette. Several cases where it was not are fixed.
* The primary colour now comes from the company's own mark rather than from whatever colour the page happened to use most, and a palette leads with that colour instead of the page background
* Your operating system's font is no longer reported as a company's typeface
* A mark that cannot say what colour it is loses to one that can, and a saturated mark is classified as mono rather than by brightness alone
* We never serve an SVG that renders blank, which is what happened when a mark was being coloured by a stylesheet we do not have
* A domain whose certificate does not cover the bare name is retried on `www` rather than failing to resolve
* A styleguide that cannot read one value returns the rest instead of nothing
Logos are cached for a week now rather than a day, so a page rendering the same mark repeatedly stays fast.
## Logos you can put straight in an `![]()
` tag
A company's logo as an image URL, with no request to make first and no credits spent.
```html theme={null}
```
* **Publishable keys** (`hf_pk_`) are safe to commit and safe to ship in client-side markup. They serve logos and nothing else, so a key lifted from your page source cannot scrape on your account
* **Logo serves never spend credits.** They have their own per-cycle allowance, so a page rendering forty logos costs forty serves and zero credits
* Lock a key to your own domains if you want, and pick the size, theme and shape per URL
* Served from the edge, so the second visitor gets it immediately
There are drop-in components for React and a browser client if you would rather not build the URL yourself.
## PDFs come back as Markdown
Point `scrape` at a PDF and you get Markdown, the same as any page. No separate endpoint, no extra credits, one scrape.
* Headings stay headings and tables stay tables, instead of collapsing into a flat run of text in reading order
* Works on anything with a text layer, including exports from Word, LaTeX and design tools
* Previously a PDF could come back carrying page markup rather than clean Markdown. It no longer does
**Scanned PDFs are not supported.** A scan is a picture of a document with no text to read, so you get empty `markdown`, `complete: false`, `confidence: 0` and a `warning` saying exactly that. Retrying will not change it.
## A publication date only when the page has one
`metadata.publishedTime` is now read from what the page actually declares, and is `null` when it declares nothing.
* A copyright year in a footer is no longer treated as a publication date. An undated pricing page used to come back "published 1 January" of whatever year the footer mentioned
* The same applies to the date line at the top of the Markdown: present when the page states one, absent when it does not
* Pages that declare a real date are unaffected
A wrong date is worse than no date, because you cannot tell it apart from a real one, and anything sorting or filtering on recency quietly inherits the error.
## Search returns results; scraping them is opt-in
`search` used to fetch every result's page by default, so a call that reads like one credit billed six. Scraping is now something you ask for.
* A search costs **1 credit** and returns the ranked title, url and snippet
* Set `scrapeResults: true` to also fetch each result, at **1 credit per page scraped**, the same as scraping that page yourself
* `data` on each result is `null` unless you asked for the scrape
If you were relying on the old default, add `scrapeResults: true` and the behaviour and cost are exactly what they were.
The `engine` parameter and the `data.engine` response field are also gone. Which index answered is an implementation detail we route on for cost and availability, and pinning it stopped us falling back when an engine was unavailable.
## One URL, one response shape
A scrape used to wait, then hand back a job id if the page ran long. That meant a second code path most callers never wrote, appearing only on the rare slow page. It is gone.
* A single URL is answered inline, always: `/scrape`, `/markdown`, `/html`, `/raw-html`, `/links` and `/structured`
* The synchronous budget is 90 seconds. Past it you get `504` with code `REQUEST_TIMEOUT`, naming the budget
* The fetch keeps running in the background after a timeout, so the capture usually lands in the cache and an immediate retry is fast
* Want a job instead? Send `async: true`. It is opt-in now, never automatic
* Crawl and batch are unchanged: they span many URLs and can genuinely run for minutes
Set your own client timeout below 90 seconds if you want to give up sooner.
## Every sync endpoint is an MCP tool
The MCP server now exposes the whole synchronous surface, and describes itself so you never have to guess what it can do.
* New tools: `styleguide`, `screenshot` and `images`, joining `scrape`, `map`, `search`, `extract` and `brand`
* `GET /mcp/tools` lists every tool with its description, credit cost and full JSON Schema — no authentication required, so a client can see the capabilities before connecting
* Tools cost exactly what the equivalent REST call costs
* The new tools sit under the existing `web:scrape` scope, so an existing grant needs no re-authorising
Crawl and batch stay off MCP for now: they are jobs, and a tool call is not.
## Brand and styleguide
Answer "what is this company" and "what does it look like" from nothing but a domain, instead of scraping a homepage and guessing.
* `GET /v1/web/brand` returns logos suited to light and dark backgrounds, the real brand colours, name, description and social profiles. Every field states whether the company declared it or we harvested it
* `POST /v1/web/brand` resolves up to 50 domains in one call
* `GET /v1/web/brand/search` finds brands we already hold by company name or domain, so you can turn what a user typed into a domain before resolving it. Returns enough to render a picker, never resolves a new domain, and is **not billed**
* `GET /v1/web/styleguide` reads the design system a site actually paints, from computed styles in a real browser, so a colour defined through four layers of CSS variables comes back as the hex it resolved to. Colours arrive by role with contrast ratios, so they drop into a theme without guesswork
A brand resolves once and is served from storage after that, so the second call for a domain is the same price and immediate.
## Billing that matches what you got
Several cases where a call could be charged without delivering a usable page are now not billed at all.
* An origin's own error page is no longer recorded as a delivered scrape
* A response with no bytes is never treated as a success, whatever status the origin returned
* A failed render no longer discards the static capture it was meant to improve on
* Upstream failures surface as errors rather than empty `200`s
Failures were always meant to be free. These were the paths where that promise leaked.
## Ask for structure when you need it
`preferStructure` keeps headings, lists and tables as real markdown. It was already on the REST API; it is now on the MCP tools too, so agents can ask for it.
* `metadata.format` is now `metadata.structure`, reporting `markdown` or `plain`
* `plain` is not a failure — the default extractor optimises for capturing every word and wins on listing and forum pages — but if you wanted headings, retry with `preferStructure: true`
* Available on `scrape`, `search` and `extract` over both REST and MCP
## See whether your webhooks landed
Every callback we attempt for a crawl or batch is now recorded and readable, so a webhook that never arrives is something you can diagnose instead of guess at.
* `GET /v1/web/crawl/{id}/deliveries` and `GET /v1/web/batch/{id}/deliveries`
* Each delivery reports its status, attempt count, the HTTP status your endpoint returned, and why the last attempt failed
* A `failed` row with a 5xx means we couldn't reach you; a 4xx means we did and you rejected it
## Errors you can branch on
Every error now returns the same envelope with a stable, machine-readable `code`. Branch on the code — the wording of `message` may change, the code won't. Each response also carries a `requestId` to quote if you need us to look something up.
## Trustworthy extraction
Structured extraction now returns, per field, how confident it is and the exact passage a value came from. Turn it on with `showConfidence` and `showSources`.
* Extract across a whole site: pass a `/*` scope in `urls` and every discovered page is merged into one result.
* New `entities` primitive returns normalized people, companies, and products found on a page.
* Every response reports the credits it consumed and how fresh the data is.
## Dedicated format endpoints
Prefer a single, purpose-built route? Each output now has its own endpoint, alongside the flexible `/scrape`.
* `POST /v1/web/markdown` for clean Markdown
* `POST /v1/web/html` for cleaned main-content HTML
* `POST /v1/web/raw-html` for unmodified page HTML
* `POST /v1/web/links` for a page's links
* `POST /v1/web/structured` for the page's own structured data
They share every option and response shape with `/scrape`, so you can switch anytime.
## Search, crawl, and media
* **Search** the web and get ranked results, each scraped to clean data. Filter by time, domain, and country.
* **Crawl** a whole site as one asynchronous job, with completion webhooks so you do not have to poll.
* **Media**: capture a full-page screenshot or collect a page's images.
## Launch
The first public version of the Hydrafetch API.
* `POST /v1/web/scrape` takes one URL and returns clean Markdown and structured data.
* `POST /v1/web/map` discovers every URL on a site.
* Credit-based billing, charged only on success, with a live balance on every response.
* API-key authentication through the `X-API-Key` header.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.