> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hydrafetch.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Watch a site for changes
> Re-read a set of pages on a schedule and get told what changed, rather than polling by hand.
**Have an agent build it.** Copy the brief below into Claude Code, Cursor, or any coding agent with access to your project. It states the calls, the questions worth asking you first, and the mistakes to avoid.
```text theme={null}
Implement this blueprint in my project:
https://docs.hydrafetch.com/blueprints/watch-a-site
Read that page, inspect this project's stack, then build the flow end to end.
Build a change monitor over a small set of URLs using the Hydrafetch API.
Per run: POST /v1/web/batch with {urls, scrapeOptions: {formats: ["markdown"], maxAge: 0}}. It is asynchronous, returning a batchId to poll at GET /v1/web/batch/{id}, or attach a webhook and receive the result instead.
Ask me before writing code:
- Which pages, and how often should they be re-read?
- Where does the previous version live so we can diff against it?
- What counts as a change worth alerting on, and what is noise to normalise away first?
- Where should the alert go: email, Slack, a webhook of our own?
- Should a page that starts failing be reported as a change, and after how many consecutive failures?
The response shape: data.pages[] each with url, status, error, errorCode, and data holding markdown, finalUrl and metadata (title, wordCount).
Store the previous markdown itself rather than a hash, so the alert can show a real diff instead of just saying that something moved. Normalise before comparing: strip dates, relative timestamps, rotating counters and build hashes, and collapse whitespace, or the watch fires every run and gets muted. Compare the markdown, not the html. Treat a page that fails several consecutive runs as removed, but never on a single failure. Record every run including the quiet ones, so a watch that silently broke can be told apart from a site that is stable.
Notes: authenticate with the X-API-Key header. 1 credit per page read, failures are free. Set maxAge to 0 so a stored copy cannot answer a monitoring run. Daily is almost always enough, and reading a site hourly is how you become a problem for it. Keep the API key on the server and never ship it in client code.
```
Pricing pages move. Competitors quietly add a feature. A supplier changes their terms. You want to know when, without a person checking every week.
There is no change endpoint. There is something better: fetches that are cheap when nothing moved, so a scheduled re-read is affordable and you diff the results yourself.
## The pipeline
### 1. Pick the pages, once
Use `map` to list the site, and keep the handful that actually matter. Watching a whole site produces noise; watching a pricing page and a features page produces signal.
### 2. Re-read them on your schedule
Run a `batch` over the watched URLs from your own cron, and set `maxAge` to control how stale a stored copy may be before we go back to the origin.
```bash theme={null}
curl -X POST https://api.hydrafetch.com/v1/web/batch \
-H "X-API-Key: $HYDRAFETCH_API_KEY" \
-H "Content-Type: application/json" \
-d '{"urls": ["https://example.com/pricing", "https://example.com/features"],
"scrapeOptions": {"formats": ["markdown"], "maxAge": 0}}'
```
Set `maxAge` to `0` when the whole point is to see the current state. Leave it high when you are rebuilding an index and a day old copy is fine.
### 3. Normalise before you compare
This is the step that decides whether anyone still reads your alerts in a month.
A page changes on every single request for reasons that have nothing to do with the content: a rotating testimonial, a "trusted by 4,312 teams" counter, a copyright year, a build hash in a footer, a relative timestamp reading "3 hours ago". Diff the raw text and every one of those fires an alert, and a channel that cries wolf daily gets muted in a week.
Strip them before comparing. Drop anything matching a date or time pattern, collapse runs of whitespace, and remove the lines you have already seen change without meaning anything. Keep the list of exclusions in code next to the watch, because it grows every time a new false positive appears.
Compare the cleaned Markdown rather than the HTML. HTML changes constantly and means nothing: a reordered script tag, a rotated asset hash, a session id. The Markdown changes when the words change, which is the thing you actually wanted to be told about.
### 4. Treat a failure as a change too
A page that stops answering is news. If a URL you have watched for six months starts returning a 404, that is a product that got retired or a page that moved, and it is usually more interesting than an edited sentence.
Every batch result carries a per-page `status`, and a failed page carries an `error` and an `errorCode`. Report those rather than skipping them, but do not treat one bad run as a deletion. A page has to fail two or three consecutive runs before it is gone, otherwise a momentary blip reads as a removal.
### 5. Get told, instead of polling
Attach a webhook to the batch and receive the results when they land, rather than holding a polling loop open. See [Jobs and webhooks](/concepts/jobs-and-webhooks) for the delivery shape and retry behaviour.
## Making the alert worth reading
An alert saying "example.com/pricing changed" is a task, not information. The reader still has to open two tabs and compare them by eye, and after the third such alert they stop.
Store the previous Markdown itself, not a hash of it. A hash tells you something changed and nothing about what, which turns every alert into an investigation. With the previous version in hand you can show a real diff in the message.
Send the diff, trimmed to the changed lines with a little surrounding context. For most watches that is three or four lines, and it is the difference between an alert someone acts on and one they archive.
Say what did not change too, at least in a weekly summary. Silence is ambiguous: it reads the same whether nothing moved or the watch quietly broke three weeks ago.
## Knowing the watch still works
A monitor that fails silently is worse than no monitor, because you believe you are covered.
Record every run, not just the ones that found something. If a watch has reported no changes for a month, check that it is still running and still fetching successfully rather than assuming the site is stable.
Watch the fetch quality as well as the content. A page that starts coming back with a much lower `wordCount` than usual has probably started serving a consent wall or a login page rather than losing most of its content, and diffing that against last week produces a spectacular and completely useless alert.
Re-run discovery occasionally. A `map` every month or so catches the pages that appeared since you picked your list, which is how a competitor's new pricing tier gets noticed rather than missed because it lives at a URL you never knew existed.
## What it costs
1 credit per page read. Watching six pages daily is 6 credits a day, or about 180 a month.
Discovery is 1 credit per `map`, so a monthly re-check of the URL list is a rounding error against the watching itself.
## What to watch for
Daily is almost always enough. Nobody's pricing page changes hourly, and reading it hourly is how you become a problem for someone else's site.
Watch few pages deliberately rather than a whole site automatically. Ten pages you chose produce signal; four hundred pages you crawled produce a feed nobody reads.
Set `maxAge` to `0` when the whole point is to see the current state. Leaving the default means a stored copy can answer, which is right for indexing and wrong for monitoring.