Agents build. Code runs. A QA gate decides what ships.
Language models changed how fast a scraper can be written and repaired. They did not make running one reliable. So we use each part of the stack for what it is good at.
Figure out the web, write and repair code
Reading a new site, mapping its structure, writing an extractor, testing it on live pages, drafting a fix when the layout changes. Language models are now very good at this, and fast.
Run it thousands of times without drifting
Production runs use deterministic extractors and workers, not a model reading screenshots. That keeps the cost per page at fractions of a cent and every run reproducible.
Decide what ships and own the outcome
An engineer approves every extractor change, owns the SLA and talks to you. Agents propose; a person is accountable.
From request to a feed you can depend on
- 1
Scope
We map the sources, the fields and the most reliable way to get each one, and send you a fixed quote.
- 2
Agents build
Coding agents write extractors against your schema and test them on live pages. A person reviews the change.
- 3
QA gate
Every run passes deterministic checks, then an independent model spot-checks a sample. Failures go back, not out.
- 4
Run and monitor
Plain code runs the feed on schedule. When a site changes we are alerted, an agent drafts the fix, we review it.
The most reliable route, chosen per source
More of the web now blocks automated traffic by default: in our November 2026 index, 14% of top sites turned away a plain automated client outright. We don't fight that with brittle tricks. We pick the route that will still work next month.
- 1
Official API or feed
If the source publishes an API, export or feed, we use it. Fastest, most stable, least friction.
- 2
Open pages
Public pages fetched politely, with sensible rates and caching, and respect for the site's crawl rules.
- 3
Licensed or paid access
Some sites offer data licences or pay-per-crawl access. Where that is the reliable route, we arrange it and include or pass through the cost.
- 4
Engineered collection
Rendering, session handling and specialist infrastructure for hard sources, run through established providers rather than ad-hoc hacks.
Every batch is checked before it reaches you.
Schema validation
Types, required fields, allowed values and keys on every row.
Volume checks
Row counts compared with previous runs; sudden drops or spikes stop delivery.
Empty-field tracking
Fill rates per field tracked over time to catch silent layout changes.
Duplicate detection
Stable IDs and content hashes to merge or flag duplicates.
Cross-model spot check
A sample of each batch re-verified against the source by a model from a different family than the one that built it.
Human review
Anything that fails, and every extractor change, is reviewed by a person before it ships.
This is the gate that runs our own Nomad Watch dataset. In its third batch the cross-model check confirmed 100% of city scores, 87% of guides and 82% of cost records. Earlier cost batches passed only 55–64% and were sent back for fixes. That is the gate doing its job.
Questions we get
Something else? Ask us directly.
Which AI models do you use?
Several, from different vendors. Building and checking with models from different families catches errors one model would repeat. On our own Nomad Watch pipeline, five backends shared the work over four weeks.
Do language models ever write values into my data?
No. Values come from the page through an extractor. Where a field is derived, for example a category inferred from a description, it is labelled as derived and keeps the source text it was based on.
What do I see as a customer?
The data, a run log with row counts and health per source, the schema and its version history, and a change log of extractor fixes. Team plans add a shared channel.
What happens if a source becomes impossible to collect?
We tell you immediately, propose alternatives (another source, an official dataset, a licence), and stop billing for that source if there is no reliable route.
Get a free 100‑row sample
Tell us what you need to track. Within 2 business days you'll get a sample, the sources we'd use and a fixed price, or an honest note that an off-the-shelf tool will do.
- Sample and fixed quote in 2 business days
- Source URL and timestamp on every row
- No commitment, no sales sequence
Prefer email? [email protected]