How a scraper is built
A build is the one moment AI is involved. It explores the site with a real browser to find the way to your data, records that way as a plan, walks it, and then prints a program from what it observed — a deterministic step with no model in it. A verdict judges the rows, and anything it is unsure of comes back as a doubt instead of a failure.
- Follow the stages of a build in order
- Know why an example value is needed
- Read the verdict and its match score
- Understand doubts and what repair does
The stages
| Stage | What happens |
|---|---|
| 1. Example | Every input needs a real value to explore with. If you gave none, the chat asks one question and waits. |
| 2. Explore | The AI looks at each page — its outline and a screenshot — and picks one action: fill, select, click, submit, navigate, wait, extract, done, or cannot. |
| 3. Plan | The explored actions become the plan’s steps, each with its target and value. |
| 4. Walk | The plan is walked in a browser; a picture of every step reaches Browser playback as it happens. |
| 5. Program | The walk is printed as a Python program. No model writes this code: it is rendered from what was observed. |
| 6. Prove | The program runs once to show it works, and its rows fill Extracted results. |
Exploring the site
Exploration is a conversation between the page and the AI, one action at a time, and every action is performed by the engine itself with the same rules a run uses. It has a budget of 10 minutes and 200 pages. When the AI cannot go on — a choice only you can make — it posts the question in the chat and the build waits for your answer. A site that needs a login it was not given is reported as such; it never invents credentials.
Asking the verifier during the walk
While walking, three moments are judged on the spot: when two tables could be the list you asked for, whether a page reached is the page the step meant, and whether a loop is stuck repeating itself. The best candidate wins; a wrong page or a stuck loop is never retried — it becomes a doubt.
Verdict, doubts and repair
After the walk, a verdict compares the rows with your sentence field by field and gives a Match score: passed, close or failed. It is taken by the platform’s verifier or, when that is not available, by this scraper’s own AI — the verdict, in the build run’s Logs under the results, says which.
- A field the verdict rejects goes to repair: the extraction is asked again with a hint about what was wrong — up to 3 tries per stage and 10 per build, within the time left.
- What repair cannot settle is delivered with doubts: the build still ends successfully, and the result says Delivered with doubts about: … with a reason for each, such as the value may not be the field asked for.
- Every result carries a
doubtslist — empty when nothing is in doubt — so a flow or script can check it before trusting the rows.
Limits
- Exploration stops at 10 minutes or 200 pages; running out is reported as a doubt (the exploration ran out of time or pages before it finished).
- The verifier is less accurate when the description is not in English; the verdict says so.
- Without any AI for the project, no verdict is taken and the build says why.