The program and its versions

What the program looks like

The top of the file holds the decisions — where to start, which rows to read and how each field is read. The main(inputs) function walks the steps, each wrapped in waiso.step(...) so it gets its picture and its line in Program session, and returns the rows.

from playwright.sync_api import sync_playwright

import waiso_runtime as waiso  # steps, pictures, reading rules

START_URL = "https://shop.example.com/category/{{category}}"

ROW_SELECTOR = "div.product"
FIELDS = {
    "name": {"selector": ".name"},
    "price": {"selector": ".price", "type": "number"},
    "inStock": {"selector": ".available", "read": "attr",
                "attr": "data-available", "type": "boolean"},
}


def main(inputs: dict) -> list[dict]:
    with sync_playwright() as p:
        browser = waiso.launch(p)
        page = browser.new_page()
        with waiso.step("Open the category", page, mark=None):
            waiso.goto(page, START_URL, inputs)
        with waiso.step("Read the products", page, mark=ROW_SELECTOR):
            rows = waiso.extract(page, ROW_SELECTOR, FIELDS, [])
        browser.close()
    return rows

The library behind it

The rules for driving and reading a page live in the waiso_runtime library, not in each file: goto, click and fill drive the page, extract reads rows and fields, each_page follows pagination and crawl walks a whole site. That keeps a program short enough to read in one screen, and a fix to a reading rule reaches every scraper at once. The program needs the library to run, so it runs inside Waiso, not on its own.

Change it

Through the assistant

Ask in the chat — “also read the rating”, “skip items out of stock”. The assistant modifies the existing program, the chat shows the change as a diff, and the result is a new version marked Changed by the assistant.

By hand

Edit the Scraper code tab and press Save as new version. If the code changed meanwhile — the assistant saved a version while you typed — the save is refused: The code changed meanwhile (now v7). Reload it before saving — your edit was not applied. Nothing is ever merged silently.

Test and publish

1. 1
   Set the inputs

   The Inputs tab holds the values the next Test run uses.

2. 2
   Press Test run

   The draft runs as it is, with no model. A version that ran cleanly is marked Tested.

3. 3
   Press Publish

   The draft becomes the version every flow, schedule and endpoint calls — Published — this version is now live.

   daily-report.tsdraftSavePublish

Versions

Compare with draft shows any version against the draft. Restore makes an old version the draft as a new version, so history only ever grows and nothing is lost.