S3 storage

How files reach the bucket

A script asks; the worker does. When your code calls s3.upload, the bytes go from the sandbox to the worker process next to it, which signs the request with the connection’s keys and sends it to S3. The keys stay in the project’s secret variables and the worker’s memory — never in your code, your logs or the run record.

Every function takes a file as an S3 object — { "s3": { "path": "…", "name": "…" } }, what an upload returns — or as its path string. That shape is also what a step returns, so one script can upload a file and the next step of a flow can read it.

Connect a bucket

1. 1
   Pick Amazon S3

   On Integrations, click Add integration and choose Amazon S3 in the Storage group.

2. 2
   Fill the bucket and keys

   Give a Name, the Bucket and its Region (a list of AWS regions), an Access key ID and a Secret access key of an IAM user allowed to read and write that bucket.

3. 3
   Set an upload prefix (optional)

   Upload prefix puts every upload under a folder, for example waiso/, so the bucket can be shared with other systems. Slashes at the ends are tidied for you.

4. 4
   Save and test

   The confirmation lists the two variables the keys become — a connection named files gives S3_FILES_ACCESS_KEY_ID and S3_FILES_SECRET_ACCESS_KEY. Test connection then checks the bucket with those keys: a wrong secret reports Failed.

Upload from code

upload takes text, bytes or (Python, JavaScript) a readable stream, plus a name and an optional folder. A text source is always the file’s content, never a path on disk. The example writes a small CSV, then makes a link valid for ten minutes:

from waiso import s3

def main(title: str = "Weekly stock"):
    csv = "sku,qty\nA1,3\nB2,5\n"
    obj = s3.upload(csv, name="stock.csv", folder="reports")
    link = s3.presign(obj, expires=600)
    return {"title": title, "path": obj.s3.path, "link": link}

{
  "title": "Weekly stock",
  "path": "reports/3f9a1c2e_stock.csv",
  "link": "https://my-bucket.s3.us-east-1.amazonaws.com/reports/3f9a1c2e_stock.csv?X-Amz-Expires=600&…"
}

Where a file lands

The stored key is <upload prefix><folder>/<8 random hex>_<name>. With the prefix waiso/, the call above stores waiso/reports/3f9a1c2e_stock.csv. Keep the returned path — it is the only way to find the file again from code.

Download, inspect and share

Reading goes the other way: download returns the bytes, or text when you give an encoding. This script counts the data rows of a CSV whose path arrives as an input:

from waiso import s3

def main(path: str):
    text = s3.download(path, encoding="utf-8")
    rows = [line for line in text.splitlines()[1:] if line]
    info = s3.stat(path)
    return {"rows": len(rows), "bytes": info.size}

{
  "rows": 2,
  "bytes": 18
}

upload_from_url (uploadFromUrl, waiso_s3_upload_from_url) copies a file from another web address straight into the bucket. The worker fetches it — your code still has no network — and refuses private addresses at every redirect with host_not_allowed.

The file browser

On Integrations, the S3 card’s Browse Files opens the File Browser. It walks the bucket folder by folder from Bucket root, searches with Search files…, filters by type (Folders, Images, Documents, Data, Archives, Code, Other) and sorts by name, size or date. Upload file and New folder work in the folder you are in; Copy bucket name and Refresh sit beside them. The side panel totals the bucket — Total Size, Objects, Largest Files — estimated above 100,000 objects.

Clicking a file opens its details: S3 Key, Content Type, Detected MIME, Visibility, and the actions Download, Copy Presign URL, Make Public and Make Private — the last asks first, because the permanent URL stops working at once.

Limits and errors

• s3:credential_missing — the connection’s key variables were deleted. Open the connection and enter the keys again.
• S3AclBlockedError (C: WAISO_ERR_ACL_BLOCKED) — the bucket has Block Public Access on, so a file cannot be made public. S3AclNotSupportedError (WAISO_ERR_ACL_NOT_SUPPORTED) — the bucket has ACLs turned off. Use presign instead.
• Downloads are read into memory; C++ s3::toBytes reads up to 64 MB, and C can stream a large file in 64 KB chunks with waiso_s3_download_stream.
• Uploads over 5 MB go in parts of 10 MB by default (chunk_size).