From 546aa35801027018a45a3c3d4e9c73136b238cbc Mon Sep 17 00:00:00 2001 From: Lucas Giordano Date: Tue, 18 Aug 2026 18:15:46 +0200 Subject: [PATCH 1/2] docs: worked local-vs-hosted comparison across examples and the site MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rounds out the funnel with the content layer. - examples/hosted: the same job three ways — local capture, hosted over HTTP, hosted over MCP — with a comparison table and two dependency-free scripts. `discover.py` searches the public marketplace and needs no credentials; `run_hosted.py` executes one function and needs an API key. - Every existing example README gains a footer pointing at the hosted route, phrased around maintenance rather than claiming a hosted equivalent exists. None of those six sites are in the marketplace today, so the footers link to a search and to building one, which stays true either way. - A /docs/cloud page covering when to use which, the pre-capture lookup, how to turn it off, CLI search, and the MCP endpoint. - A "Run it, or don't." section on the home page, plus a card and a short section in the docs index and a callout in agent mode. The comparison is deliberately even. Overselling the hosted side to an audience that just installed an MIT CLI would read as a bait-and-switch, so both columns list what the other does better — offline, code review, and network-local execution stay firmly in the local column. Co-Authored-By: Claude Opus 5 (1M context) --- examples/apple/README.md | 22 ++++ examples/ashby/README.md | 22 ++++ examples/autoscout24/README.md | 22 ++++ examples/hosted/README.md | 108 ++++++++++++++++++ examples/hosted/discover.py | 72 ++++++++++++ examples/hosted/run_hosted.py | 74 ++++++++++++ examples/ikea/README.md | 22 ++++ examples/mintlify/README.md | 22 ++++ examples/uber/README.md | 22 ++++ website/content/docs/cloud.mdx | 107 ++++++++++++++++++ website/content/docs/index.mdx | 8 ++ website/content/docs/meta.json | 3 +- website/content/docs/modes/agent.mdx | 7 ++ website/src/app/(home)/page.tsx | 2 + website/src/components/local-or-hosted.tsx | 124 +++++++++++++++++++++ 15 files changed, 636 insertions(+), 1 deletion(-) create mode 100644 examples/hosted/README.md create mode 100644 examples/hosted/discover.py create mode 100644 examples/hosted/run_hosted.py create mode 100644 website/content/docs/cloud.mdx create mode 100644 website/src/components/local-or-hosted.tsx diff --git a/examples/apple/README.md b/examples/apple/README.md index e06ef059..1ff4207b 100644 --- a/examples/apple/README.md +++ b/examples/apple/README.md @@ -407,3 +407,25 @@ print(json.dumps(results, indent=2, ensure_ascii=False)) ``` Save this as `quick_example.py` and run it to get started immediately! + +--- + +## Don't want to maintain this? + +This client was reverse-engineered locally and is yours to keep — but it is +pinned to jobs.apple.com's API as it looked on the day of capture, and nothing here +re-engineers it when that changes. + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=apple) +is the hosted version of this project: you describe the task and get back a +deployed API function, with the proxies, retries, and repair-on-change handled +for you. Check whether Apple's careers site is already covered before building anything: + +```bash +reverse-api-engineer marketplace search --site jobs.apple.com +``` + +If nothing matches yet, describe the task at +[anything.notte.cc](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=apple) +and it gets built for you. Agents can search the same catalogue by pointing an +MCP client at `https://anything.notte.cc/mcp`. diff --git a/examples/ashby/README.md b/examples/ashby/README.md index 97cd563e..d6892908 100644 --- a/examples/ashby/README.md +++ b/examples/ashby/README.md @@ -700,3 +700,25 @@ Please document and share your findings. ## Contact For questions about this reverse engineering project, refer to the HAR analysis source file. + +--- + +## Don't want to maintain this? + +This client was reverse-engineered locally and is yours to keep — but it is +pinned to jobs.ashbyhq.com's API as it looked on the day of capture, and nothing here +re-engineers it when that changes. + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=ashby) +is the hosted version of this project: you describe the task and get back a +deployed API function, with the proxies, retries, and repair-on-change handled +for you. Check whether Ashby-hosted job boards is already covered before building anything: + +```bash +reverse-api-engineer marketplace search --site jobs.ashbyhq.com +``` + +If nothing matches yet, describe the task at +[anything.notte.cc](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=ashby) +and it gets built for you. Agents can search the same catalogue by pointing an +MCP client at `https://anything.notte.cc/mcp`. diff --git a/examples/autoscout24/README.md b/examples/autoscout24/README.md index 0fd6a47a..bffc232b 100644 --- a/examples/autoscout24/README.md +++ b/examples/autoscout24/README.md @@ -334,3 +334,25 @@ This client was generated through automated reverse engineering. Improvements an --- **Generated by Claude Code** | 2026-01-03 + +--- + +## Don't want to maintain this? + +This client was reverse-engineered locally and is yours to keep — but it is +pinned to autoscout24.com's API as it looked on the day of capture, and nothing here +re-engineers it when that changes. + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=autoscout24) +is the hosted version of this project: you describe the task and get back a +deployed API function, with the proxies, retries, and repair-on-change handled +for you. Check whether AutoScout24 listings is already covered before building anything: + +```bash +reverse-api-engineer marketplace search --site autoscout24.com +``` + +If nothing matches yet, describe the task at +[anything.notte.cc](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=autoscout24) +and it gets built for you. Agents can search the same catalogue by pointing an +MCP client at `https://anything.notte.cc/mcp`. diff --git a/examples/hosted/README.md b/examples/hosted/README.md new file mode 100644 index 00000000..2ad35b54 --- /dev/null +++ b/examples/hosted/README.md @@ -0,0 +1,108 @@ +# The same job, three ways + +Every other folder in `examples/` is the output of a local capture: a client +file you own, pinned to the site as it looked that day. This one is the +comparison — the same "get data out of a website" job done locally, done +hosted over HTTP, and done hosted from an agent. + +Nothing here is a pitch for one of them. They fail differently, and which one +you want depends on who maintains the result. + +## The three routes + +| | Local (this repo) | Hosted over HTTP | Hosted over MCP | +|---|---|---|---| +| You get | a client file in your repo | a callable endpoint | a tool your agent can call | +| Setup | install the CLI, run a capture | an API key | one config line | +| Runs on | your machine | Anything's browsers | Anything's browsers | +| Site changes | you re-run the capture | re-engineered for you | re-engineered for you | +| Cost | your model tokens | per run | per run | +| Offline | yes | no | no | +| Code review | you read every line | you read the output | you read the output | + +## 1. Local — reverse-engineer it yourself + +```bash +reverse-api-engineer +> get NFL team standings for the current season +``` + +A browser opens, traffic is captured, and your model writes the client into +`./scripts/nfl_standings_api/`. See any of the sibling example folders for +what that output looks like in practice. + +The client is MIT, yours, and runs anywhere. It also stops working the day +nfl.com changes its endpoints, and re-running the capture is on you. + +## 2. Hosted over HTTP + +First, find out whether the function already exists. The marketplace search +endpoint is public — no key, no account: + +```bash +python discover.py nfl.com +``` + +``` +5 function(s) for 'nfl.com': + + get_nfl_team_standings [nfl.com] · 7 runs + Returns NFL team standings from https://nfl.com for a selected season, + season type, and week... + https://anything.notte.cc/marketplace/365309fa-acb6-4226-b410-9a5f86fb72d9 +``` + +`discover.py` filters by domain rather than trusting the ranking, because the +endpoint ranks instead of filtering: query a site it has never seen and it +still returns a page of other people's functions. + +Then run one. This part needs a key from +[console.notte.cc](https://console.notte.cc): + +```bash +export NOTTE_API_KEY=... +python run_hosted.py '{"season": 2025}' +``` + +Each function declares its own variables; the names and types are on that +function's marketplace page, which `discover.py` prints for every result. + +## 3. Hosted over MCP + +Point an MCP client at the endpoint and your agent searches the same catalogue +on its own — checking for an existing function before it builds anything: + +```json +{ + "mcpServers": { + "anything": { + "url": "https://anything.notte.cc/mcp" + } + } +} +``` + +The server exposes `search` (public), plus `spec`, `run`, and `build` once +authenticated. `build` describes a task in plain English and deploys a new +function, which is the hosted equivalent of a capture run here. + +## Which to use + +Use the **local** route when the client belongs in your repo, when you need it +to run offline or inside your own network, when the site needs your logged-in +session, or when you want to read every line before it executes. + +Use the **hosted** route when you want an endpoint rather than a file, when +you would rather not own the repair work each time the site shifts, or when an +agent needs to reach hundreds of sites without you building each one. + +The [CLI ships with a marketplace lookup](../../README.md#hosted-version) for +exactly this reason: before a capture, it tells you if the site is already +covered, and enter carries on capturing anyway. + +## Files + +- `discover.py` — public marketplace search, runs with no credentials. +- `run_hosted.py` — execute one hosted function, needs `NOTTE_API_KEY`. + +Both are dependency-free (standard library only) and target Python 3.11+. diff --git a/examples/hosted/discover.py b/examples/hosted/discover.py new file mode 100644 index 00000000..0bd481f4 --- /dev/null +++ b/examples/hosted/discover.py @@ -0,0 +1,72 @@ +"""Search the Anything marketplace for a function that already exists. + +Runs with no API key and no account — the marketplace search endpoint is +public. Use it before spending a capture run on a site somebody has already +reverse-engineered. + + python discover.py nfl.com + python discover.py "instagram post comments" +""" + +import json +import sys +from urllib.parse import urlencode +from urllib.request import Request, urlopen + +SEARCH_ENDPOINT = "https://anything.notte.cc/api/marketplace/search" +MARKETPLACE_URL = "https://anything.notte.cc/marketplace" + + +def search(query: str, limit: int = 5) -> list[dict]: + """Return marketplace functions matching `query`. + + The endpoint ranks rather than filters, so a query it cannot match still + comes back full of unrelated functions. Filter by domain yourself when you + are asking "does this specific site have coverage?" — see + `functions_for_site` below. + """ + url = f"{SEARCH_ENDPOINT}?{urlencode({'q': query, 'limit': limit})}" + with urlopen(Request(url, headers={"User-Agent": "anything-example"}), timeout=15) as response: + payload = json.load(response) + return payload.get("results", []) + + +def functions_for_site(domain: str, limit: int = 5) -> list[dict]: + """Return only the functions that genuinely belong to `domain`.""" + domain = domain.lower().removeprefix("www.") + matches = [ + fn + for fn in search(domain, limit=20) + if (fn.get("domain") or "").lower() == domain or (fn.get("domain") or "").lower().endswith(f".{domain}") + ] + matches.sort(key=lambda fn: fn.get("run_count", 0), reverse=True) + return matches[:limit] + + +def main() -> int: + if len(sys.argv) < 2: + print(__doc__) + return 2 + + query = " ".join(sys.argv[1:]) + # A bare hostname is almost always a "is this site covered?" question. + looks_like_domain = "." in query and " " not in query + results = functions_for_site(query) if looks_like_domain else search(query) + + if not results: + print(f"No hosted function matches {query!r} yet.") + print("Describe the task at https://anything.notte.cc and it gets built for you.") + return 0 + + print(f"{len(results)} function(s) for {query!r}:\n") + for fn in results: + print(f" {fn['label']} [{fn.get('domain', '?')}] · {fn.get('run_count', 0)} runs") + if fn.get("description"): + print(f" {fn['description']}") + print(f" {MARKETPLACE_URL}/{fn['function_id']}") + print() + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/examples/hosted/run_hosted.py b/examples/hosted/run_hosted.py new file mode 100644 index 00000000..94ed458e --- /dev/null +++ b/examples/hosted/run_hosted.py @@ -0,0 +1,74 @@ +"""Run a hosted Anything function instead of a locally generated client. + +Unlike `discover.py`, this one needs a key: get one at https://console.notte.cc +and export it as NOTTE_API_KEY. + + export NOTTE_API_KEY=... + python run_hosted.py 365309fa-acb6-4226-b410-9a5f86fb72d9 '{"season": 2025}' + +Each function declares its own variables. The names and types are listed on +that function's marketplace page — `discover.py` prints the URL for every +result, so start there rather than guessing. +""" + +import json +import os +import sys +from urllib.error import HTTPError +from urllib.request import Request, urlopen + +RUN_ENDPOINT = "https://anything.notte.cc/api/functions/{function_id}/run" + + +def run(function_id: str, variables: dict, api_key: str) -> dict: + """Execute one hosted function and return its result JSON. + + The request mirrors the `run` tool exposed on the MCP endpoint: the + function id identifies which skill to execute, and `variables` carries the + values it declared. + """ + request = Request( + RUN_ENDPOINT.format(function_id=function_id), + data=json.dumps({"variables": variables}).encode(), + headers={ + "Authorization": f"Bearer {api_key}", + "Content-Type": "application/json", + "User-Agent": "anything-example", + }, + method="POST", + ) + # Hosted runs drive a real browser, so they are slow by design. + with urlopen(request, timeout=180) as response: + return json.load(response) + + +def main() -> int: + api_key = os.environ.get("NOTTE_API_KEY") + if not api_key: + print("error: set NOTTE_API_KEY (get one at https://console.notte.cc)", file=sys.stderr) + return 2 + + if len(sys.argv) < 2: + print(__doc__, file=sys.stderr) + return 2 + + function_id = sys.argv[1] + try: + variables = json.loads(sys.argv[2]) if len(sys.argv) > 2 else {} + except json.JSONDecodeError as exc: + print(f"error: variables must be a JSON object ({exc})", file=sys.stderr) + return 2 + + try: + result = run(function_id, variables, api_key) + except HTTPError as exc: + body = exc.read().decode(errors="replace") + print(f"error: HTTP {exc.code} — {body}", file=sys.stderr) + return 1 + + print(json.dumps(result, indent=2, ensure_ascii=False)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/examples/ikea/README.md b/examples/ikea/README.md index f5dcdc18..88a37bdf 100644 --- a/examples/ikea/README.md +++ b/examples/ikea/README.md @@ -348,3 +348,25 @@ This is an unofficial API client and is not affiliated with, endorsed by, or spo --- **Generated with Claude Code** 🤖 + +--- + +## Don't want to maintain this? + +This client was reverse-engineered locally and is yours to keep — but it is +pinned to ikea.com's API as it looked on the day of capture, and nothing here +re-engineers it when that changes. + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=ikea) +is the hosted version of this project: you describe the task and get back a +deployed API function, with the proxies, retries, and repair-on-change handled +for you. Check whether IKEA's product catalogue is already covered before building anything: + +```bash +reverse-api-engineer marketplace search --site ikea.com +``` + +If nothing matches yet, describe the task at +[anything.notte.cc](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=ikea) +and it gets built for you. Agents can search the same catalogue by pointing an +MCP client at `https://anything.notte.cc/mcp`. diff --git a/examples/mintlify/README.md b/examples/mintlify/README.md index 905b71d5..3c74c725 100644 --- a/examples/mintlify/README.md +++ b/examples/mintlify/README.md @@ -225,3 +225,25 @@ This client should work with any documentation site using Mintlify's AI assistan ## License MIT License - Feel free to use and modify. + +--- + +## Don't want to maintain this? + +This client was reverse-engineered locally and is yours to keep — but it is +pinned to mintlify.com's API as it looked on the day of capture, and nothing here +re-engineers it when that changes. + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=mintlify) +is the hosted version of this project: you describe the task and get back a +deployed API function, with the proxies, retries, and repair-on-change handled +for you. Check whether Mintlify-hosted docs sites is already covered before building anything: + +```bash +reverse-api-engineer marketplace search --site mintlify.com +``` + +If nothing matches yet, describe the task at +[anything.notte.cc](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=mintlify) +and it gets built for you. Agents can search the same catalogue by pointing an +MCP client at `https://anything.notte.cc/mcp`. diff --git a/examples/uber/README.md b/examples/uber/README.md index a6da0d91..c6a4ea5e 100644 --- a/examples/uber/README.md +++ b/examples/uber/README.md @@ -319,3 +319,25 @@ For questions or issues, please open an issue in the repository. **Generated**: 2025-12-22 **HAR Analysis ID**: 0c786aa76c2f + +--- + +## Don't want to maintain this? + +This client was reverse-engineered locally and is yours to keep — but it is +pinned to uber.com's API as it looked on the day of capture, and nothing here +re-engineers it when that changes. + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=uber) +is the hosted version of this project: you describe the task and get back a +deployed API function, with the proxies, retries, and repair-on-change handled +for you. Check whether Uber's job board is already covered before building anything: + +```bash +reverse-api-engineer marketplace search --site uber.com +``` + +If nothing matches yet, describe the task at +[anything.notte.cc](https://anything.notte.cc?utm_source=rae&utm_medium=example&utm_campaign=uber) +and it gets built for you. Agents can search the same catalogue by pointing an +MCP client at `https://anything.notte.cc/mcp`. diff --git a/website/content/docs/cloud.mdx b/website/content/docs/cloud.mdx new file mode 100644 index 00000000..9ee465df --- /dev/null +++ b/website/content/docs/cloud.mdx @@ -0,0 +1,107 @@ +--- +title: Hosted version +description: When to use Anything, the managed version of Reverse API Engineer, and how the CLI talks to its marketplace. +--- + +[Anything](https://anything.notte.cc?utm_source=rae&utm_medium=docs&utm_campaign=cloud_page) +is the hosted version of this project. Same job — *describe the task, we +engineer the skill* — but you get back a deployed API function instead of a +file on your disk. + +Reverse API Engineer is not going anywhere, and it is not a trial. It stays +local, MIT, and account-free, and the client it generates is yours to keep. +The two just fail differently, and the difference is who owns the maintenance. + +## Which one you want + +| | Reverse API Engineer | Anything | +| ------------------ | --------------------- | --------------------------- | +| You get | a client in your repo | a callable endpoint | +| Runs on | your machine | hosted browsers | +| When the site changes | you re-run the capture | re-engineered for you | +| Setup | install the CLI | an API key | +| Cost | your model tokens | per run | +| Works offline | yes | no | +| Authenticated sites | your own session | your credentials, hosted | + +Stay local when the client belongs in your repository, when it has to run +inside your own network, when the target needs your logged-in browser session, +or when you want to read every line before it runs. + +Go hosted when you want an endpoint rather than a file, when you would rather +not own the repair work each time a site shifts, or when an agent needs to +reach hundreds of sites without you building each one by hand. + +## Check before you capture + +A capture run costs time and tokens, and the marketplace already holds +hundreds of functions. So in agent mode the CLI looks first: + +``` + anything already has 3 functions for nfl.com + · get_current_nfl_week (8 runs) + Returns the NFL season, week number, season and week types... + https://anything.notte.cc/marketplace/b9d2b07c-935c-44d6-84af-706f786f3455 + +? Open the marketplace instead of capturing? (y/N) +``` + +Press enter and the capture proceeds as normal. It is an offer, not a gate. + +The lookup is anonymous — the marketplace search endpoint is public, so no key +or account is involved — and it only ever reports functions whose own domain +matches your target. The endpoint ranks rather than filters, so a site it has +never seen still returns other people's functions; the CLI discards those +rather than showing you an irrelevant suggestion. + + + Set `RAE_NO_CLOUD=1` to disable the network call entirely, or switch + **Cloud Suggestions** off in `/settings`. The environment variable wins over + the setting, so CI and scripted wrappers can opt out without touching a + user's config file. The lookup is also skipped whenever the CLI is + non-interactive or headless, so `--json`, `--json-stream`, and + `--no-interactive` are unaffected. + + +## Searching from the CLI + +You can query the marketplace directly, without starting a capture: + +```bash +# Everything that covers one site. Only exact domain matches are returned. +reverse-api-engineer marketplace search --site https://www.nfl.com + +# Or search by what you want to do. +reverse-api-engineer marketplace search "nfl standings" + +# Scriptable. +reverse-api-engineer marketplace search instagram --json | jq '.results[].label' +``` + +`/cloud` inside the REPL prints the same set of links. + +## From an agent + +Point any MCP client at the hosted endpoint and your agent searches the +catalogue itself, before it builds anything: + +```json +{ + "mcpServers": { + "anything": { + "url": "https://anything.notte.cc/mcp" + } + } +} +``` + +The server exposes `search` without authentication, plus `spec`, `run`, and +`build` once a key is supplied. `build` takes a plain-English description and +deploys a new function — the hosted equivalent of a capture run. + +## Worked comparison + +The [`examples/hosted`](https://github.com/kalil0321/reverse-api-engineer/tree/main/examples/hosted) +folder runs the same job all three ways — local capture, hosted over HTTP, and +hosted over MCP — with two dependency-free scripts you can run yourself. +`discover.py` needs no credentials at all. diff --git a/website/content/docs/index.mdx b/website/content/docs/index.mdx index 383836da..67ff5546 100644 --- a/website/content/docs/index.mdx +++ b/website/content/docs/index.mdx @@ -19,6 +19,7 @@ engineering: browse, capture, and get clean API code. + ## What you can do with it @@ -31,6 +32,13 @@ can map that backend for you. The generated client wraps the same endpoints your browser called, with auth, pagination, and error handling already wired up. +### Check whether it already exists + +The [hosted version](/docs/cloud) publishes a marketplace of ready-made API +functions, and agent mode searches it for your target site before spending a +capture run. When something matches you get told; pressing enter carries on +capturing anyway. The lookup is anonymous and can be turned off entirely. + ### Where it fits Use it when the browser already knows the workflow, but the API is undocumented diff --git a/website/content/docs/meta.json b/website/content/docs/meta.json index 4cea1841..ea355c28 100644 --- a/website/content/docs/meta.json +++ b/website/content/docs/meta.json @@ -8,6 +8,7 @@ "modes", "cli", "---Reference---", - "configuration" + "configuration", + "cloud" ] } diff --git a/website/content/docs/modes/agent.mdx b/website/content/docs/modes/agent.mdx index 193ae643..6c17d869 100644 --- a/website/content/docs/modes/agent.mdx +++ b/website/content/docs/modes/agent.mdx @@ -27,6 +27,13 @@ reverse-api-engineer > click on the first job listing ``` + + Before the capture starts, agent mode searches the + [hosted marketplace](/docs/cloud) for your target site and lists anything + that already covers it. Press enter to carry on capturing. The lookup is + anonymous, and `RAE_NO_CLOUD=1` disables it. + + ## Running non-interactively ```bash diff --git a/website/src/app/(home)/page.tsx b/website/src/app/(home)/page.tsx index 20e23365..569c9bfc 100644 --- a/website/src/app/(home)/page.tsx +++ b/website/src/app/(home)/page.tsx @@ -6,6 +6,7 @@ import { appName, appTagline, cloudHost, cloudLink, cloudUrl, gitConfig, githubU import { InstallCommand } from '@/components/install-command'; import { BuiltInTheOpen } from '@/components/built-in-the-open'; import { WorksWithAgents } from '@/components/works-with-agents'; +import { LocalOrHosted } from '@/components/local-or-hosted'; import { Reveal } from '@/components/reveal'; import { StepBrowse, StepCapture, StepGenerate, StepReview } from '@/components/step-illustrations'; import { JsonLd } from '@/components/json-ld'; @@ -81,6 +82,7 @@ export default function HomePage() { + ); diff --git a/website/src/components/local-or-hosted.tsx b/website/src/components/local-or-hosted.tsx new file mode 100644 index 00000000..ff6c08bc --- /dev/null +++ b/website/src/components/local-or-hosted.tsx @@ -0,0 +1,124 @@ +import Link from 'next/link'; +import { ArrowRightIcon } from 'lucide-react'; +import { cloudHost, cloudLink, cloudMarketplaceStats, cloudMarketplaceUrl, cloudUrl } from '@/lib/shared'; +import { Reveal } from './reveal'; + +/* Two index cards, pinned side by side. The comparison is deliberately even — + the local column is the product this site is for, and overselling the hosted + one here would read as a bait-and-switch to exactly the audience that + installs an MIT CLI. */ + +const ROWS = [ + { label: 'You get', local: 'a client in your repo', hosted: 'a callable endpoint' }, + { label: 'Runs on', local: 'your machine', hosted: 'hosted browsers' }, + { label: 'Site changes', local: 'you re-run the capture', hosted: 're-engineered for you' }, + { label: 'Setup', local: 'install the CLI', hosted: 'an API key' }, + { label: 'Offline', local: 'yes', hosted: 'no' }, +]; + +function Card({ + eyebrow, + title, + values, + accent, + rotate, + children, +}: { + eyebrow: string; + title: string; + values: string[]; + accent: string; + rotate: string; + children: React.ReactNode; +}) { + return ( +
+ +

{eyebrow}

+

+ {title} +

+ +
+ {ROWS.map((row, i) => ( +
+
{row.label}
+
{values[i]}
+
+ ))} +
+ +
{children}
+
+ ); +} + +export function LocalOrHosted() { + return ( +
+
+
+

Run it, or don't.

+

+ Same job, two form factors. The difference is who owns it when the site changes. +

+
+ + +
+ r.local)} + > + + Get started + + + + + r.hosted)} + > + + {cloudHost} + + + +
+ + +

+ {cloudMarketplaceStats.functions} functions across {cloudMarketplaceStats.sites} sites are already built —{' '} + + check before you capture + + . +

+
+
+ ); +} From 892f6426747a9255183b1a10fe9d8d189ca7d25e Mon Sep 17 00:00:00 2001 From: Lucas Giordano Date: Thu, 20 Aug 2026 13:48:56 +0200 Subject: [PATCH 2/2] docs: use server-side base_url scoping in the hosted example Follows the search endpoint gaining `base_url`. `discover.py` now scopes with it instead of over-fetching and filtering by domain locally, and the docs drop the "the endpoint ranks rather than filters" caveat, which no longer holds. Co-Authored-By: Claude Opus 5 (1M context) --- examples/hosted/README.md | 7 ++++--- examples/hosted/discover.py | 35 ++++++++++++++++++---------------- website/content/docs/cloud.mdx | 13 ++++++++----- 3 files changed, 31 insertions(+), 24 deletions(-) diff --git a/examples/hosted/README.md b/examples/hosted/README.md index 2ad35b54..636fd838 100644 --- a/examples/hosted/README.md +++ b/examples/hosted/README.md @@ -52,9 +52,10 @@ python discover.py nfl.com https://anything.notte.cc/marketplace/365309fa-acb6-4226-b410-9a5f86fb72d9 ``` -`discover.py` filters by domain rather than trusting the ranking, because the -endpoint ranks instead of filtering: query a site it has never seen and it -still returns a page of other people's functions. +`discover.py` passes the site straight through as `base_url`, which scopes +the search server-side. Any form works — a bare hostname, a full URL, or a +glob like `*.nfl.*` — and subdomains are covered. Add a query alongside it to +narrow further: the two filters compose. Then run one. This part needs a key from [console.notte.cc](https://console.notte.cc): diff --git a/examples/hosted/discover.py b/examples/hosted/discover.py index 0bd481f4..e7852dde 100644 --- a/examples/hosted/discover.py +++ b/examples/hosted/discover.py @@ -17,29 +17,32 @@ MARKETPLACE_URL = "https://anything.notte.cc/marketplace" -def search(query: str, limit: int = 5) -> list[dict]: - """Return marketplace functions matching `query`. +def search(query: str | None = None, base_url: str | None = None, limit: int = 5) -> list[dict]: + """Return marketplace functions matching `query`, `base_url`, or both. - The endpoint ranks rather than filters, so a query it cannot match still - comes back full of unrelated functions. Filter by domain yourself when you - are asking "does this specific site have coverage?" — see - `functions_for_site` below. + `base_url` scopes to one site and understands any form it is written in: + a bare hostname, a full URL, or a glob like `*.nfl.*`. `query` ranks + within whatever is left. The two compose. """ - url = f"{SEARCH_ENDPOINT}?{urlencode({'q': query, 'limit': limit})}" + params = {"limit": limit} + if query: + params["q"] = query + if base_url: + params["base_url"] = base_url + + url = f"{SEARCH_ENDPOINT}?{urlencode(params)}" with urlopen(Request(url, headers={"User-Agent": "anything-example"}), timeout=15) as response: payload = json.load(response) return payload.get("results", []) -def functions_for_site(domain: str, limit: int = 5) -> list[dict]: - """Return only the functions that genuinely belong to `domain`.""" - domain = domain.lower().removeprefix("www.") - matches = [ - fn - for fn in search(domain, limit=20) - if (fn.get("domain") or "").lower() == domain or (fn.get("domain") or "").lower().endswith(f".{domain}") - ] - matches.sort(key=lambda fn: fn.get("run_count", 0), reverse=True) +def functions_for_site(site: str, limit: int = 5) -> list[dict]: + """Return the functions that belong to one site, most-run first.""" + matches = sorted( + search(base_url=site, limit=limit), + key=lambda fn: fn.get("run_count", 0), + reverse=True, + ) return matches[:limit] diff --git a/website/content/docs/cloud.mdx b/website/content/docs/cloud.mdx index 9ee465df..374c6689 100644 --- a/website/content/docs/cloud.mdx +++ b/website/content/docs/cloud.mdx @@ -49,10 +49,9 @@ hundreds of functions. So in agent mode the CLI looks first: Press enter and the capture proceeds as normal. It is an offer, not a gate. The lookup is anonymous — the marketplace search endpoint is public, so no key -or account is involved — and it only ever reports functions whose own domain -matches your target. The endpoint ranks rather than filters, so a site it has -never seen still returns other people's functions; the CLI discards those -rather than showing you an irrelevant suggestion. +or account is involved — and it only ever reports functions that belong to your +target site. The endpoint scopes by site server-side, and the CLI re-checks the +domain on the way back, because an irrelevant suggestion is worse than none. Set `RAE_NO_CLOUD=1` to disable the network call entirely, or switch @@ -68,12 +67,16 @@ rather than showing you an irrelevant suggestion. You can query the marketplace directly, without starting a capture: ```bash -# Everything that covers one site. Only exact domain matches are returned. +# Everything that covers one site. A full URL, bare host, or glob all work. reverse-api-engineer marketplace search --site https://www.nfl.com # Or search by what you want to do. reverse-api-engineer marketplace search "nfl standings" +# Filters compose, and categories are their own axis. +reverse-api-engineer marketplace search "standings" --site nfl.com +reverse-api-engineer marketplace search --category Jobs + # Scriptable. reverse-api-engineer marketplace search instagram --json | jq '.results[].label' ```