Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/assets/logo/dark.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions .github/assets/logo/light.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
.idea
node_modules
dist
coverage
83 changes: 78 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
# ZenRows Node.js SDK
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset=".github/assets/logo/dark.svg"/>
<img alt="Zenrows Logo" src=".github/assets/logo/light.svg" width="300" />
</picture>
</p>

SDK to access [ZenRows](https://www.zenrows.com/) API directly from Node.js. ZenRows handles proxies rotation, headless browsers, and CAPTCHAs for you.
# Zenrows Node.js SDK

SDK to access [Zenrows](https://www.zenrows.com/) API directly from Node.js. Zenrows handles proxies rotation, headless browsers, and CAPTCHAs for you.

## Installation

Expand All @@ -21,6 +28,8 @@ The SDK uses the official [fetch api](https://nodejs.org/dist/latest-v18.x/docs/

It also uses [fetch-retry](https://github.com/jonbern/fetch-retry) to automatically retry failed requests (status code 429 and 5XX). Retries are not active by default; you need to specify the number of retries, as shown below. It already includes an exponential back-off retry delay between failed requests.

> `client.fetch()` is the primary method for the main page-scraping product. `client.get()` still works and is kept as a deprecated alias — new code should call `fetch()`.

```javascript
const { ZenRows } = require("zenrows");

Expand All @@ -30,7 +39,7 @@ const url = "https://www.zenrows.com/";
(async () => {
const client = new ZenRows(apiKey, { retries: 1 });

const response = await client.get(
const response = await client.fetch(
url,
{
// Our algorithm allows to automatically extract content from any website
Expand Down Expand Up @@ -136,6 +145,70 @@ const url = "https://httpbin.org/anything";
})();
```

### Extract

[Extract](https://docs.zenrows.com) (beta) runs a page through Zenrows' AI-powered structured extraction instead of returning raw HTML. Use `client.extract()` — it's the same request as `fetch()`, with the `extract` param set for you (defaults to `"auto"`; pass `"native"` or `"standard"` for the other contracts).

```javascript
const { ZenRows } = require("zenrows");

const apiKey = "YOUR-API-KEY";
const url = "https://www.zenrows.com/";

(async () => {
const client = new ZenRows(apiKey);

const response = await client.extract(url); // extract: "auto"
// const response = await client.extract(url, { extract: "native" });

const data = await response.json();
console.log(data);
})();
```

### Batch

The [Batch API](https://docs.zenrows.com) (beta) runs many URLs asynchronously as a job, and is reachable via `client.batch`. A job is either **open** — it keeps accepting tasks via `addTasks()` until you `closeJob()` it, useful when you're streaming URLs in over time — or a normal one-shot batch where every task is known upfront.

```javascript
const { ZenRows } = require("zenrows");

const apiKey = "YOUR-API-KEY";

(async () => {
const client = new ZenRows(apiKey);

// One-shot batch: every task known upfront.
const job = await client.batch.submitJob({
tasks: [{ url: "https://example.com/1" }, { url: "https://example.com/2" }],
});

// Streaming batch: keep the job open, add tasks as they arrive, close when done.
const streamingJob = await client.batch.submitJob({ status: "open" });
await client.batch.addTasks(streamingJob.job_id, [{ url: "https://example.com/3" }]);
// Or close it as part of the last addTasks() call instead of a separate request:
// await client.batch.addTasks(streamingJob.job_id, [...], { lastBatch: true });
await client.batch.closeJob(streamingJob.job_id);

// Poll for progress and page through results once it's done.
const status = await client.batch.getJob(job.job_id);
const { results, next_cursor } = await client.batch.getResults(job.job_id);

// Retry only the tasks that failed, inheriting the rest from the previous run.
await client.batch.rerun(job.job_id, { status: "failed" });
})();
```

`client.batch` also exposes `listJobs()`, `deleteJob()`, `stopRun()`, `rerun()`, `listRuns()`, `getRun()`, `deleteRun()`, and `getTaskContent()` (returns the scraped page's raw content as a string, not JSON — the endpoint can return HTML or plain text depending on what the target page served). Scheduling, webhook config, HMAC key rotation, CSV task uploads, and results exports aren't wrapped yet — call the [Batch API](https://docs.zenrows.com) directly for those.

The batch client (`ZenRowsBatchClient`) also works standalone, without a `ZenRows` instance — matching the Go and Python SDKs' batch clients:

```javascript
const { ZenRowsBatchClient } = require("zenrows");

const batch = new ZenRowsBatchClient(apiKey, { baseURL: "https://async.api.zenrows.com/v1" }); // baseURL is optional
```

### Concurrency

To limit the concurrency, it uses [fastq](https://github.com/mcollina/fastq), which will simultaneously send a maximum of requests. The concurrency is determined by the plan you are in, so take a look at the [pricing](https://www.zenrows.com/pricing) and set it accordingly. Take into account that each client instance will have its own limit, meaning that two different scripts will not share it, and 429 (Too Many Requests) errors might arise.
Expand All @@ -155,7 +228,7 @@ const apiKey = "YOUR-API-KEY";
// ...
];

const promises = urls.map((url) => client.get(url));
const promises = urls.map((url) => client.fetch(url));

const results = await Promise.allSettled(promises);
console.log(results);
Expand All @@ -182,7 +255,7 @@ const apiKey = "YOUR-API-KEY";
`Promise.allSettled()` does not narrow the type of the array elements in the callback function. This means that you will need to cast the type of the array elements to `PromiseSettledResult<Response>` to access the `status` and `value` properties.

```typescript
const promises = urls.map((url) => client.get(url));
const promises = urls.map((url) => client.fetch(url));

const results = await Promise.allSettled(promises);

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"@biomejs/biome": "1.8.1",
"@types/async-retry": "^1.4.9",
"@types/node": "^20.19.39",
"@vitest/coverage-v8": "1.6.1",
"msw": "^2.13.4",
"tsup": "^8.5.1",
"typescript": "^5.9.3",
Expand Down
Loading
Loading