Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@ jobs:
strategy:
fail-fast: false
matrix:
node-version: [18, 20, 22]
# 22 is the floor: the real-time WebSocket client uses the global
# WebSocket, which Node ships from 22.
node-version: [22, 24]
steps:
- uses: actions/checkout@v4

Expand Down
109 changes: 105 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
[![npm version](https://img.shields.io/npm/v/newsdata-nodejs-client?logo=npm&color=cb3837)](https://www.npmjs.com/package/newsdata-nodejs-client)
[![npm downloads](https://img.shields.io/npm/dm/newsdata-nodejs-client?color=cb3837)](https://www.npmjs.com/package/newsdata-nodejs-client)
[![CI](https://img.shields.io/github/actions/workflow/status/newsdataapi/newsdata-nodejs-client/ci.yml?branch=main&logo=github&label=CI)](https://github.com/newsdataapi/newsdata-nodejs-client/actions/workflows/ci.yml)
[![Node](https://img.shields.io/badge/node-%3E%3D18-green?logo=node.js)](https://nodejs.org)
[![Node](https://img.shields.io/badge/node-%3E%3D22-green?logo=node.js)](https://nodejs.org)
[![License](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
[![OpenAPI](https://img.shields.io/badge/OpenAPI-3.1-85EA2D)](https://newsdata.io/openapi.json)

Expand All @@ -17,9 +17,12 @@ Official Node.js client for the [Newsdata.io](https://newsdata.io) News API. It
wraps every endpoint (`latest`, `archive`, `sources`, `crypto`, `market`,
`count`, `crypto/count`, `market/count`) with client-side parameter validation,
automatic retries with exponential backoff, scroll/paginate helpers, and a
typed error hierarchy.
typed error hierarchy. It also covers the real-time WebSocket service end to
end with `NewsDataApiWebSocket`: register, list, and delete queries, and stream
the matching news as it is published.

Zero runtime dependencies — uses the built-in `fetch` (Node 18+).
Zero runtime dependencies — uses the built-in `fetch` and `WebSocket`
(Node 22+).

## Installation

Expand Down Expand Up @@ -66,6 +69,9 @@ const { NewsDataApiClient } = await import('newsdata-nodejs-client');
| `countApi(params)` | `/1/count` | Aggregate counts (requires `from_date`, `to_date`) |
| `cryptoCountApi(params)` | `/1/crypto/count` | Aggregate crypto counts (requires dates) |
| `marketCountApi(params)` | `/1/market/count` | Aggregate market counts (requires dates) |
| `websocketRegister(params)` | `/1/websocket/register` | Register a real-time query |
| `websocketFetch()` | `/1/websocket/fetch` | List registered queries |
| `websocketDelete(id)` | `/1/websocket/delete` | Delete a registered query |

Each `params` value may be a single value or an array (arrays are sent
comma-separated). Parameter names are case-insensitive. See the
Expand Down Expand Up @@ -99,6 +105,99 @@ await client.latestApi({ rawQuery: 'q=bitcoin&country=us&language=en' });
`rawQuery` is mutually exclusive with all other parameters and is validated
against the endpoint's allowed keys.

## Real-time news (WebSocket)

Register a query first — the returned `registration_id` identifies it from then on:

```js
import { NewsDataApiClient, NewsDataApiWebSocket } from 'newsdata-nodejs-client';

const client = new NewsDataApiClient('YOUR_API_KEY');
const ws = new NewsDataApiWebSocket(client);

const { results } = await ws.websocketRegister({ q: 'bitcoin', language: 'en' });
const registrationId = results.registration_id;
```

`websocketRegister` takes the familiar filter parameters (`q`, `country`,
`language`, `domain`, …) — no date or paging filters, since a registered query
matches news as it is published. Registering an identical query twice rejects
with a `NewsdataApiError` whose `statusCode` is 409; the existing id is at
`err.responseBody.results.registration_id`. `websocketFetch()` lists every
registered query and `websocketDelete(id)` removes one.

Then stream — each response has the familiar `status` / `totalResults` /
`results` shape:

```js
for await (const response of ws.stream(registrationId)) {
for (const article of response.results) {
console.log(article.title, '-', article.link);
}
}
```

Break out of the loop to stop; the connection closes either way. `ws.close()`
ends an in-flight stream from outside the loop.

Transient drops (network errors, server restarts, abnormal closes) are
reconnected automatically with a capped exponential backoff. Pass
`reconnect: false` to stop on the first disconnect instead. A permanent
rejection — bad API key or unknown
`registration_id`, exhausted API credits, or too many simultaneous devices — throws
`NewsdataWebSocketAuthError` and is **not** retried.

The server always accepts the handshake and then closes with code **1008** when
the connection is refused, carrying one of three reasons: `invalid credentials
or registration not found`, `api limit reached`, or `device limit reached` (more
than 5 devices on one `registration_id`). Every other close code — including
`1013` (`send timeout`, meaning the client read too slowly) — is transient and
reconnects.

**Each delivered article consumes 1 API credit per connected device.**

Catch it like any other client error:

```js
import {
NewsdataWebSocketAuthError,
NewsdataWebSocketError,
} from 'newsdata-nodejs-client';

try {
for await (const response of ws.stream(registrationId)) {
// ...
}
} catch (err) {
if (err instanceof NewsdataWebSocketAuthError) console.error('rejected:', err.message);
else if (err instanceof NewsdataWebSocketError) console.error('stream error:', err.message);
else throw err;
}
```

All connection options are optional:

```js
const ws = new NewsDataApiWebSocket(client, {
baseUrl: 'wss://ws.newsdata.io/ws/event', // staging / self-hosted / proxied
reconnect: true, // auto-reconnect on transient drops; default true
reconnectDelay: 1000, // ms before the first reconnect (doubles each retry)
reconnectDelayMax: 30000, // cap on the reconnect delay
openTimeout: 10000, // ms to wait for the opening handshake
WebSocket: undefined, // override the implementation (default: global WebSocket)
});
```

> **Node 22+.** Streaming uses the global `WebSocket`, which Node ships from
> v22. On older runtimes pass your own implementation, e.g.
> `new NewsDataApiWebSocket(client, { WebSocket: require('ws') })`.
>
> Node's global `WebSocket` does not expose the handshake HTTP status. When a
> connection fails before it opens, the client probes the same URL over HTTP to
> tell a permanent rejection (401 / 403) from a transient failure.

Runnable example: [`examples/websocket.js`](examples/websocket.js).

## Client-side validation

Before any request is sent, parameters are validated and normalized. A
Expand Down Expand Up @@ -146,7 +245,9 @@ NewsdataError (catch-all base)
│ ├── NewsdataAuthError (401 / 403)
│ ├── NewsdataRateLimitError (429; .retryAfter)
│ └── NewsdataServerError (5xx)
└── NewsdataNetworkError (.cause)
├── NewsdataNetworkError (.cause)
└── NewsdataWebSocketError (real-time stream)
└── NewsdataWebSocketAuthError (policy-violation close 1008)
```

## Configuration
Expand Down
72 changes: 72 additions & 0 deletions examples/websocket.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
// Real-time news streaming.
//
// NEWSDATA_API_KEY=<your key> node examples/websocket.js
//
// Requires Node 22+ for the global WebSocket.
//
// Articles are matched by a registered query. If NEWSDATA_REGISTRATION_ID is
// set, that query is streamed directly; otherwise the example registers a demo
// query (q="pizza") first and prints the resulting registration_id so you can
// reuse it on the next run — or remove it later with websocketDelete().

import {
NewsDataApiClient,
NewsDataApiWebSocket,
NewsdataApiError,
NewsdataWebSocketAuthError,
} from '../src/index.js';

const apiKey = process.env.NEWSDATA_API_KEY;
if (!apiKey) {
console.error('Set NEWSDATA_API_KEY in your environment before running this example.');
process.exit(1);
}

const client = new NewsDataApiClient(apiKey);
const ws = new NewsDataApiWebSocket(client);

/**
* Register q="pizza" and return its registration_id. Registering an identical
* query again answers HTTP 409 with the existing id in the response body —
* reuse it instead of failing.
*/
async function registerDemoQuery() {
try {
const { results } = await ws.websocketRegister({ q: 'pizza' });
console.log(`registered demo query q="pizza" -> ${results.registration_id}`);
return results.registration_id;
} catch (err) {
if (err instanceof NewsdataApiError && err.statusCode === 409) {
const existing = err.responseBody?.results?.registration_id;
if (existing) {
console.log(`query already registered; reusing ${existing}`);
return existing;
}
}
throw err;
}
}

const registrationId = process.env.NEWSDATA_REGISTRATION_ID ?? await registerDemoQuery();

// Stop cleanly on Ctrl-C.
process.on('SIGINT', () => {
console.log('\nstopping');
ws.close();
});

console.log(`streaming ${registrationId} — Ctrl-C to stop`);

try {
for await (const response of ws.stream(registrationId)) {
for (const article of response.results ?? []) {
console.log(`${article.title} - ${article.link}`);
}
}
} catch (err) {
if (err instanceof NewsdataWebSocketAuthError) {
console.error(`rejected: ${err.message}`);
process.exit(1);
}
throw err;
}
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "newsdata-nodejs-client",
"version": "0.1.0",
"description": "Official Node.js client (SDK) for the Newsdata.io News API fetch real-time, historical, crypto, and stock-market news via REST with validation, retries, pagination, and typed errors.",
"description": "Official Node.js client (SDK) for the Newsdata.io News API \u2014 fetch real-time, historical, crypto, and stock-market news via REST with validation, retries, pagination, and typed errors.",
"type": "module",
"main": "src/index.js",
"types": "types/index.d.ts",
Expand All @@ -18,7 +18,7 @@
"LICENSE"
],
"engines": {
"node": ">=18"
"node": ">=22"
},
"scripts": {
"test": "node --test"
Expand Down
84 changes: 73 additions & 11 deletions src/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
import {
BASE_URL,
ENDPOINTS,
ENDPOINT_METHODS,
RESULTS_OPTIONAL,
WS_NEWS_TYPE,
DEFAULT_REQUEST_TIMEOUT,
DEFAULT_MAX_RETRIES,
DEFAULT_RETRY_BACKOFF,
Expand Down Expand Up @@ -124,6 +127,67 @@ export class NewsDataApiClient {
return this.#request('sources', validated);
}

// ---- real-time query management ---------------------------------------

/**
* Register a real-time WebSocket query. POST /1/websocket/register
*
* Takes the familiar filter names (`q`, `country`, `language`, `domain`, …);
* no date or paging filters apply, since a registered query matches news as
* it is published. The new query's id is at `results.registration_id` — pass
* it to `NewsDataApiWebSocket#stream`.
*
* Registering an identical query twice rejects with a `NewsdataApiError`
* whose `statusCode` is 409; the existing id is at
* `err.responseBody.results.registration_id`.
* @returns {Promise<object>}
*/
websocketRegister(params = {}) {
const { rawQuery = null, ...rest } = params;
const validated = validateParams('websocket_register', rest, rawQuery);
validated.news_type = WS_NEWS_TYPE;
return this.#request('websocket_register', validated);
}

/**
* List the account's registered real-time queries. GET /1/websocket/fetch
* One entry per query at `results.queries`.
* @returns {Promise<object>}
*/
websocketFetch() {
return this.#request('websocket_fetch', {});
}

/**
* Delete a registered real-time query. DELETE /1/websocket/delete
* @param {string} registrationId
* @returns {Promise<object>}
*/
websocketDelete(registrationId) {
if (typeof registrationId !== 'string' || registrationId === '') {
throw new NewsdataValidationError(
'registrationId must be a non-empty string',
'registration_id',
);
}
return this.#request('websocket_delete', { registration_id: registrationId });
}

/** The API key, for the WebSocket handshake URL. @internal */
get apiKeyForWebSocket() {
return this.#apiKey;
}

/** The configured fetch, reused by the WebSocket handshake probe. @internal */
get fetchForWebSocket() {
return this.#fetch;
}

/** Forward a log line from the WebSocket layer. @internal */
logForWebSocket(level, message) {
this.#log(level, message);
}

// ---- dispatch ---------------------------------------------------------

/**
Expand Down Expand Up @@ -179,16 +243,17 @@ export class NewsDataApiClient {
const search = new URLSearchParams({ ...params, apikey: this.#apiKey });
const fullUrl = `${this.#endpointUrl(endpoint)}?${search.toString()}`;
const logUrl = redactApiKey(fullUrl);
const method = ENDPOINT_METHODS[endpoint] ?? 'GET';

for (let attempt = 1; attempt <= this.#maxRetries; attempt += 1) {
this.#log('info', `GET ${logUrl} (attempt ${attempt}/${this.#maxRetries})`);
this.#log('info', `${method} ${logUrl} (attempt ${attempt}/${this.#maxRetries})`);

let res;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.#timeout);
try {
res = await this.#fetch(fullUrl, {
method: 'GET',
method,
headers: { Accept: 'application/json' },
signal: controller.signal,
});
Expand Down Expand Up @@ -220,7 +285,7 @@ export class NewsDataApiClient {
throw new NewsdataApiError(`Non-JSON response from API (status ${status})`, status);
}

if (status === 200 && this.#isSuccess(body)) {
if (status === 200 && this.#isSuccess(body, endpoint)) {
if (this.#includeHeaders) {
body.responseHeaders = Object.fromEntries(res.headers.entries());
}
Expand Down Expand Up @@ -325,14 +390,11 @@ export class NewsDataApiClient {
}
}

#isSuccess(body) {
return (
body
&& typeof body === 'object'
&& body.status === 'success'
&& body.results !== null
&& body.results !== undefined
);
#isSuccess(body, endpoint) {
if (!body || typeof body !== 'object' || body.status !== 'success') return false;
// The websocket management endpoints may answer without a `results` field.
if (RESULTS_OPTIONAL.includes(endpoint)) return true;
return body.results !== null && body.results !== undefined;
}

#errorMessage(body, status) {
Expand Down
Loading
Loading