diff --git a/README.md b/README.md index 57364e0..9dd4dc0 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # AgentCLI -**One CLI. Any MCP server.** Call MCP servers from the command line — dynamically, with no per-tool wrapper code. +**One CLI. Any tool backend.** Call MCP servers and OpenAPI (REST) APIs from the command line — dynamically, with no per-tool wrapper code. > ```bash > $ agentcli github search_issues --repo apache/hertzbeat --query "memory leak" @@ -9,11 +9,18 @@ > "meta": { "durationMs": 412, "via": "daemon" } } > ``` -Every MCP tool becomes a CLI command the first time you type it. Flags are compiled from the tool's JSON Schema on the fly. +Every MCP tool — and every OpenAPI operation — becomes a CLI command the first time you type it. Flags are compiled from the tool's JSON Schema on the fly. ## Why -Agents that register every MCP tool upfront pay for it in context: hundreds of schemas loaded at startup, most never used. A CLI flips the model to **progressive disclosure**: discover with `--help`, drill down only as far as needed. stdout/stderr separation, a binary exit code (0/1), and self-describing JSON errors keep the whole thing machine-consumable. +Agents that register every tool upfront pay for it in context: hundreds of schemas loaded at startup, most never used. A CLI flips the model to **progressive disclosure**: discover with `--help`, drill down only as far as needed. stdout/stderr separation, a binary exit code (0/1), and self-describing JSON errors keep the whole thing machine-consumable. + +The same loop works for both backend kinds: + +| Backend | Register | Tools come from | +|---|---|---| +| MCP (stdio / HTTP) | `agentcli server add -- ` or `--url` | `tools/list` over the protocol | +| OpenAPI 3.x | `agentcli server add --openapi ` | the spec — every operation becomes a tool | ## Install @@ -23,7 +30,7 @@ pnpm add -g @happyvibing/agentcli # or: npm install -g @happyvibing/agentcli Requires Node >= 20.10. Or run from source: `git clone && pnpm install && pnpm build && pnpm link --global`. -## Quick start +## Quick start (MCP) ```bash # 1. Register a server @@ -68,10 +75,50 @@ Global: --schema Print the raw tool input schema ``` +## OpenAPI APIs + +Register an OpenAPI 3.x JSON spec and every operation becomes a CLI command — same discovery, same flags, same output envelope: + +```bash +# Register (the spec is snapshotted locally; --refresh re-pulls it) +agentcli server add petstore --openapi https://petstore3.swagger.io/api/v3/openapi.json \ + --header "Authorization: Bearer ${PETSTORE_TOKEN}" # ${ENV} expands at call time + +# Discover — operations grouped by their spec tags +agentcli petstore -h + +# Execute — path/query params and JSON body properties are all flags +agentcli petstore getPetById --petId 1 +agentcli petstore addPet --name rex --kind dog +``` + +How an OpenAPI spec maps onto the CLI: + +- **operationId** becomes the tool name (missing ones get a mechanical slug like `get_pets_petId`) +- path / query / header parameters and **request-body properties flatten into flags** — `POST /pets {name, tag}` is `--name x --tag y` +- HTTP failures map to the same error codes: 401/403 → `AUTH_REQUIRED`, 404 → `NOT_FOUND`, other 4xx/5xx → `EXECUTION_ERROR` with `httpStatus` in details +- OpenAPI calls are stateless HTTP — `meta.via` is always `direct` (no daemon involved) + +Swagger 2.0 and YAML specs are rejected with an upgrade/conversion hint. Local spec files work too (`--openapi ./api.json`) — useful for internal APIs. See [`examples/`](examples/) for a 30-second walkthrough against a real, auth-free API. + +## Architecture + +The core only knows "tools + JSON Schema". Backends produce tool definitions; everything below (flag compiler, help, output, caching, errors) is shared: + +``` +agentcli --flags + │ + dispatch (backend-agnostic) + │ ToolDef { name, description, inputSchema } + ┌──────┴──────┐ + McpBackend OpenApiBackend + daemon/direct snapshot → compile → fetch +``` + ## Agent Skill `skills/agentcli/SKILL.md` teaches an agent this runtime — the discover → inspect → execute → observe loop and self-describing error recovery. Install it with the skills CLI (also listed on [skills.sh](https://skills.sh)): ```bash npx skills add happyvibing/agentcli@agentcli -g -``` +``` \ No newline at end of file diff --git a/docs/design-openapi-adapter.md b/docs/design-openapi-adapter.md new file mode 100644 index 0000000..384f663 --- /dev/null +++ b/docs/design-openapi-adapter.md @@ -0,0 +1,249 @@ +# OpenAPI 统一动态加载适配器 — 设计文档 + +> 状态: 设计稿 (openapi-adapter-design 分支) +> 前置: AgentCLI v0.1.1 已实现 MCP → CLI 的动态转换。本文设计把 OpenAPI (3.x JSON) 纳入同一运行时。 + +## 0. 一句话 + +**CLI 核心只认识 "工具 + JSON Schema";后端负责生产它们。** +MCP 后端从协议里拿工具列表;OpenAPI 后端把 spec 编译成工具列表。往下 (flag 编译 / 帮助 / 执行 / 缓存 / 错误 / 退出码) 全部复用,零分叉。 + +``` + agentcli --flags + │ + dispatch.ts (backend 无关) + │ ToolDef { name, description, inputSchema, tags? } + ┌────────────┴─────────────┐ + McpBackend OpenApiBackend + stdio/http + daemon snapshot → compile → fetch + tools cache (TTL) tools cache (同一套) +``` + +统一的四个锚点: + +| 锚点 | 含义 | +|---|---| +| ToolDef | 同一种工具定义 (JSON Schema inputSchema) → 同一个 flag 编译器 (`flags.ts` 不动) | +| ToolResult | callTool 返回同一种**中立内部结果** `{data, text?, isError?}` — MCP 形状在 McpBackend 边界内一次转换,内部总线不耦合任何协议 | +| Cache | 同一个 `.tools.json` TTL/refresh 语义 | +| Errors | 同一张错误码表 → 同一套 exit code 协议 | + +## 1. 目标 / 非目标 + +**目标** +1. `agentcli server add petstore --openapi ` 之后,`agentcli petstore -h` 列出全部 operation,`agentcli petstore getPetById --petId 1` 直接执行。 +2. 对 dispatch / flags / help / 输出层 **零改动或近零改动** — "统一"由一个 Backend 接口完成,不是新写一个平行 CLI。 +3. 离线友好: spec 快照落盘,`--refresh` 才回源。 +4. 凭据与 MCP 同等隔离: header 值支持 `${ENV_VAR}` 展开,永不落盘。 + +**非目标 (明确不做)** +- Swagger 2.0 (报错并提示升级到 3.x) +- YAML spec (P2; JSON 先行,检测到 `.yaml` 报错给出明确 hint) +- OAuth2 流程、token 刷新 — 只做 header 注入,凭据由用户提供 +- multipart / form body (报错带 hint) +- 响应体按 schema 校验、分页自动翻页 (`--all` 留作 P3) +- OpenAPI → MCP 反向桥接 (架构上已具备 — Backend 产 ToolDef,未来可包一层 MCP server;本文不含) + +## 2. Backend 接口 (统一点) + +```ts +// src/backend/types.ts +export interface Backend { + readonly kind: "mcp" | "openapi"; + listTools(opts: { refresh?: boolean; timeoutMs?: number }): Promise; + callTool(tool: string, args: Record, opts: { timeoutMs?: number; daemon?: boolean }): Promise; +} + +export function openBackend(cfg: AgentCliConfig, name: string): Backend; +// 按 cfg.servers[name].type 分发: "mcp 类" (stdio/http) → McpBackend, "openapi" → OpenApiBackend +``` + +- `McpBackend`: 封装现有 `client.ts` 的 `listTools/callTool`,并在边界内把 MCP 形状结果转换为中立的 `ToolResult` (现 dispatch.ts 里的 `extractData`/`extractText`/`parseMaybeEncoded` 迁入此处 — 那本就是 MCP 特有的解析逻辑)。 +- `OpenApiBackend`: 新增。忽略 `daemon` 选项 (无状态 HTTP,无持久连接需求),`via` 恒为 `"direct"`。 +- **改造点**: `dispatch.ts` 与 `index.ts` 的 `server tools` 改调 `openBackend(...)`;`client.ts` 保持 MCP 专用。 +- `ToolDef` = 现有 `McpTool` 更名 (纯重命名,字段不变,新增可选 `tags?: string[]` 供 help 分组)。 + +内部结果类型与两个后端各自的转换 (MCP 形状**不越过** backend 边界): + +```ts +// src/backend/types.ts — 内部中立契约 +export interface ToolResult { data: unknown; text?: string; isError?: boolean; } +// McpBackend 转换: structuredContent → data;纯文本 content → parseMaybeEncoded(text) → data, text=原文 +// OpenApiBackend 转换: JSON body → data;原文 → text;status≥400 已在后端抛映射后的 AgentCliError +``` +``` + +dispatch 消费 `{data, text, isError}`:`--output json` 打印 data;`--output text` = data 为对象则 pretty-print,否则用 text;isError → `EXECUTION_ERROR`。错误处理统一为**后端抛类型化 AgentCliError**(连接/超时/认证/openapi 的 HTTP 状态码映射),dispatch 只透传 — exit code 协议不变。 + +## 3. 配置层 + +### 3.1 ServerSpec 新变体 + +```ts +export interface OpenApiServerSpec { + type: "openapi"; + spec: string; // 本地快照路径 (add 时已下载/复制),相对 dataDir + origin: string; // 原始来源: URL 或绝对文件路径 (--refresh 回源用) + baseUrl?: string; // 覆盖 spec.servers[0].url + headers?: Record; // 可含 ${ENV_VAR} 占位 +} +``` + +### 3.2 注册: server add + +```bash +agentcli server add petstore --openapi https://petstore3.swagger.io/api/v3/openapi.json \ + [--base-url https://...] [--header "Authorization: Bearer ${PETSTORE_TOKEN}"] +agentcli server add myapi --openapi ./myapi.json # 本地文件同样快照 +``` + +add 时的动作 (fail-fast,注册即验证): +1. 拉/读 spec → 校验 `openapi: "3.x"` → **快照落盘** `~/.agentcli/specs/.json` (0600) +2. 试编译一遍: operation 数、命名冲突、致命 $ref 问题在 add 时就报,不留到调用时 +3. 写 config (`spec` 指向快照,`origin` 记录来源) → 输出 `{ ok, server, operations: N }` + +`--refresh` 语义: 有 origin URL → 重新拉取 + 重新快照 + 重新编译;origin 是本地文件 → 重新复制 + 重编译 (方便用户改了本地 spec)。 + +### 3.3 凭据 + +- header 值里的 `${VAR}` 在 **请求时** 展开;引用了未定义变量 → `AUTH_REQUIRED` (hint: 设置环境变量)。 +- 与 MCP stdio 的 env whitelist 哲学一致: 凭据不进 config 文件明文,不进日志。 +- P2 增强: 读 `components.securitySchemes`,发现 spec 需要 bearer 而配置没给 Authorization → 调用时给一条 helpful hint。 + +## 4. OpenAPI → ToolDef 编译规则 (src/openapi/compile.ts) + +### 4.1 operation → 工具名 + +| 优先级 | 规则 | 例子 | +|---|---|---| +| 1 | `operationId` (做 slug 清洗: 非 `[a-zA-Z0-9_.-]` → `_`) | `getPetById` | +| 2 | 兜底 `METHOD + "_" + pathSlug`: `{param}` → 参数名, `/` → `_` | `GET /pets/{petId}` → `get_pets_petId` | + +- 重名: 确定性自动加后缀 `_2`, `_3`;工具 description 首行标注 `METHOD /path — origin: `,保证 agent 可追溯。 +- description = `operation.summary` + `operation.description`。 + +### 4.2 参数 → inputSchema.properties + +把 path / query / header 三类参数**打平**到同一个顶层 properties (cookie 忽略): + +```jsonc +// GET /pets/{petId}?verbose= → inputSchema: +{ "type": "object", + "required": ["petId"], + "properties": { + "petId": { "type": "integer", "description": "…" }, + "verbose":{ "type": "boolean" } } } +``` + +- path 参数强制 required (spec 没写也补上)。 +- OpenAPI param schema 支持 `schema.$ref` → 解引用后内联 (见 4.4)。 +- query 数组: flag 重复传值 → 序列化为 repeat (`?tags=a&tags=b`,form/explode 默认);`style: csv` 的 spec 少见,P1 按单值字符串透传 (spec 里写了 enum 的照常生成 choices)。 + +### 4.3 requestBody → body 参数打平 + +仅处理 `application/json`: + +| body schema 形状 | 编译结果 | +|---|---| +| object,顶层属性是简单类型 | **属性全部提升为顶层 flags** — `POST /pets` 的 `{name, tag}` 直接 `--name x --tag y` | +| object,含嵌套/复杂属性 | 简单属性照常 flag;复杂属性进 `plan.complex` (→ `--input`),与 MCP 路径完全一致 | +| 顶层 `allOf` | 合并各段的 properties/required 后按上两行处理 (allOf-merge 常见,值得做) | +| array / string / 无 schema | 单个复杂参数 `--input '{"body": [...]}'` 形态,body 键名固定 | +| 其他 content-type | 编译期不报错;调用时若真传了 body → `INVALID_ARGUMENT` + hint | + +命名冲突 (如 path 有 `id`,body 也有 `id`): 冲突键以 body 侧加前缀 `body_` 消解,并在 description 标注原名。编译期不再因冲突失败 — 自动消解优于报错 (真实 spec 里同名不罕见)。 + +### 4.4 $ref 解析 (src/openapi/ref.ts) + +- 只支持 **本地 ref** `#/components/...`;内部 `$ref` 递归内联,**循环检测**: 二次命中同一 ref 路径 → 替换为 `{type: "object", description: "(circular ref)"}` 并视为 complex。 +- 内联深度上限 (如 8) 防深递归炸弹。 +- 外部 ref (`file://`, `http://`) → 编译期 `INVALID_ARGUMENT`,hint 指明哪个 operation 引用了它 (常见于拆分的多文件 spec,P2 再考虑 bundle)。 + +### 4.5 tags + +`operation.tags` 收集进 `ToolDef.tags`。服务级 help (`agentcli petstore -h`) 有 tag 就按 tag 分组显示 (无 tag 的进 `—` 组);工具名保持扁平。解决"一个 spec 500 个 operation 刷屏"的发现问题,命令面不引入层级 (与 MCP 侧一致)。 + +## 5. 执行层 (src/openapi/exec.ts) + +``` +callTool(tool, args) → + 1. 查编译产物 (operation 元数据: method, pathTemplate, paramLocations, bodyInfo, securityHint) + 2. URL = baseUrl 解析 + path 参数替换 ({petId} → String(args.petId)) + 3. query: 非 path/header/body 的参数 → URLSearchParams (数组 repeat) + 4. header params → 请求头; config headers (env 展开) 合并 + 5. body 参数存在 → JSON.stringify + content-type: application/json + 6. fetch (AbortSignal.timeout(--timeout-ms, 默认 60s)) + 7. 包成 MCP 形状 envelope (见 §2) +``` + +### 5.1 HTTP → 错误码映射 + +| 情况 | code | 附加 | +|---|---|---| +| DNS/网络失败 | `CONNECT_FAILED` | — | +| 超时 | `TIMEOUT` | hint 同现有 | +| 401 / 403 | `AUTH_REQUIRED` | — | +| 404 | `NOT_FOUND` | — | +| 429 | `EXECUTION_ERROR` | details 带 `retryAfter` (读 header) | +| 其余 4xx / 5xx | `EXECUTION_ERROR` | details: `{ httpStatus, body }` (body 截断 2KB) | +| 响应非 JSON | data 走 text 路径 | `--output text` 原样输出 | + +错误响应体里若能解出 `message/error` 字段,提升为错误 message 首行 — agent 第一眼看到 API 自己的话术。 + +### 5.2 servers/variables + +`spec.servers[0]` 为基准;URL 模板变量用 variables 默认值展开;`--base-url` (config `baseUrl`) 覆盖一切。多个 server entry 只取第一个 (P2: `server add --server-label` 选择)。 + +## 6. 缓存与 daemon + +- 编译产物写现有 tools cache (`.tools.json`,同一 TTL 10min / `AGENTCLI_TTL_MS` / `--refresh`)。cache 里存的是 **编译后的 ToolDef[] + 每工具的 operation 元数据** (method/path/paramLocations/bodyInfo),调用时不需要重新碰 spec。 +- 快照 (specs/.json) 与编译缓存 (cache/.tools.json) 分层: 快照是"源",编译缓存是"产物"。`--refresh`: TTL 之外重编译;快照损坏 → 自动回源重拉。 +- daemon 不参与 openapi (无状态);`meta.via: "direct"`。daemon status 里也不计入。 +- `server remove` 同步删快照与编译缓存 (现有删 cache 的地方加一行)。 + +## 7. 模块布局与改造点 + +``` +src/backend/types.ts 新 Backend 接口 + ToolDef (自 types.ts 迁移更名) +src/backend/index.ts 新 openBackend(cfg, name) 工厂 +src/backend/mcp.ts 新 封装 client.ts + MCP→ToolResult 转换 (extractData 自 dispatch.ts 迁入) +src/openapi/specstore.ts 新 快照下载/复制/回源刷新 (fetch + fs, 0600) +src/openapi/ref.ts 新 $ref 内联解析 (循环/深度保护) +src/openapi/compile.ts 新 spec → ToolDef[] + operation 元数据 +src/openapi/exec.ts 新 args → HTTP 请求 → envelope + 错误映射 +src/dispatch.ts 改 改调 openBackend(...);MCP 解析函数迁出 (净减代码) +src/index.ts 改 server add --openapi/--base-url;server tools 走 backend +src/types.ts 改 +OpenApiServerSpec;McpTool → ToolDef (tags?) +src/config.ts 改 removeServer 清快照 +src/flags.ts / jsonout / errors / fuzzy / daemon/* 不动 +``` + +外部依赖: **零新增** (fetch 是 Node ≥18 内建)。 + +## 8. 测试计划 + +| 层 | 内容 | +|---|---| +| 单元: compile | petstore 迷你 fixture: 命名 (operationId/兜底/冲突后缀)、path 强制 required、query/header 参数、body 打平 (简单/复杂/allOf/array)、$ref 内联与循环、tags 收集 | +| 单元: exec | URL 构建 (path 替换/query 数组 repeat)、header env 展开 (含未定义 → AUTH_REQUIRED)、错误码映射表全分支 | +| e2e | test 内起一个真 http server (node:http): GET+path、POST+JSON body、401→AUTH_REQUIRED、500→EXECUTION_ERROR、非 JSON 响应 → text 输出;快照 add → 断网 (删 origin 指向) 仍可执行 | +| 回归 | 现有 MCP 全部测试不动通过 (证明"统一"没破坏旧路径) | + +## 9. 分阶段落地 + +- **P1 (本分支目标)**: Backend 接口 + openapi 全链路 (JSON/3.x、参数打平、body 打平、本地 $ref、快照、错误映射、tag 分组 help) + 全部测试 +- **P2**: securitySchemes 自动提示、YAML、query style (csv/spaceDelimited)、外部 $ref bundle、`--server-label` +- **P3**: 响应 schema 感知输出、`--all` 分页、OpenAPI→MCP 反向桥接 (Backend 已产 ToolDef,包一层 stdio server 即可) + +## 10. 关键决策记录 (ADR 摘要) + +| # | 决策 | 备选与理由 | +|---|---|---| +| 1 | 统一点放在 Backend 接口而非"编译期生成 CLI 代码" | 动态运行时与现有 MCP 路径同构;不引入代码生成、构建步骤 | +| 2 | 内部契约 = ToolDef (入) + ToolResult (出),中立类型;后端抛类型化错误 | 备选"openapi 结果伪装成 MCP envelope"被否 — 内部格式被单一协议绑架;备选"openapi→MCP server 桥"被否 — 多一跳进程、协议协商开销、错误映射失控 | +| 3 | add 时快照落盘 | 离线可用、可审计、spec 变更不破坏已注册 CLI;live 模式 (每次调用回源拉 spec) 被否 — 慢且不稳 | +| 4 | body 属性打平为顶层 flags | agent 体验优先 (不用每次 --input);冲突用 `body_` 前缀消解而非报错 | +| 5 | operationId 缺失兜底为机械 slug | 语义化改名 (如 list_issues) 需要启发式,不可预测;机械规则 agent 可推理 | +| 6 | daemon 不服务 openapi | 无状态 HTTP 无持久连接收益;少一条守护进程职责,复杂度换不来性能 | +| 7 | 零新增依赖 | fetch 内建;$ref 自写解析器 (<100 行) 比引入 api-ref-parser 轻 10 倍 | +| 8 | 统一发生在 **Tool 层**,不是 MCP 层 | OpenAPI spec → ToolDef 直达,不经 MCP 协议;MCP 只是"另一个生产 ToolDef 的后端"。反向桥接 (ToolDef→MCP) 是 P3 可选衍生品 | diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..681efe4 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,51 @@ +# Examples + +Ready-to-try OpenAPI specs for `agentcli server add --openapi`. + +## open-meteo-mini.json + +A trimmed spec for the [Open-Meteo](https://open-meteo.com) weather API — +free, no auth, so it works out of the box: + +```bash +# 1. register (use an absolute path or run from the repo root) +agentcli server add weather --openapi examples/open-meteo-mini.json + +# 2. discover — operations grouped by tag +agentcli weather -h + +# 3. inspect — flags generated from the spec's JSON Schema +agentcli weather getForecast -h + +# 4. execute against the real API +agentcli weather getForecast --latitude 39.9 --longitude 116.4 --current temperature_2m +agentcli weather getForecast --latitude 31.2 --longitude 121.5 --current temperature_2m,wind_speed_10m +``` + +Pipe-friendly: + +```bash +agentcli weather getForecast --latitude 39.9 --longitude 116.4 \ + --current temperature_2m --output text | jq -r '.current.temperature_2m' +``` + +## Your own API + +The same flow works for any OpenAPI 3.x JSON spec (Swagger 2.0 and YAML are +rejected with a conversion hint): + +```bash +agentcli server add myapi --openapi ./api.json \ + --base-url https://staging.example.com \ + --header "Authorization: Bearer ${MY_TOKEN}" # ${ENV} expands at call time +``` + +Path/query/header parameters and JSON body properties all become flags: + +```bash +# POST /pets { "name": "rex", "kind": "dog" } +agentcli myapi addPet --name rex --kind dog + +# complex nested bodies go through --input (flags still override) +agentcli myapi createOrder --input '{"order":{"items":[{"sku":"a","qty":2}]}}' +``` \ No newline at end of file diff --git a/examples/open-meteo-mini.json b/examples/open-meteo-mini.json new file mode 100644 index 0000000..477350c --- /dev/null +++ b/examples/open-meteo-mini.json @@ -0,0 +1,61 @@ +{ + "openapi": "3.0.3", + "info": { + "title": "Open-Meteo Weather", + "version": "1.0" + }, + "servers": [ + { + "url": "https://api.open-meteo.com" + } + ], + "tags": [ + { + "name": "forecast" + } + ], + "paths": { + "/v1/forecast": { + "get": { + "tags": [ + "forecast" + ], + "summary": "Get current & hourly weather for a location", + "operationId": "getForecast", + "parameters": [ + { + "name": "latitude", + "in": "query", + "required": true, + "schema": { + "type": "number" + }, + "description": "Latitude, e.g. 39.9" + }, + { + "name": "longitude", + "in": "query", + "required": true, + "schema": { + "type": "number" + }, + "description": "Longitude, e.g. 116.4" + }, + { + "name": "current", + "in": "query", + "schema": { + "type": "string" + }, + "description": "Comma list: temperature_2m,wind_speed_10m" + } + ], + "responses": { + "200": { + "description": "ok" + } + } + } + } + } +} diff --git a/fixtures/openapi.json b/fixtures/openapi.json new file mode 100644 index 0000000..751ecbe --- /dev/null +++ b/fixtures/openapi.json @@ -0,0 +1,103 @@ +{ + "openapi": "3.0.3", + "info": { "title": "MiniAPI", "version": "1.0.0", "description": "Test fixture spec" }, + "servers": [{ "url": "https://api.mini.test/v1" }], + "tags": [{ "name": "pets" }, { "name": "store" }], + "paths": { + "/pets/{petId}": { + "get": { + "tags": ["pets"], + "summary": "Get a pet by id", + "operationId": "getPetById", + "parameters": [ + { "name": "petId", "in": "path", "required": true, "schema": { "type": "integer" } }, + { "name": "verbose", "in": "query", "schema": { "type": "boolean" } }, + { "$ref": "#/components/parameters/Limit" } + ], + "responses": { "200": { "description": "ok" } } + } + }, + "/pets": { + "get": { + "tags": ["pets"], + "summary": "List pets", + "operationId": "listPets", + "parameters": [ + { "name": "tags", "in": "query", "schema": { "type": "array", "items": { "type": "string" } } }, + { "$ref": "#/components/parameters/Limit" } + ], + "responses": { "200": { "description": "ok" } } + }, + "post": { + "tags": ["pets"], + "summary": "Create a pet", + "operationId": "addPet", + "requestBody": { + "required": true, + "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Pet" } } } + }, + "responses": { "201": { "description": "created" } } + } + }, + "/store/order": { + "get": { + "tags": ["store"], + "summary": "Get an order", + "operationId": "getOrder", + "parameters": [{ "name": "X-Request-Id", "in": "header", "schema": { "type": "string" } }], + "responses": { "200": { "description": "ok" } } + } + }, + "/no-op-id": { + "get": { + "summary": "Operation without operationId", + "parameters": [{ "name": "q", "in": "query", "schema": { "type": "string" } }], + "responses": { "200": { "description": "ok" } } + } + }, + "/circular": { + "post": { + "summary": "Circular ref body", + "operationId": "postCircular", + "requestBody": { + "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Node" } } } + }, + "responses": { "200": { "description": "ok" } } + } + }, + "/batch": { + "post": { + "summary": "Array body", + "operationId": "postBatch", + "requestBody": { + "content": { "application/json": { "schema": { "type": "array", "items": { "type": "string" } } } } + }, + "responses": { "200": { "description": "ok" } } + } + } + }, + "components": { + "parameters": { + "Limit": { "name": "limit", "in": "query", "schema": { "type": "integer", "default": 10 }, "description": "Page size" } + }, + "schemas": { + "Pet": { + "type": "object", + "required": ["name"], + "properties": { + "name": { "type": "string", "description": "Pet name" }, + "tag": { "type": "string" }, + "kind": { "$ref": "#/components/schemas/Kind" } + } + }, + "Kind": { "type": "string", "enum": ["cat", "dog"] }, + "Node": { + "type": "object", + "properties": { + "name": { "type": "string" }, + "children": { "type": "array", "items": { "$ref": "#/components/schemas/Node" } } + } + } + } + } +} \ No newline at end of file diff --git a/package.json b/package.json index 6179cd6..57fd8c3 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,11 @@ { "name": "@happyvibing/agentcli", - "version": "0.1.1", + "version": "0.2.0", "repository": { "type": "git", "url": "git+ssh://git@github.com/happyvibing/agentcli.git" }, - "description": "AgentCLI \u2014 call MCP servers from the command line. One CLI, any MCP server.", + "description": "AgentCLI \u2014 call MCP servers and OpenAPI APIs from the command line. One CLI, any tool backend.", "type": "module", "bin": { "agentcli": "bin/agentcli.js" diff --git a/skills/agentcli/SKILL.md b/skills/agentcli/SKILL.md index 0199ae7..753fc1e 100644 --- a/skills/agentcli/SKILL.md +++ b/skills/agentcli/SKILL.md @@ -1,6 +1,6 @@ --- name: agentcli -description: Use the agentcli runtime to discover and execute MCP server tools from the command line. Use this skill whenever the user asks to work with external systems — GitHub, Feishu/Lark, Notion, Slack, databases, k8s, internal APIs — through MCP servers, or whenever a task involves finding or calling tools via agentcli. Triggers include mentions of agentcli, MCP servers/tools, or tasks like 'search GitHub issues', 'send a Feishu message', 'create a Notion page' when an MCP server may provide the capability. +description: Use the agentcli runtime to discover and execute tools from MCP servers and OpenAPI APIs via the command line. Use this skill whenever the user asks to work with external systems — GitHub, Feishu/Lark, Notion, Slack, databases, k8s, internal REST APIs — through MCP servers or OpenAPI specs, or whenever a task involves finding or calling tools via agentcli. Triggers include mentions of agentcli, MCP servers/tools, OpenAPI specs, or tasks like 'search GitHub issues', 'send a Feishu message', 'call the internal API' when a configured server may provide the capability. license: MIT metadata: compatibility: [claude-code, codex, cursor, opencode] @@ -8,9 +8,9 @@ metadata: # AgentCLI -`agentcli` turns configured MCP servers into CLI commands. You do not need to know -what tools exist ahead of time — discover them at runtime. Never dump tool docs into -memory; look them up on demand. +`agentcli` turns configured MCP servers and OpenAPI APIs into CLI commands. You +do not need to know what tools exist ahead of time — discover them at runtime. +Never dump tool docs into memory; look them up on demand. ## Core loop: discover → inspect → execute → observe @@ -35,6 +35,19 @@ Machine-readable schema instead of help text: agentcli --schema ``` +## Registering servers + +```bash +agentcli server add -- # stdio MCP +agentcli server add --url # HTTP MCP +agentcli server add --openapi # OpenAPI 3.x API + [--base-url ] [--header "Authorization: Bearer ${ENV_VAR}"] +``` + +OpenAPI: every operation becomes a tool — path/query params and JSON body +properties are all flags. `${ENV_VAR}` in headers expands at call time; an +unset variable is AUTH_REQUIRED. Ask before adding servers. + ## Rules 1. **Never guess flags.** If you have not seen this tool's `--help` in this session, run it first. @@ -71,6 +84,7 @@ Every failure prints one self-describing JSON line to stderr — no lookup table ``` - `code` names the class: INVALID_ARGUMENT | NOT_FOUND | AUTH_REQUIRED | TIMEOUT | CONNECT_FAILED | EXECUTION_ERROR | INTERNAL - `message` is the diagnosis; `hint` (when present) is the recommended next action — follow it. +- OpenAPI servers map HTTP statuses onto the same codes: 401/403 → AUTH_REQUIRED, 404 → NOT_FOUND, other 4xx/5xx → EXECUTION_ERROR (details carry httpStatus and the API's own error message). ## Composability @@ -85,10 +99,10 @@ Prefer filtering with tool flags or `jq` over loading everything into context. ## Housekeeping -- Tool listings are cached (10 min). Use `--refresh` if a server's tools changed. -- `meta.via` in the result envelope says `daemon` (persistent connection) or `direct`; purely informational. -- A background daemon (`agentcli daemon start`) holds server connections so repeated +- Tool listings are cached (10 min). Use `--refresh` if a server's tools changed + (for OpenAPI servers it also re-pulls the spec from its origin). +- `meta.via` in the result envelope says `daemon` (persistent MCP connection) or `direct`; + OpenAPI calls are always `direct`. Purely informational. +- A background daemon (`agentcli daemon start`) holds MCP server connections so repeated calls return in milliseconds; calls route through it automatically. Force the -per-call path with `--no-daemon` or `AGENTCLI_NO_DAEMON=1` if it misbehaves. -- Register servers yourself if missing: `agentcli server add -- ` - (stdio) or `agentcli server add --url ` (HTTP). Ask before adding servers. \ No newline at end of file +per-call path with `--no-daemon` or `AGENTCLI_NO_DAEMON=1` if it misbehaves. \ No newline at end of file diff --git a/src/backend/index.ts b/src/backend/index.ts new file mode 100644 index 0000000..a48e814 --- /dev/null +++ b/src/backend/index.ts @@ -0,0 +1,21 @@ +// Backend factory: dispatches on the configured server spec type. Everything +// above this line (dispatch.ts, help, output) is backend-agnostic. +import { errors } from "../errors.js"; +import type { AgentCliConfig } from "../types.js"; +import type { Backend } from "./types.js"; +import { mcpBackend } from "./mcp.js"; +import { openApiBackend } from "../openapi/backend.js"; + +export function openBackend(cfg: AgentCliConfig, serverName: string): Backend { + const spec = cfg.servers[serverName]; + if (!spec) { + throw errors.notFound( + 'server "' + serverName + '" is not configured', + "Add one: agentcli server add " + serverName + " -- | --url | --openapi " + ); + } + if (spec.type === "openapi") return openApiBackend(serverName, spec); + return mcpBackend(cfg, serverName); +} + +export type { Backend, ToolResult, ListToolsResult, CallToolResult } from "./types.js"; \ No newline at end of file diff --git a/src/backend/mcp.ts b/src/backend/mcp.ts new file mode 100644 index 0000000..45d7376 --- /dev/null +++ b/src/backend/mcp.ts @@ -0,0 +1,77 @@ +// MCP backend: adapts the MCP client path (daemon/direct, client.ts) to the +// neutral Backend contract. MCP-specific result parsing lives here — it never +// leaks past this boundary. +import { listTools as mcpListTools, callTool as mcpCallTool } from "../client.js"; +import type { AgentCliConfig, ToolDef } from "../types.js"; +import type { Backend, CallToolResult, ListToolsOptions, ListToolsResult, ToolResult } from "./types.js"; + +interface McpContentItem { + type: string; + text?: string; + [key: string]: unknown; +} + +interface McpCallToolResult { + content?: McpContentItem[]; + isError?: boolean; + structuredContent?: unknown; + [key: string]: unknown; +} + +function extractText(result: McpCallToolResult): string { + return (result.content || []) + .filter((c) => c && c.type === "text") + .map((c) => c.text || "") + .join("\n"); +} + +// Some servers return JSON.stringify(JSON.stringify(payload)) — text content that +// is still a JSON string after one parse. Unwrap while it stays a string (bounded). +function parseMaybeEncoded(text: string, maxDepth = 3): unknown { + let v: unknown = text; + for (let i = 0; typeof v === "string" && i <= maxDepth; i++) { + try { + v = JSON.parse(v); + } catch { + break; + } + } + return v; +} + +// MCP result -> neutral ToolResult: +// structuredContent if the server sent one; +// else if all items are text: parsed JSON (double-encoded strings unwrapped), +// joined text when it is not JSON; +// else pass the content items through. +function toToolResult(mcp: McpCallToolResult): ToolResult { + let data: unknown; + if (mcp.structuredContent !== undefined) { + data = mcp.structuredContent; + } else { + const items = mcp.content || []; + if (items.length === 0) { + data = null; + } else if (items.every((c) => c && c.type === "text")) { + data = parseMaybeEncoded(items.map((c) => c.text || "").join("\n")); + } else { + data = { content: items }; + } + } + const text = extractText(mcp); + return { data, text: text === "" ? undefined : text, isError: !!mcp.isError }; +} + +export function mcpBackend(cfg: AgentCliConfig, serverName: string): Backend { + return { + kind: "mcp", + async listTools(opts: ListToolsOptions = {}): Promise { + const { tools, cached, via } = await mcpListTools(cfg, serverName, opts); + return { tools: tools as ToolDef[], cached, via }; + }, + async callTool(tool: string, args: Record, opts: { timeoutMs?: number; daemon?: boolean } = {}): Promise { + const { result, via } = await mcpCallTool(cfg, serverName, tool, args, opts); + return { result: toToolResult(result as McpCallToolResult), via }; + }, + }; +} \ No newline at end of file diff --git a/src/backend/types.ts b/src/backend/types.ts new file mode 100644 index 0000000..99b7494 --- /dev/null +++ b/src/backend/types.ts @@ -0,0 +1,42 @@ +// The Backend contract — the single unification point of the runtime. +// Core (dispatch/flags/help/output) only ever sees ToolDef in and ToolResult +// out; protocol shapes (MCP content arrays, HTTP responses) are converted +// inside each backend and never cross this boundary. +import type { ToolDef } from "../types.js"; + +// Neutral result: every backend converts its native result into this shape. +// data — machine-readable payload (JSON envelope `data`, jq-ready) +// text — raw human-readable text, when the source produced one +// isError — the tool ran but reported a logical error (MCP isError) +export interface ToolResult { + data: unknown; + text?: string; + isError?: boolean; +} + +export interface ListToolsResult { + tools: ToolDef[]; + cached: boolean; + via: "daemon" | "direct"; +} + +export interface CallToolResult { + result: ToolResult; + via: "daemon" | "direct"; +} + +export interface ListToolsOptions { + refresh?: boolean; + timeoutMs?: number; +} + +export interface CallToolOptions { + timeoutMs?: number; + daemon?: boolean; // MCP only; the OpenAPI backend is always direct +} + +export interface Backend { + readonly kind: "mcp" | "openapi"; + listTools(opts?: ListToolsOptions): Promise; + callTool(tool: string, args: Record, opts?: CallToolOptions): Promise; +} \ No newline at end of file diff --git a/src/client.ts b/src/client.ts index c6eb487..774b0cb 100644 --- a/src/client.ts +++ b/src/client.ts @@ -6,9 +6,21 @@ import net from "node:net"; import { readToolsCache, writeToolsCache } from "./config.js"; import { errors, reviveError, AgentCliError } from "./errors.js"; import { socketPath } from "./daemon/paths.js"; -import type { AgentCliConfig, ServerSpec, McpTool, ListToolsResultEnvelope, CallToolResultEnvelope, DaemonResponse } from "./types.js"; +import type { AgentCliConfig, ServerSpec, ToolDef, DaemonResponse } from "./types.js"; import pkg from "../package.json" with { type: "json" }; +// MCP-side result envelopes (the neutral Backend contract lives in backend/types.ts). +export interface ListToolsResultEnvelope { + tools: ToolDef[]; + cached: boolean; + via: "daemon" | "direct"; +} + +export interface CallToolResultEnvelope { + result: unknown; + via: "daemon" | "direct"; +} + const CLIENT_INFO = { name: "agentcli", version: pkg.version }; const DEFAULT_TTL_MS = 10 * 60 * 1000; const DEFAULT_TIMEOUT_MS = 60 * 1000; @@ -32,8 +44,6 @@ export function timeoutFromEnv(): number { // us accept servers speaking the newer spec. Core methods (initialize / // tools/list / tools/call) are wire-stable across these versions. // Override with AGENTCLI_PROTOCOL_VERSIONS (comma-separated). Dedup-safe once -// the SDK ships these versions itself. - let sdkPromise: Promise<{ Client: typeof import("@modelcontextprotocol/sdk/client/index.js").Client; StdioClientTransport: typeof import("@modelcontextprotocol/sdk/client/stdio.js").StdioClientTransport; @@ -97,6 +107,9 @@ type Transport = InstanceType { const { StdioClientTransport, StreamableHTTPClientTransport } = await sdk(); + if (spec.type === "openapi") { + throw errors.invalidArgument("an openapi server spec reached the MCP transport (this is a bug — openapi servers bypass MCP)"); + } if (spec.type === "http") { return new StreamableHTTPClientTransport(new URL(spec.url), { requestInit: { headers: parseHeaders(spec.headers) }, @@ -238,7 +251,7 @@ export async function listTools( const spec = requireServer(cfg, serverName); if (daemonEnabled(daemon)) { try { - const r = (await daemonRequest("listTools", { server: serverName, refresh }, { timeoutMs: timeoutMs ?? 30000 })) as { tools: McpTool[]; cached: boolean }; + const r = (await daemonRequest("listTools", { server: serverName, refresh }, { timeoutMs: timeoutMs ?? 30000 })) as { tools: ToolDef[]; cached: boolean }; return { tools: r.tools || [], cached: !!r.cached, via: "daemon" }; } catch (e) { if (!(e instanceof DaemonUnavailable) && !(e as DaemonUnavailable)?.unavailable) throw e; @@ -251,7 +264,7 @@ export async function listTools( try { const tools = await withClient(spec, serverName, async (client: McpClient) => { const res = await client.listTools(undefined, { timeout: timeoutMs }); - return (res.tools as McpTool[]) || []; + return (res.tools as ToolDef[]) || []; }, timeoutMs); writeToolsCache(serverName, tools); return { tools, cached: false, via: "direct" }; diff --git a/src/config.ts b/src/config.ts index 9be8683..1e241f1 100644 --- a/src/config.ts +++ b/src/config.ts @@ -3,7 +3,7 @@ import fs from "node:fs"; import path from "node:path"; import os from "node:os"; import { errors } from "./errors.js"; -import type { AgentCliConfig, ServerSpec, McpTool, ToolsCache } from "./types.js"; +import type { AgentCliConfig, ServerSpec, ToolDef, ToolsCache } from "./types.js"; export const RESERVED_NAMES = new Set(["server", "call", "help", "version", "config", "doctor", "completion", "daemon"]); @@ -66,9 +66,10 @@ export function removeServer(cfg: AgentCliConfig, name: string): void { delete cfg.servers[name]; saveConfig(cfg); - // Clean up the stale tools cache file for the removed server. + // Clean up the stale tools cache and openapi spec snapshot for the removed server. try { fs.rmSync(path.join(cacheDir(), name + ".tools.json"), { force: true }); + fs.rmSync(path.join(path.dirname(configPath()), "specs", name + ".json"), { force: true }); } catch { // ignore } @@ -80,7 +81,7 @@ function cacheFile(server: string): string { return path.join(cacheDir(), server + ".tools.json"); } -export function readToolsCache(server: string, ttlMs: number): { tools: McpTool[]; fresh: boolean } | null { +export function readToolsCache(server: string, ttlMs: number): { tools: ToolDef[]; fresh: boolean } | null { let raw: ToolsCache; try { raw = JSON.parse(fs.readFileSync(cacheFile(server), "utf8")); @@ -91,7 +92,7 @@ export function readToolsCache(server: string, ttlMs: number): { tools: McpTool[ return { tools: raw.tools, fresh: Date.now() - raw.fetchedAt < ttlMs }; } -export function writeToolsCache(server: string, tools: McpTool[]): void { +export function writeToolsCache(server: string, tools: ToolDef[]): void { fs.mkdirSync(cacheDir(), { recursive: true }); fs.writeFileSync(cacheFile(server), JSON.stringify({ fetchedAt: Date.now(), tools }, null, 2)); } diff --git a/src/daemon/server.ts b/src/daemon/server.ts index e905ae0..f421e41 100644 --- a/src/daemon/server.ts +++ b/src/daemon/server.ts @@ -7,7 +7,7 @@ import { createTransport, mapError, sdk, ttlFromEnv } from "../client.js"; import { loadConfig } from "../config.js"; import { errors, serializeError, AgentCliError } from "../errors.js"; import { socketPath, pidPath, ensureDaemonDir } from "./paths.js"; -import type { AgentCliConfig, ServerSpec, McpTool, DaemonStatusData, DaemonResponse } from "../types.js"; +import type { AgentCliConfig, ServerSpec, ToolDef, DaemonStatusData, DaemonResponse } from "../types.js"; import pkg from "../../package.json" with { type: "json" }; const CLIENT_INFO = { name: "agentcli-daemon", version: pkg.version }; @@ -22,7 +22,7 @@ type McpClient = InstanceType = new Map(); - tools: Map = new Map(); + tools: Map = new Map(); stats = { startedAt: Date.now(), requests: 0, toolCalls: 0 }; shuttingDown = false; idleTimer: ReturnType | null = null; @@ -69,7 +69,7 @@ class DaemonState { return client; } - async listTools({ server, refresh }: { server: string; refresh?: boolean }): Promise<{ tools: McpTool[]; cached: boolean }> { + async listTools({ server, refresh }: { server: string; refresh?: boolean }): Promise<{ tools: ToolDef[]; cached: boolean }> { const cfg = this.freshConfig(); const spec = cfg.servers[server]; if (!spec) { @@ -86,7 +86,7 @@ class DaemonState { } catch (e) { throw mapError(e, server); } - const tools = (res.tools as McpTool[]) || []; + const tools = (res.tools as ToolDef[]) || []; this.tools.set(server, { tools, fetchedAt: Date.now() }); return { tools, cached: false }; } diff --git a/src/dispatch.ts b/src/dispatch.ts index 0a9d99d..8c78934 100644 --- a/src/dispatch.ts +++ b/src/dispatch.ts @@ -1,66 +1,18 @@ // Dispatch `agentcli [flags]` — the core execution path. -// Hand-rolled token parsing (commander is bypassed for the dynamic part). -import { listTools, callTool } from "./client.js"; +// Backend-agnostic: all server interaction goes through the Backend contract. +import { openBackend } from "./backend/index.js"; +import type { Backend } from "./backend/types.js"; import { suggest } from "./fuzzy.js"; import { buildFlagPlan, parseToolArgs, readInputJson, mergeArgs, renderToolHelp, validateRequired } from "./flags.js"; import { errors, EXIT } from "./errors.js"; import { printJson } from "./jsonout.js"; -import type { AgentCliConfig, McpTool } from "./types.js"; +import type { AgentCliConfig, ToolDef } from "./types.js"; function firstLine(text: string | undefined): string { return String(text || "").split("\n")[0]; } -interface McpContentItem { - type: string; - text?: string; - [key: string]: unknown; -} - -interface McpCallToolResult { - content?: McpContentItem[]; - isError?: boolean; - structuredContent?: unknown; - [key: string]: unknown; -} - -function extractText(result: McpCallToolResult): string { - return (result.content || []) - .filter((c) => c && c.type === "text") - .map((c) => c.text || "") - .join("\n"); -} - -// Some servers return JSON.stringify(JSON.stringify(payload)) — text content that -// is still a JSON string after one parse. Unwrap while it stays a string (bounded). -function parseMaybeEncoded(text: string, maxDepth = 3): unknown { - let v: unknown = text; - for (let i = 0; typeof v === "string" && i <= maxDepth; i++) { - try { - v = JSON.parse(v); - } catch { - break; - } - } - return v; -} - -// Result -> data: -// structuredContent if the server sent one; -// else if all items are text: parsed JSON (double-encoded strings unwrapped), -// joined text when it is not JSON; -// else pass the content items through. -function extractData(result: McpCallToolResult): unknown { - if (result.structuredContent !== undefined) return result.structuredContent; - const items = result.content || []; - if (items.length === 0) return null; - if (items.every((c) => c && c.type === "text")) { - return parseMaybeEncoded(items.map((c) => c.text || "").join("\n")); - } - return { content: items }; -} - -function toolNotFound(serverName: string, toolName: string, tools: McpTool[]): AgentCliError { +function toolNotFound(serverName: string, toolName: string, tools: ToolDef[]): Error { const similar = suggest(toolName, tools.map((t) => t.name)).slice(0, 5); const hints: string[] = []; if (similar.length) hints.push("Similar tools: " + similar.join(", ")); @@ -68,25 +20,58 @@ function toolNotFound(serverName: string, toolName: string, tools: McpTool[]): A return errors.notFound('tool "' + toolName + '" not found on server "' + serverName + '"', hints.join(" | ")); } -import type { AgentCliError } from "./errors.js"; +function kindLabel(backend: Backend): string { + return backend.kind === "openapi" ? "OpenAPI server" : "MCP server"; +} -async function printServerHelp(cfg: AgentCliConfig, serverName: string, refresh: boolean): Promise { - const { tools } = await listTools(cfg, serverName, { refresh }); - const lines = [serverName + " — MCP server (" + tools.length + " tools)", "", "Usage:", " agentcli " + serverName + " [flags]", "", "Tools:"]; - for (const t of tools) { +function printServerHelp(backend: Backend, serverName: string, tools: ToolDef[]): number { + const lines = [serverName + " — " + kindLabel(backend) + " (" + tools.length + " tools)", "", "Usage:", " agentcli " + serverName + " [flags]", ""]; + const row = (t: ToolDef) => { const name = String(t.name); const pad = name.length >= 28 ? " " : " ".repeat(28 - name.length); - lines.push(" " + name + pad + firstLine(t.description)); + return " " + name + pad + firstLine(t.description); + }; + if (tools.some((t) => t.tags && t.tags.length)) { + // Group by first tag (spec order preserved); untagged tools go last. + const groups = new Map(); + const untagged: ToolDef[] = []; + for (const t of tools) { + const g = t.tags && t.tags[0]; + if (g) { + if (!groups.has(g)) groups.set(g, []); + (groups.get(g) as ToolDef[]).push(t); + } else { + untagged.push(t); + } + } + for (const [tag, list] of groups) { + lines.push(tag + ":"); + for (const t of list) lines.push(row(t)); + lines.push(""); + } + if (untagged.length) { + lines.push("other:"); + for (const t of untagged) lines.push(row(t)); + lines.push(""); + } + lines.push(" agentcli " + serverName + " --help Help for a specific tool"); + console.log(lines.join("\n")); + return EXIT.OK; } + lines.push("Tools:"); + for (const t of tools) lines.push(row(t)); lines.push("", " agentcli " + serverName + " --help Help for a specific tool"); console.log(lines.join("\n")); return EXIT.OK; } export async function runServerCommand(cfg: AgentCliConfig, serverName: string, tail: string[]): Promise { + const backend = openBackend(cfg, serverName); + // 1. server-level help: `agentcli ` or `agentcli --help` if (tail.length === 0 || tail[0] === "--help" || tail[0] === "-h") { - return printServerHelp(cfg, serverName, tail.includes("--refresh")); + const { tools } = await backend.listTools({ refresh: tail.includes("--refresh") }); + return printServerHelp(backend, serverName, tools); } const toolName = tail[0]; @@ -99,7 +84,7 @@ export async function runServerCommand(cfg: AgentCliConfig, serverName: string, const wantsHelp = rest.includes("--help") || rest.includes("-h"); const wantsSchema = rest.includes("--schema"); const noDaemon = rest.includes("--no-daemon"); - const { tools, cached } = await listTools(cfg, serverName, { refresh: rest.includes("--refresh"), daemon: !noDaemon }); + const { tools, cached } = await backend.listTools({ refresh: rest.includes("--refresh") }); const tool = tools.find((t) => t.name === toolName); if (!tool) throw toolNotFound(serverName, toolName, tools); @@ -132,34 +117,24 @@ export async function runServerCommand(cfg: AgentCliConfig, serverName: string, // 4. execute const started = Date.now(); - const { result, via } = await callTool(cfg, serverName, toolName, args, { timeoutMs, daemon: !noDaemon }); - - const mcpResult = result as McpCallToolResult; + const { result, via } = await backend.callTool(toolName, args, { timeoutMs, daemon: !noDaemon }); - if (mcpResult.isError) { - throw errors.execution(extractText(mcpResult) || "tool reported an error without a message"); + if (result.isError) { + throw errors.execution(result.text || "tool reported an error without a message"); } // 5. observe if (output === "text") { - const text = extractText(mcpResult); - if (text === "") { - // Non-text content items: fall back to the machine-readable form. - process.stdout.write(JSON.stringify(extractData(mcpResult)) + "\n"); - } else { - // Double-encoded JSON pretty-prints; genuine prose passes through raw. - const v = parseMaybeEncoded(text); - const out = v !== null && typeof v === "object" ? JSON.stringify(v, null, 2) : text; - process.stdout.write(out + "\n"); - } + const out = result.data !== null && typeof result.data === "object" ? JSON.stringify(result.data, null, 2) : result.text ?? JSON.stringify(result.data); + process.stdout.write(out + "\n"); return EXIT.OK; } printJson({ ok: true, server: serverName, tool: toolName, - data: extractData(mcpResult), + data: result.data, meta: { durationMs: Date.now() - started, schemaCached: cached, via }, }); return EXIT.OK; -} +} \ No newline at end of file diff --git a/src/flags.ts b/src/flags.ts index 93cc6fa..f17f0d6 100644 --- a/src/flags.ts +++ b/src/flags.ts @@ -3,7 +3,7 @@ // --input accepts inline JSON, @file, or - (stdin). Flags override --input keys. import fs from "node:fs"; import { errors } from "./errors.js"; -import type { ToolInputSchema, ToolPropertySchema, FlagPlan, FlagSpec, McpTool } from "./types.js"; +import type { ToolInputSchema, ToolPropertySchema, FlagPlan, FlagSpec, ToolDef } from "./types.js"; const CONTROL_FLAGS = new Set(["input", "output", "schema", "refresh", "help", "timeout-ms", "no-daemon"]); @@ -180,7 +180,7 @@ export function validateRequired(plan: FlagPlan, args: Record): ); } -export function renderToolHelp(serverName: string, tool: McpTool): string { +export function renderToolHelp(serverName: string, tool: ToolDef): string { const plan = buildFlagPlan(tool.inputSchema); const lines: string[] = []; lines.push(serverName + " " + tool.name); diff --git a/src/index.ts b/src/index.ts index bff55e0..1d796d1 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,13 +1,16 @@ import { Command } from "commander"; +import path from "node:path"; import pkg from "../package.json" with { type: "json" }; -import { loadConfig, addServer, removeServer } from "./config.js"; +import { loadConfig, addServer, removeServer, writeToolsCache } from "./config.js"; import { AgentCliError, errors, EXIT } from "./errors.js"; import { printError, printJson } from "./jsonout.js"; import { runServerCommand } from "./dispatch.js"; -import { listTools } from "./client.js"; +import { openBackend } from "./backend/index.js"; +import { snapshotSpec, snapshotPath } from "./openapi/specstore.js"; +import { compileSpec } from "./openapi/compile.js"; import { startDaemon, stopDaemon, daemonStatus } from "./daemon/lifecycle.js"; import { suggest } from "./fuzzy.js"; -import type { AgentCliConfig, ServerSpec, StdioServerSpec, HttpServerSpec } from "./types.js"; +import type { AgentCliConfig, ServerSpec, StdioServerSpec, HttpServerSpec, OpenApiServerSpec } from "./types.js"; const { version } = pkg; @@ -79,14 +82,16 @@ function serversHelpSection(cfg: AgentCliConfig): string { return [ "", "No servers configured yet:", - " agentcli server add -- [args...] # stdio server", - " agentcli server add --url # Streamable HTTP server", + " agentcli server add -- [args...] # stdio MCP server", + " agentcli server add --url # Streamable HTTP MCP server", + " agentcli server add --openapi # OpenAPI 3.x API", "", ].join("\n"); } const lines = entries.map(([name, spec]: [string, ServerSpec]) => { - const detail = spec.type === "http" ? spec.url : [spec.command, ...(spec.args || [])].join(" "); - return " " + name.padEnd(16) + (spec.type === "http" ? "http - " : "stdio - ") + detail; + const kind = spec.type === "http" ? "http" : spec.type === "openapi" ? "openapi" : "stdio"; + const detail = spec.type === "http" ? spec.url : spec.type === "openapi" ? spec.origin : [spec.command, ...(spec.args || [])].join(" "); + return " " + name.padEnd(16) + kind + " - " + detail; }); return [ "", @@ -102,13 +107,13 @@ function buildBuiltins(cfg: AgentCliConfig): Command { program .name("agentcli") .version(version) - .description("AgentCLI -- call MCP servers from the command line.") + .description("AgentCLI -- call MCP servers and OpenAPI APIs from the command line.") .exitOverride() .configureOutput({ writeErr: () => {} }); program.addHelpText("after", serversHelpSection(cfg)); - const server = program.command("server").description("Manage configured MCP servers."); + const server = program.command("server").description("Manage configured servers (MCP or OpenAPI)."); server.addHelpText( "after", "\nConfigured servers: " + (Object.keys(cfg.servers).join(", ") || "none") + "\n agentcli server list Details as JSON\n" @@ -116,13 +121,35 @@ function buildBuiltins(cfg: AgentCliConfig): Command { server .command("add ") - .description("Register a server: stdio via `-- `, or Streamable HTTP via --url.") + .description("Register a server: stdio via `-- `, HTTP via --url, or an OpenAPI 3.x spec via --openapi.") .option("--url ", "Streamable HTTP endpoint of the MCP server") - .option("--header ", "HTTP header sent on every request (repeatable)") + .option("--openapi ", "OpenAPI 3.x JSON spec (local path or URL) — every operation becomes a tool") + .option("--base-url ", "Override the API base URL (with --openapi)") + .option("--header ", "HTTP header sent on every request (repeatable; ${ENV_VAR} placeholders expand at call time)") .option("--env ", "Extra env vars for a stdio server (repeatable)") .argument("[cmd...]", "stdio server command and args (after --)") - .action(async (name: string, cmd: string[], opts: { url?: string; header?: string[]; env?: string[] }) => { + .action(async (name: string, cmd: string[], opts: { url?: string; openapi?: string; baseUrl?: string; header?: string[]; env?: string[] }) => { const current = loadConfig(); + if (opts.openapi) { + if (opts.url) throw errors.invalidArgument("--openapi and --url are mutually exclusive"); + if (cmd && cmd.length) throw errors.invalidArgument("--openapi and a command are mutually exclusive"); + // fail fast: snapshot + compile at add time so bad specs never register + const hasScheme = /^[a-zA-Z][a-zA-Z0-9+.-]*:\/\//.test(opts.openapi); + const origin = hasScheme ? opts.openapi : path.resolve(opts.openapi); + const doc = await snapshotSpec(name, origin); + const tools = compileSpec(doc, { baseUrl: opts.baseUrl, originUrl: hasScheme ? origin : undefined }); + const spec: OpenApiServerSpec = { + type: "openapi", + spec: snapshotPath(name), + origin, + ...(opts.baseUrl ? { baseUrl: opts.baseUrl } : {}), + headers: parseKeyValueList(opts.header, "header", '"Name: value"', ":"), + }; + addServer(current, name, spec); + writeToolsCache(name, tools); + printJson({ ok: true, server: name, operations: tools.length, config: process.env.AGENTCLI_CONFIG }); + return; + } if (opts.url) { if (cmd && cmd.length) throw errors.invalidArgument("--url and a command are mutually exclusive"); const spec: HttpServerSpec = { type: "http", url: opts.url, headers: parseKeyValueList(opts.header, "header", '"Name: value"', ":") }; @@ -131,7 +158,7 @@ function buildBuiltins(cfg: AgentCliConfig): Command { if (!cmd || cmd.length === 0) { throw errors.invalidArgument( "a stdio server needs a command", - "agentcli server add -- [args...] | agentcli server add --url " + "agentcli server add -- [args...] | agentcli server add --url | agentcli server add --openapi " ); } const [command, ...args] = cmd; @@ -147,12 +174,13 @@ function buildBuiltins(cfg: AgentCliConfig): Command { .option("-o, --output ", "json | text (default: json)") .action((opts: { output?: string }) => { const current = loadConfig(); - const rows: Array<{ name: string; type: string; command?: string; url?: string }> = Object.entries(current.servers).map(([name, spec]: [string, ServerSpec]) => { + const rows: Array<{ name: string; type: string; command?: string; url?: string; origin?: string }> = Object.entries(current.servers).map(([name, spec]: [string, ServerSpec]) => { if (spec.type === "http") return { name, type: spec.type, url: spec.url }; + if (spec.type === "openapi") return { name, type: spec.type, origin: spec.origin }; return { name, type: spec.type, command: [spec.command, ...(spec.args || [])].join(" ") }; }); if (opts.output === "text") { - for (const r of rows) console.log(r.name.padEnd(16) + r.type.padEnd(8) + (r.command || r.url || "")); + for (const r of rows) console.log(r.name.padEnd(16) + r.type.padEnd(8) + (r.command || r.url || r.origin || "")); return; } printJson({ ok: true, data: rows }); @@ -169,10 +197,10 @@ function buildBuiltins(cfg: AgentCliConfig): Command { server .command("tools ") .description("List the tools a server exposes.") - .option("--refresh", "Bypass the tools cache") + .option("--refresh", "Bypass the tools cache (OpenAPI: also re-pull the spec from its origin)") .option("-o, --output ", "json | text (default: json)") .action(async (name: string, opts: { refresh?: boolean; output?: string }) => { - const { tools } = await listTools(loadConfig(), name, { refresh: !!opts.refresh }); + const { tools } = await openBackend(loadConfig(), name).listTools({ refresh: !!opts.refresh }); if (opts.output === "text") { for (const t of tools) console.log(String(t.name).padEnd(28) + String(t.description || "").split("\n")[0]); return; @@ -264,4 +292,4 @@ export async function run(argv: string[]): Promise { } catch (e) { exitWithError(e); } -} +} \ No newline at end of file diff --git a/src/openapi/backend.ts b/src/openapi/backend.ts new file mode 100644 index 0000000..6cd2cbe --- /dev/null +++ b/src/openapi/backend.ts @@ -0,0 +1,42 @@ +// OpenAPI backend: serves ToolDefs compiled from the spec snapshot and executes +// them over plain HTTP. Stateless — no daemon involvement; `via` is "direct". +import { readToolsCache, writeToolsCache } from "../config.js"; +import { ttlFromEnv, timeoutFromEnv } from "../client.js"; +import { errors } from "../errors.js"; +import type { OpenApiServerSpec, ToolDef } from "../types.js"; +import type { Backend, ListToolsResult, ToolResult } from "../backend/types.js"; +import { compileSpec } from "./compile.js"; +import { isHttpUrl, loadSnapshotDoc, refreshSnapshot } from "./specstore.js"; +import { executeOperation } from "./exec.js"; + +export function openApiBackend(name: string, spec: OpenApiServerSpec): Backend { + async function listToolsInternal({ refresh = false, timeoutMs }: { refresh?: boolean; timeoutMs?: number } = {}): Promise { + if (!refresh) { + const cached = readToolsCache(name, ttlFromEnv()); + if (cached && cached.fresh) return { tools: cached.tools, cached: true, via: "direct" }; + } else { + // --refresh: re-pull the spec from its origin (URL or local file) + await refreshSnapshot(name, spec.origin, timeoutMs); + } + const doc = await loadSnapshotDoc(name, spec.origin); + const tools = compileSpec(doc, { baseUrl: spec.baseUrl, originUrl: isHttpUrl(spec.origin) ? spec.origin : undefined }); + writeToolsCache(name, tools); + return { tools, cached: false, via: "direct" }; + } + + return { + kind: "openapi", + async listTools(opts = {}): Promise { + return listToolsInternal(opts); + }, + async callTool(tool: string, args: Record, opts: { timeoutMs?: number } = {}): Promise<{ result: ToolResult; via: "direct" }> { + const { tools } = await listToolsInternal(); + const def = tools.find((t: ToolDef) => t.name === tool); + if (!def || !def.openapiMeta) { + throw errors.notFound('tool "' + tool + '" not found on server "' + name + '"', "List tools: agentcli " + name + " --help"); + } + const result = await executeOperation(name, spec, def.openapiMeta, args, opts.timeoutMs ?? timeoutFromEnv()); + return { result, via: "direct" }; + }, + }; +} \ No newline at end of file diff --git a/src/openapi/compile.ts b/src/openapi/compile.ts new file mode 100644 index 0000000..1c2411e --- /dev/null +++ b/src/openapi/compile.ts @@ -0,0 +1,208 @@ +// OpenAPI 3.x spec -> ToolDef[] compiler. Each operation becomes one tool: +// path/query/header params and JSON body properties are flattened into a single +// inputSchema, so the existing flag compiler (flags.ts) works unchanged. +import { errors } from "../errors.js"; +import type { OpenApiOperationMeta, ToolDef, ToolInputSchema, ToolPropertySchema } from "../types.js"; +import { resolveRefs } from "./ref.js"; + +const METHODS = ["get", "put", "post", "delete", "options", "head", "patch", "trace"]; + +interface OpenApiParam { + name?: string; + in?: string; + required?: boolean; + description?: string; + schema?: unknown; + content?: Record; + [key: string]: unknown; +} + +interface OpenApiOperation { + operationId?: string; + summary?: string; + description?: string; + tags?: string[]; + parameters?: unknown[]; + requestBody?: unknown; + [key: string]: unknown; +} + +function sanitizeName(raw: string): string { + return raw.replace(/[^a-zA-Z0-9_.-]/g, "_"); +} + +// Mechanical, predictable fallback: GET /pets/{petId} -> get_pets_petId +function slugName(method: string, path: string): string { + const p = path + .replace(/^\//, "") + .replace(/\//g, "_") + .replace(/[{}]/g, ""); + return sanitizeName(method.toLowerCase() + "_" + p); +} + +function expandServerUrl(server: { url?: string; variables?: Record }): string { + const url = server.url || ""; + return url.replace(/\{([^}]+)\}/g, (_m, v: string) => server.variables?.[v]?.default ?? "{" + v + "}"); +} + +function resolveBaseUrl(doc: Record, override?: string, originUrl?: string): string { + const base = override || ""; + if (base) return base.replace(/\/+$/, ""); + const servers = (doc.servers as Array<{ url?: string; variables?: Record }> | undefined)?.[0]; + if (servers && servers.url) { + let url = expandServerUrl(servers).replace(/\/+$/, ""); + // Relative server URLs (e.g. "/api/v3") resolve against the spec's origin. + if (url.startsWith("/") && originUrl) { + try { + url = new URL(url, originUrl).toString().replace(/\/+$/, ""); + } catch { + // leave as-is; a later new URL() in exec will produce a clear error + } + } + return url; + } + throw errors.invalidArgument( + "spec declares no servers entry and no --base-url was given", + "agentcli server add --openapi --base-url https://api.example.com" + ); +} + +// Top-level allOf: merge properties/required from each resolved branch. +function mergeAllOf(schema: Record): Record { + if (!Array.isArray(schema.allOf)) return schema; + const props: Record = {}; + const required = new Set((schema.required as string[]) || []); + for (const sub of schema.allOf as Record[]) { + Object.assign(props, (sub.properties as Record) || {}); + for (const r of (sub.required as string[]) || []) required.add(r); + } + Object.assign(props, (schema.properties as Record) || {}); + const out: Record = { ...schema, properties: props }; + if (required.size) out.required = [...required]; + else delete out.required; + delete out.allOf; + return out; +} + +export interface CompileOptions { + baseUrl?: string; // --base-url override + originUrl?: string; // spec origin URL, for resolving relative server entries +} + +export function compileSpec(doc: unknown, opts: CompileOptions = {}): ToolDef[] { + if (!doc || typeof doc !== "object") throw errors.invalidArgument("OpenAPI spec is not a JSON object"); + const d = doc as Record; + if (typeof d.swagger === "string") { + throw errors.invalidArgument("spec is Swagger 2.0, only OpenAPI 3.x is supported", "Upgrade the spec to 3.x (e.g. with openapi-diff / swagger2openapi)"); + } + if (typeof d.openapi !== "string" || !d.openapi.startsWith("3.")) { + throw errors.invalidArgument('spec has no "openapi: 3.x" version field', "Expected OpenAPI 3.x JSON"); + } + const baseUrl = resolveBaseUrl(d, opts.baseUrl, opts.originUrl); + + const paths = (d.paths as Record>) || {}; + const tools: ToolDef[] = []; + const nameCount = new Map(); + + for (const [path, pathItem] of Object.entries(paths)) { + if (!path.startsWith("/") || !pathItem || typeof pathItem !== "object") continue; + const sharedParams = Array.isArray(pathItem.parameters) ? (pathItem.parameters as unknown[]) : []; + + for (const method of METHODS) { + const op = pathItem[method] as OpenApiOperation | undefined; + if (!op || typeof op !== "object") continue; + + // --- name --- + let name = op.operationId ? sanitizeName(op.operationId) : slugName(method, path); + const seen = nameCount.get(name) || 0; + nameCount.set(name, seen + 1); + if (seen > 0) name = name + "_" + (seen + 1); + + // --- parameters: path-item level + operation level (op wins by name+in) --- + const opParams = Array.isArray(op.parameters) ? op.parameters : []; + const merged: OpenApiParam[] = []; + const keyed = (p: OpenApiParam) => (p.in || "") + ":" + (p.name || ""); + const opResolved = opParams.map((p) => resolveRefs(p, d) as OpenApiParam); + for (const p of sharedParams.map((p) => resolveRefs(p, d) as OpenApiParam)) { + if (!opResolved.some((q) => keyed(q) === keyed(p))) merged.push(p); + } + merged.push(...opResolved); + + const properties: Record = {}; + const required = new Set(); + const meta: OpenApiOperationMeta = { method: method.toUpperCase(), path, baseUrl, pathParams: {}, queryParams: {}, headerParams: {}, bodyProps: {} }; + const taken = new Set(); // flag names already used + + for (const param of merged) { + const loc = param.in; + if (loc !== "path" && loc !== "query" && loc !== "header") continue; // cookie skipped + if (!param.name) continue; + let schema = (param.schema as ToolPropertySchema) || undefined; + if (!schema && param.content) { + const json = Object.entries(param.content).find(([mime]) => mime.startsWith("application/json")); + if (json) schema = json[1].schema as ToolPropertySchema; + } + schema = (resolveRefs(schema || { type: "string" }, d) as ToolPropertySchema) || { type: "string" }; + if (param.description && !schema.description) schema = { ...schema, description: param.description }; + + let flag = param.name; + if (taken.has(flag)) flag = loc + "_" + param.name; // same name in two locations + taken.add(flag); + properties[flag] = schema; + if (loc === "path") { + required.add(flag); + meta.pathParams[flag] = param.name; + } else if (loc === "query") { + if (param.required) required.add(flag); + meta.queryParams[flag] = param.name; + } else { + if (param.required) required.add(flag); + meta.headerParams[flag] = param.name; + } + } + + // --- requestBody (application/json only) --- + let bodyNote = ""; + const rb = resolveRefs(op.requestBody, d) as { content?: Record } | undefined; + const jsonContent = rb?.content && Object.entries(rb.content).find(([mime]) => mime.startsWith("application/json")); + if (rb && !jsonContent) { + const mimes = Object.keys(rb.content || {}).join(", ") || "(none)"; + bodyNote = "\n(body content type not supported: " + mimes + ")"; + } + if (jsonContent && jsonContent[1].schema) { + let schema = resolveRefs(jsonContent[1].schema, d) as Record; + schema = mergeAllOf(schema); + const props = (schema.properties as Record) || {}; + const bodyRequired = new Set((schema.required as string[]) || []); + if (schema.type === "object" && Object.keys(props).length) { + // flatten body properties to top-level flags + for (const [propName, pschema] of Object.entries(props)) { + let flag = propName; + if (taken.has(flag)) flag = "body_" + propName; // collision with a param + taken.add(flag); + properties[flag] = pschema; + if (bodyRequired.has(propName)) required.add(flag); + meta.bodyProps[flag] = propName; + } + } else { + // non-object body (array/string/...): single param holding the whole body + let flag = "body"; + if (taken.has(flag)) flag = "body_"; + taken.add(flag); + properties[flag] = schema as ToolPropertySchema; + meta.rawBody = flag; + } + } + + // --- description --- + const descParts = [op.summary, op.description].filter((s) => typeof s === "string" && s) as string[]; + const description = (descParts.join("\n\n") || method.toUpperCase() + " " + path) + "\n(" + method.toUpperCase() + " " + path + ")" + bodyNote; + + const inputSchema: ToolInputSchema = { type: "object", properties }; + if (required.size) inputSchema.required = [...required]; + + tools.push({ name, description, inputSchema, tags: op.tags, openapiMeta: meta }); + } + } + return tools; +} \ No newline at end of file diff --git a/src/openapi/exec.ts b/src/openapi/exec.ts new file mode 100644 index 0000000..6c31ef9 --- /dev/null +++ b/src/openapi/exec.ts @@ -0,0 +1,125 @@ +// OpenAPI executor: compiled operation metadata + CLI args -> HTTP request -> +// neutral ToolResult. HTTP failures throw typed AgentCliErrors (the dispatch +// layer never sees an HTTP status). +import { errors } from "../errors.js"; +import { timeoutFromEnv } from "../client.js"; +import type { OpenApiOperationMeta, OpenApiServerSpec } from "../types.js"; +import type { ToolResult } from "../backend/types.js"; +import pkg from "../../package.json" with { type: "json" }; + +const ENV_VAR = /\$\{([A-Za-z_][A-Za-z0-9_]*)\}/g; + +// Expand ${VAR} placeholders in configured header values. A referenced env var +// that is unset is a missing credential: AUTH_REQUIRED, never a silent empty header. +function expandEnvValue(value: string, header: string): string { + return value.replace(ENV_VAR, (m, v: string) => { + const val = process.env[v]; + if (val === undefined) { + throw errors.auth('missing env var ' + v + ' for header "' + header + '"'); + } + return val; + }); +} + +function truncate(s: string, max = 2048): string { + return s.length > max ? s.slice(0, max) + "…" : s; +} + +// Pull a human-readable message out of an error body if one exists. +function extractErrorMessage(bodyText: string): string | undefined { + try { + const v = JSON.parse(bodyText) as { message?: unknown; error?: { message?: unknown } | string }; + if (typeof v.message === "string" && v.message) return v.message; + if (typeof v.error === "string" && v.error) return v.error; + if (v.error && typeof v.error === "object" && typeof v.error.message === "string") return v.error.message; + } catch { + // not JSON + } + return undefined; +} + +function mapHttpError(serverName: string, status: number, bodyText: string, retryAfter: string | null): Error { + const msg = serverName + ": HTTP " + status + (extractErrorMessage(bodyText) ? " — " + extractErrorMessage(bodyText) : ""); + if (status === 401 || status === 403) return errors.auth(msg); + if (status === 404) return errors.notFound(msg, "Check the path parameters and the API base URL"); + if (status === 429) return errors.execution(msg, { httpStatus: status, retryAfter: retryAfter || undefined }); + return errors.execution(msg, { httpStatus: status, body: truncate(bodyText) }); +} + +export async function executeOperation( + serverName: string, + spec: OpenApiServerSpec, + meta: OpenApiOperationMeta, + args: Record, + timeoutMs?: number +): Promise { + // 1. headers: Accept, header params, config headers (env-expanded), UA + const headers: Record = { Accept: "application/json", "User-Agent": "agentcli/" + pkg.version }; + for (const [flag, orig] of Object.entries(meta.headerParams)) { + if (args[flag] !== undefined) headers[orig] = String(args[flag]); + } + for (const [k, v] of Object.entries(spec.headers || {})) { + headers[k] = expandEnvValue(String(v), k); + } + + // 2. path + query + let path = meta.path; + for (const [flag, orig] of Object.entries(meta.pathParams)) { + const v = args[flag]; + if (v === undefined) throw errors.invalidArgument("missing path parameter --" + flag); + path = path.split("{" + orig + "}").join(encodeURIComponent(String(v))); + } + const query = new URLSearchParams(); + for (const [flag, orig] of Object.entries(meta.queryParams)) { + const v = args[flag]; + if (v === undefined) continue; + if (Array.isArray(v)) for (const item of v) query.append(orig, String(item)); + else query.append(orig, String(v)); + } + const urlStr = meta.baseUrl + path + (query.size ? "?" + query.toString() : ""); + + // 3. body + let body: string | undefined; + if (meta.rawBody && args[meta.rawBody] !== undefined) { + body = JSON.stringify(args[meta.rawBody]); + } else if (Object.keys(meta.bodyProps).length) { + const bodyObj: Record = {}; + for (const [flag, orig] of Object.entries(meta.bodyProps)) { + if (args[flag] !== undefined) bodyObj[orig] = args[flag]; + } + if (Object.keys(bodyObj).length) body = JSON.stringify(bodyObj); + } + if (body !== undefined) headers["Content-Type"] = "application/json"; + + // 4. fetch + let res: Response; + try { + res = await fetch(urlStr, { + method: meta.method, + headers, + body, + signal: AbortSignal.timeout(timeoutMs ?? timeoutFromEnv()), + }); + } catch (e) { + const err = e as Error; + if (err.name === "TimeoutError" || err.name === "AbortError") throw errors.timeout(serverName + ": request timed out"); + throw errors.connect(serverName + ": " + (err.message || String(e))); + } + + // 5. HTTP status -> typed errors + const text = await res.text(); + if (res.status >= 400) { + throw mapHttpError(serverName, res.status, text, res.headers.get("retry-after")); + } + + // 6. neutral ToolResult + let data: unknown = null; + if (text !== "") { + try { + data = JSON.parse(text); + } catch { + data = text; // non-JSON body: keep raw (json envelope carries it as a string) + } + } + return { data, text: text === "" ? undefined : text }; +} \ No newline at end of file diff --git a/src/openapi/ref.ts b/src/openapi/ref.ts new file mode 100644 index 0000000..b09675c --- /dev/null +++ b/src/openapi/ref.ts @@ -0,0 +1,56 @@ +// Local $ref resolver: inlines #/components/... references into standalone +// schemas. External refs are rejected with a clear error; circular refs decay +// into an opaque object marker instead of recursing forever. +import { errors } from "../errors.js"; + +const MAX_DEPTH = 16; +const CIRCULAR: Record = { type: "object", description: "(circular $ref)" }; +// Keys that never affect execution and are often huge — skip resolving inside. +const SKIP_KEYS = new Set(["example", "examples", "externalDocs"]); + +function lookupPointer(root: unknown, ref: string): unknown { + const parts = ref + .slice(2) + .split("/") + .map((s) => decodeURIComponent(s).replace(/~1/g, "/").replace(/~0/g, "~")); + let cur: unknown = root; + for (const p of parts) { + if (cur === null || typeof cur !== "object") return undefined; + cur = (cur as Record)[p]; + } + return cur; +} + +export function resolveRefs(node: unknown, root: unknown, seen: ReadonlySet = new Set(), depth = 0): unknown { + if (depth > MAX_DEPTH) return { type: "object", description: "(too deeply nested)" }; + if (Array.isArray(node)) { + return node.map((v) => resolveRefs(v, root, seen, depth + 1)); + } + if (node === null || typeof node !== "object") return node; + const obj = node as Record; + const ref = typeof obj.$ref === "string" ? obj.$ref : undefined; + if (ref) { + if (!ref.startsWith("#/")) { + throw errors.invalidArgument( + 'external $ref is not supported: "' + ref + '"', + "Inline the referenced schema into the spec, or bundle it (tools like openapi-cli / redocly can bundle)" + ); + } + if (seen.has(ref)) return { ...CIRCULAR }; + const target = lookupPointer(root, ref); + if (target === undefined) { + throw errors.invalidArgument('unresolvable $ref: "' + ref + '"', "Fix the spec — the referenced path does not exist"); + } + const { $ref: _drop, ...siblings } = obj; + const resolved = resolveRefs(target, root, new Set(seen).add(ref), depth + 1); + if (resolved !== null && typeof resolved === "object" && Object.keys(siblings).length) { + return { ...(resolved as object), ...siblings }; + } + return resolved; + } + const out: Record = {}; + for (const [k, v] of Object.entries(obj)) { + out[k] = SKIP_KEYS.has(k) ? v : resolveRefs(v, root, seen, depth + 1); + } + return out; +} \ No newline at end of file diff --git a/src/openapi/specstore.ts b/src/openapi/specstore.ts new file mode 100644 index 0000000..d1f8b75 --- /dev/null +++ b/src/openapi/specstore.ts @@ -0,0 +1,112 @@ +// Spec snapshots: every registered OpenAPI server gets a local copy under +// /specs/.json at add time — offline-friendly and immune to +// upstream spec churn. --refresh re-pulls from the recorded origin. +import fs from "node:fs"; +import path from "node:path"; +import { configPath } from "../config.js"; +import { errors } from "../errors.js"; + +const FETCH_TIMEOUT_MS = 30000; + +export function specsDir(): string { + return process.env.AGENTCLI_SPECS_DIR || path.join(path.dirname(configPath()), "specs"); +} + +export function snapshotPath(name: string): string { + return path.join(specsDir(), name + ".json"); +} + +function looksLikeYaml(text: string): boolean { + return /^\s*(openapi|swagger)\s*:\s*["']?3/.test(text); +} + +// Parse + validate a raw spec document (OpenAPI 3.x JSON only). +export function parseSpecJson(raw: string, source: string): Record { + let doc: unknown; + try { + doc = JSON.parse(raw); + } catch (e) { + const err = e as Error; + if (looksLikeYaml(raw)) { + throw errors.invalidArgument("spec at " + source + " looks like YAML — only JSON is supported", "Convert it: redocly bundle spec.yaml --output spec.json, then re-add"); + } + throw errors.invalidArgument("spec at " + source + " is not valid JSON", err.message); + } + if (!doc || typeof doc !== "object" || Array.isArray(doc)) { + throw errors.invalidArgument("spec at " + source + " is not a JSON object"); + } + return doc as Record; +} + +export function isHttpUrl(s: string): boolean { + return /^https?:\/\//i.test(s); +} + +async function fetchOrigin(origin: string, timeoutMs = FETCH_TIMEOUT_MS): Promise { + let res: Response; + try { + res = await fetch(origin, { signal: AbortSignal.timeout(timeoutMs), headers: { Accept: "application/json" } }); + } catch (e) { + const err = e as Error; + if (err.name === "TimeoutError" || err.name === "AbortError") throw errors.timeout("fetching spec timed out: " + origin); + throw errors.connect("cannot fetch spec from " + origin + ": " + err.message); + } + if (!res.ok) { + throw errors.connect("cannot fetch spec from " + origin + ": HTTP " + res.status); + } + return res.text(); +} + +// Snapshot from a URL or local file path. Validates JSON + 3.x shape. +export async function snapshotSpec(name: string, origin: string, timeoutMs?: number): Promise> { + let raw: string; + if (isHttpUrl(origin)) { + raw = await fetchOrigin(origin, timeoutMs); + } else { + try { + raw = fs.readFileSync(origin, "utf8"); + } catch { + throw errors.invalidArgument("cannot read spec file: " + origin, "Check the path (or use an http(s) URL)"); + } + } + const doc = parseSpecJson(raw, origin); + // fail fast on version problems (compileSpec re-checks, this covers add time) + if (typeof doc.swagger === "string") { + throw errors.invalidArgument("spec is Swagger 2.0, only OpenAPI 3.x is supported", "Upgrade the spec to 3.x (e.g. with swagger2openapi)"); + } + if (typeof doc.openapi !== "string" || !doc.openapi.startsWith("3.")) { + throw errors.invalidArgument('spec has no "openapi: 3.x" version field', "Expected OpenAPI 3.x JSON"); + } + fs.mkdirSync(specsDir(), { recursive: true }); + const p = snapshotPath(name); + fs.writeFileSync(p, JSON.stringify(doc, null, 2) + "\n", { mode: 0o600 }); + try { fs.chmodSync(p, 0o600); } catch { /* best effort */ } + return doc; +} + +// Load the local snapshot. Corrupt snapshot + URL origin -> auto re-pull. +export async function loadSnapshotDoc(name: string, origin: string): Promise> { + const p = snapshotPath(name); + const read = (): string => { + try { + return fs.readFileSync(p, "utf8"); + } catch { + throw errors.invalidArgument("spec snapshot is missing for server \"" + name + '"', "Re-add the server: agentcli server remove " + name + " && agentcli server add " + name + " --openapi "); + } + }; + let raw: string; + try { + raw = read(); + return parseSpecJson(raw, p); + } catch (e) { + if (!isHttpUrl(origin)) throw e; + // corrupt/stale snapshot with a URL origin: re-pull once + await snapshotSpec(name, origin); + return parseSpecJson(read(), p); + } +} + +// --refresh: re-pull from origin (URL or local file) and re-snapshot. +export async function refreshSnapshot(name: string, origin: string, timeoutMs?: number): Promise> { + return snapshotSpec(name, origin, timeoutMs); +} \ No newline at end of file diff --git a/src/types.ts b/src/types.ts index ed38a40..0047e27 100644 --- a/src/types.ts +++ b/src/types.ts @@ -1,4 +1,6 @@ // Shared types used across the codebase. +// ToolDef is the unified internal tool contract: every backend (MCP, OpenAPI, +// future ones) produces these; dispatch/flags consume them backend-agnostically. export interface StdioServerSpec { type: "stdio"; @@ -13,7 +15,15 @@ export interface HttpServerSpec { headers?: Record | string[]; } -export type ServerSpec = StdioServerSpec | HttpServerSpec; +export interface OpenApiServerSpec { + type: "openapi"; + spec: string; // absolute path to the local snapshot + origin: string; // original source: URL or absolute file path (--refresh re-pulls) + baseUrl?: string; // overrides spec.servers[0].url + headers?: Record; // may contain ${ENV_VAR} placeholders +} + +export type ServerSpec = StdioServerSpec | HttpServerSpec | OpenApiServerSpec; export interface AgentCliConfig { version: 1; @@ -40,13 +50,31 @@ export interface ToolPropertySchema { [key: string]: unknown; } -export interface McpTool { +// The unified tool definition (formerly McpTool). MCP tool listings map 1:1; +// the OpenAPI compiler emits these from spec operations. +export interface ToolDef { name: string; description?: string; inputSchema?: ToolInputSchema; + tags?: string[]; + // OpenAPI backend only: execution metadata (method/path/param locations). + openapiMeta?: OpenApiOperationMeta; [key: string]: unknown; } +// Execution metadata embedded in ToolDef by the OpenAPI compiler and consumed +// by the OpenAPI executor. All maps are flagName -> original spec name. +export interface OpenApiOperationMeta { + method: string; // uppercase HTTP method + path: string; // path template, e.g. /pets/{petId} + baseUrl: string; // resolved base URL (override or spec.servers[0]) + pathParams: Record; + queryParams: Record; + headerParams: Record; + bodyProps: Record; // flagName -> body property name + rawBody?: string; // flagName holding the whole request body (non-object body schemas) +} + export interface FlagSpec { name: string; type: string; @@ -69,7 +97,7 @@ export interface FlagPlan { } export interface ToolsCache { - tools: McpTool[]; + tools: ToolDef[]; fetchedAt: number; } @@ -95,17 +123,6 @@ export interface DaemonResponse { }; } -export interface CallToolResultEnvelope { - result: unknown; - via: "daemon" | "direct"; -} - -export interface ListToolsResultEnvelope { - tools: McpTool[]; - cached: boolean; - via: "daemon" | "direct"; -} - export interface DaemonStatusData { pid: number; startedAt: number; @@ -114,4 +131,4 @@ export interface DaemonStatusData { toolCalls: number; connectedServers: string[]; socket: string; -} +} \ No newline at end of file diff --git a/test/openapi-compile.test.ts b/test/openapi-compile.test.ts new file mode 100644 index 0000000..cc8dba9 --- /dev/null +++ b/test/openapi-compile.test.ts @@ -0,0 +1,154 @@ +// Unit tests for the OpenAPI spec -> ToolDef compiler. +import { test } from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { compileSpec } from "../src/openapi/compile.js"; +import { resolveRefs } from "../src/openapi/ref.js"; +import { AgentCliError } from "../src/errors.js"; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const doc = JSON.parse(fs.readFileSync(path.join(__dirname, "..", "..", "fixtures", "openapi.json"), "utf8")); + +function byName(tools: ReturnType, name: string) { + const t = tools.find((x) => x.name === name); + assert.ok(t, "expected tool " + name + " to compile"); + return t; +} + +test("compiles operations with operationId as tool names", () => { + const tools = compileSpec(doc); + const names = tools.map((t) => t.name); + assert.ok(names.includes("getPetById")); + assert.ok(names.includes("listPets")); + assert.ok(names.includes("addPet")); + assert.ok(names.includes("getOrder")); +}); + +test("missing operationId falls back to a mechanical slug", () => { + const tools = compileSpec(doc); + assert.ok(tools.some((t) => t.name === "get_no-op-id")); +}); + +test("duplicate names get deterministic suffixes", () => { + const dup = JSON.parse(JSON.stringify(doc)); + dup.paths["/pets/{petId}"].get.operationId = "listPets"; + const tools = compileSpec(dup); + const names = tools.map((t) => t.name); + assert.equal(names.filter((n) => n.startsWith("listPets")).length, 2); + assert.ok(names.includes("listPets_2")); +}); + +test("path params are forced required; query params optional unless declared", () => { + const tool = byName(compileSpec(doc), "getPetById"); + assert.deepEqual(tool.inputSchema?.required, ["petId"]); + assert.equal(tool.inputSchema?.properties?.petId.type, "integer"); + assert.equal(tool.inputSchema?.properties?.verbose.type, "boolean"); +}); + +test("parameter $refs are inlined with defaults", () => { + const tool = byName(compileSpec(doc), "getPetById"); + const limit = tool.inputSchema?.properties?.limit; + assert.equal(limit?.type, "integer"); + assert.equal(limit?.default, 10); + assert.equal(limit?.description, "Page size"); + assert.equal(tool.openapiMeta?.queryParams.limit, "limit"); +}); + +test("body properties flatten to top-level flags", () => { + const tool = byName(compileSpec(doc), "addPet"); + const props = tool.inputSchema?.properties || {}; + assert.equal(props.name.type, "string"); + assert.deepEqual(props.kind.enum, ["cat", "dog"]); // resolved $ref inlined + assert.deepEqual(tool.inputSchema?.required, ["name"]); + assert.equal(tool.openapiMeta?.bodyProps.name, "name"); +}); + +test("array body becomes a whole-body param", () => { + const tool = byName(compileSpec(doc), "postBatch"); + assert.equal(tool.openapiMeta?.rawBody, "body"); + assert.equal(tool.inputSchema?.properties?.body.type, "array"); +}); + +test("circular $refs decay instead of recursing", () => { + const tools = compileSpec(doc); // must not hang + const tool = byName(tools, "postCircular"); + const children = tool.inputSchema?.properties?.children as { items?: { description?: string } }; + assert.match(String(children?.items?.description || ""), /circular/); +}); + +test("tags are collected for help grouping", () => { + const tools = compileSpec(doc); + assert.deepEqual(byName(tools, "getPetById").tags, ["pets"]); + assert.deepEqual(byName(tools, "getOrder").tags, ["store"]); + assert.equal(byName(tools, "get_no-op-id").tags, undefined); +}); + +test("header params map to header flags", () => { + const tool = byName(compileSpec(doc), "getOrder"); + assert.equal(tool.openapiMeta?.headerParams["X-Request-Id"], "X-Request-Id"); +}); + +test("meta records method, path template, baseUrl", () => { + const tool = byName(compileSpec(doc), "getPetById"); + assert.equal(tool.openapiMeta?.method, "GET"); + assert.equal(tool.openapiMeta?.path, "/pets/{petId}"); + assert.equal(tool.openapiMeta?.baseUrl, "https://api.mini.test/v1"); +}); + +test("baseUrl override wins over spec servers", () => { + const tool = byName(compileSpec(doc, { baseUrl: "https://override.test" }), "getPetById"); + assert.equal(tool.openapiMeta?.baseUrl, "https://override.test"); +}); + +test("no servers and no override -> clear error", () => { + const noServers = { ...doc, servers: [] }; + assert.throws(() => compileSpec(noServers), (e: unknown) => { + assert.ok(e instanceof AgentCliError); + assert.match(e.message, /--base-url/); + return true; + }); +}); + +test("swagger 2.0 is rejected with an upgrade hint", () => { + assert.throws(() => compileSpec({ swagger: "2.0", paths: {} }), (e: unknown) => { + assert.ok(e instanceof AgentCliError); + assert.match(e.message, /Swagger 2.0/); + return true; + }); +}); + +test("non-3.x openapi field is rejected", () => { + assert.throws(() => compileSpec({ openapi: "4.0.0", paths: {} }), AgentCliError); +}); + +test("resolveRefs: external refs are rejected", () => { + assert.throws(() => resolveRefs({ $ref: "https://evil.example.com/schema.json" }, doc), (e: unknown) => { + assert.ok(e instanceof AgentCliError); + assert.match(e.message, /external/); + return true; + }); +}); + +test("resolveRefs: unresolvable pointer is rejected", () => { + assert.throws(() => resolveRefs({ $ref: "#/components/schemas/Missing" }, doc), (e: unknown) => { + assert.ok(e instanceof AgentCliError); + assert.match(e.message, /unresolvable/); + return true; + }); +}); + +test("relative server URL resolves against the spec origin URL", () => { + const relative = JSON.parse(JSON.stringify(doc)); + relative.servers = [{ url: "/api/v3" }]; + const tool = byName(compileSpec(relative, { originUrl: "https://petstore.example.com/specs/openapi.json" }), "getPetById"); + assert.equal(tool.openapiMeta?.baseUrl, "https://petstore.example.com/api/v3"); +}); + +test("path-level parameters merge with operation parameters", () => { + const shared = JSON.parse(JSON.stringify(doc)); + shared.paths["/pets/{petId}"].parameters = [{ name: "X-Shared", in: "header", schema: { type: "string" } }]; + const tool = byName(compileSpec(shared), "getPetById"); + assert.equal(tool.inputSchema?.properties?.["X-Shared"].type, "string"); +}); \ No newline at end of file diff --git a/test/openapi-e2e.test.ts b/test/openapi-e2e.test.ts new file mode 100644 index 0000000..e6bde73 --- /dev/null +++ b/test/openapi-e2e.test.ts @@ -0,0 +1,304 @@ +// End-to-end: register an OpenAPI spec, then call operations against a real +// local HTTP server (base-url override). Covers URL building, body flattening, +// auth header injection with ${ENV} expansion, HTTP error mapping, snapshots. +import { test, before, after } from "node:test"; +import assert from "node:assert/strict"; +import http from "node:http"; +import { spawn } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const ROOT = path.join(__dirname, "..", ".."); +const BIN = path.join(ROOT, "bin", "agentcli.js"); +const FIXTURE_SPEC = path.join(ROOT, "fixtures", "openapi.json"); + +let tmp: string; +let configFile: string; +let originSpec: string; +let httpServer: http.Server; +let port = 0; + +// Requests seen by the test server (for assertions). +let lastRequest: { method: string; url: string; headers: http.IncomingHttpHeaders; body: string }; + +// Async spawn: the in-process HTTP test server must keep serving while the +// CLI child runs (spawnSync would block the event loop and deadlock both). +interface CliResult { + status: number | null; + stdout: string; + stderr: string; +} + +function cli(args: string[], env: Record = {}): Promise { + return new Promise((resolve, reject) => { + const child = spawn(process.execPath, [BIN, ...args], { + cwd: ROOT, + env: { ...process.env, AGENTCLI_CONFIG: configFile, ...env }, + }); + let stdout = ""; + let stderr = ""; + child.stdout.on("data", (d: Buffer) => (stdout += d.toString())); + child.stderr.on("data", (d: Buffer) => (stderr += d.toString())); + child.on("error", reject); + child.on("close", (code) => resolve({ status: code ?? 0, stdout, stderr })); + }); +} + +before(async () => { + tmp = fs.mkdtempSync(path.join(os.tmpdir(), "agentcli-openapi-")); + configFile = path.join(tmp, "config.json"); + // copy the fixture so we can mutate "origin" without touching the repo file + originSpec = path.join(tmp, "origin-spec.json"); + fs.copyFileSync(FIXTURE_SPEC, originSpec); + + // the CLI child expands ${TEST_TOKEN} from its own environment + process.env.TEST_TOKEN = "secret-token"; + + httpServer = http.createServer((req, res) => { + let body = ""; + req.on("data", (c: Buffer) => (body += c.toString())); + req.on("end", () => { + lastRequest = { method: req.method || "", url: req.url || "", headers: req.headers, body }; + const url = new URL(req.url || "/", "http://x"); + // petId drives the response shape: error mapping without extra routes + const petId = url.pathname.match(/\/pets\/(\d+)$/)?.[1]; + if (petId === "404") { + res.writeHead(404, { "Content-Type": "application/json" }); + res.end(JSON.stringify({ message: "pet not found" })); + return; + } + if (petId === "401") { + res.writeHead(401, { "Content-Type": "application/json" }); + res.end(JSON.stringify({ message: "bad token" })); + return; + } + if (petId === "500") { + res.writeHead(500, { "Content-Type": "application/json" }); + res.end(JSON.stringify({ error: { message: "kaboom" } })); + return; + } + if (url.pathname === "/v1/pets" && req.method === "POST" && body.includes('"boom"')) { + res.writeHead(429, { "Content-Type": "application/json", "Retry-After": "7" }); + res.end(JSON.stringify({ message: "slow down" })); + return; + } + if (petId === "999") { + res.writeHead(200, { "Content-Type": "text/plain" }); + res.end("

not json

"); + return; + } + const query: Record = {}; + for (const k of new Set(url.searchParams.keys())) { + const all = url.searchParams.getAll(k); + query[k] = all.length > 1 ? all : all[0]; + } + res.writeHead(200, { "Content-Type": "application/json" }); + res.end(JSON.stringify({ ok: true, method: req.method, path: url.pathname, query })); + }); + }); + await new Promise((resolve) => httpServer.listen(0, "127.0.0.1", resolve)); + port = (httpServer.address() as { port: number }).port; + + // register with a header template + base-url override pointing at the test server + const add = await cli([ + "server", "add", "mini", + "--openapi", originSpec, + "--base-url", "http://127.0.0.1:" + port + "/v1", + "--header", "Authorization: Bearer ${TEST_TOKEN}", + ], { TEST_TOKEN: "secret-token" }); + assert.equal(add.status, 0, add.stderr); + const out = JSON.parse(add.stdout); + assert.equal(out.ok, true); + assert.equal(typeof out.operations, "number"); + assert.ok(out.operations >= 7); +}); + +after(() => { + httpServer.close(); + fs.rmSync(tmp, { recursive: true, force: true }); +}); + +test("add snapshots the spec and writes the compiled cache", async () => { + const snapshot = path.join(path.dirname(configFile), "specs", "mini.json"); + assert.ok(fs.existsSync(snapshot), "snapshot must exist next to the config"); + const cache = path.join(path.dirname(configFile), "cache", "mini.tools.json"); + assert.ok(fs.existsSync(cache)); + const cfg = JSON.parse(fs.readFileSync(configFile, "utf8")); + assert.equal(cfg.servers.mini.type, "openapi"); + assert.equal(cfg.servers.mini.baseUrl, "http://127.0.0.1:" + port + "/v1"); +}); + +test("server -h lists operations grouped by tag", async () => { + const r = await cli(["mini", "-h"]); + assert.equal(r.status, 0, r.stderr); + assert.match(r.stdout as string, /pets:/); + assert.match(r.stdout as string, /store:/); + assert.match(r.stdout as string, /getPetById/); + assert.match(r.stdout as string, /OpenAPI server/); +}); + +test("GET with path param builds the URL and injects the auth header", async () => { + const r = await cli(["mini", "getPetById", "--petId", "7"], { TEST_TOKEN: "t123" }); + assert.equal(r.status, 0, r.stderr); + const out = JSON.parse(r.stdout as string); + assert.equal(out.ok, true); + assert.equal(out.data.path, "/v1/pets/7"); + assert.equal(out.data.method, "GET"); + assert.equal(out.meta.via, "direct"); + assert.equal(lastRequest.headers.authorization, "Bearer t123"); +}); + +test("query params (incl. arrays via repeated flags) serialize", async () => { + const r = await cli(["mini", "listPets", "--tags", "a", "--tags", "b", "--limit", "3"]); + assert.equal(r.status, 0, r.stderr); + assert.deepEqual(JSON.parse(r.stdout as string).data.query, { tags: ["a", "b"], limit: "3" }); + assert.match(lastRequest.url, /tags=a&tags=b/); +}); + +test("flattened body props POST as JSON", async () => { + const r = await cli(["mini", "addPet", "--name", "rex", "--kind", "dog"]); + assert.equal(r.status, 0, r.stderr); + assert.deepEqual(JSON.parse(lastRequest.body), { name: "rex", kind: "dog" }); + assert.match(lastRequest.headers["content-type"] as string, /application\/json/); +}); + +test("required body prop is enforced client-side", async () => { + const r = await cli(["mini", "addPet"]); + assert.equal(r.status, 1); + assert.match(JSON.parse(r.stderr as string).error.message, /missing required parameter: name/); +}); + +test("header param maps to a request header", async () => { + const r = await cli(["mini", "getOrder"]); + assert.equal(r.status, 0, r.stderr); + const r2 = await cli(["mini", "getOrder", "--X-Request-Id", "abc-1"]); + assert.equal(r2.status, 0, r2.stderr); + assert.equal(lastRequest.headers["x-request-id"], "abc-1"); +}); + +test("array body passes through via repeated flags", async () => { + const r = await cli(["mini", "postBatch", "--body", "a", "--body", "b"]); + assert.equal(r.status, 0, r.stderr); + assert.deepEqual(JSON.parse(lastRequest.body), ["a", "b"]); +}); + +test("HTTP 404 -> NOT_FOUND with the API's own message", async () => { + const r = await cli(["mini", "getPetById", "--petId", "404"]); + assert.equal(r.status, 1); + const err = JSON.parse(r.stderr as string); + assert.equal(err.error.code, "NOT_FOUND"); + assert.match(err.error.message, /pet not found/); +}); + +test("HTTP 401 -> AUTH_REQUIRED", async () => { + const r = await cli(["mini", "getPetById", "--petId", "401"]); + assert.equal(r.status, 1); + assert.equal(JSON.parse(r.stderr as string).error.code, "AUTH_REQUIRED"); +}); + +test("HTTP 500 -> EXECUTION_ERROR with status in details", async () => { + const r = await cli(["mini", "getPetById", "--petId", "500"]); + assert.equal(r.status, 1); + const err = JSON.parse(r.stderr as string); + assert.equal(err.error.code, "EXECUTION_ERROR"); + assert.match(err.error.message, /kaboom/); + assert.equal(err.error.details.httpStatus, 500); +}); + +test("HTTP 429 -> EXECUTION_ERROR with retryAfter", async () => { + const r = await cli(["mini", "addPet", "--name", "boom"]); + assert.equal(r.status, 1); + const err = JSON.parse(r.stderr as string); + assert.equal(err.error.code, "EXECUTION_ERROR"); + assert.equal(err.error.details.retryAfter, "7"); +}); + +test("unset ${ENV} in a header -> AUTH_REQUIRED before any request", async () => { + // empty string is a defined var; force undefined by deleting it from the child env + const env: Record = { ...process.env, AGENTCLI_CONFIG: configFile }; + delete env.TEST_TOKEN; + const r2 = await new Promise((resolve, reject) => { + const child = spawn(process.execPath, [BIN, "mini", "listPets"], { cwd: ROOT, env: env as NodeJS.ProcessEnv }); + let stdout = ""; + let stderr = ""; + child.stdout.on("data", (d: Buffer) => (stdout += d.toString())); + child.stderr.on("data", (d: Buffer) => (stderr += d.toString())); + child.on("error", reject); + child.on("close", (code) => resolve({ status: code ?? 0, stdout, stderr })); + }); + assert.equal(r2.status, 1); + assert.equal(JSON.parse(r2.stderr).error.code, "AUTH_REQUIRED"); + assert.match(r2.stderr, /TEST_TOKEN/); +}); + +test("non-JSON 200 body: text output passes through raw; json wraps as string", async () => { + const text = await cli(["mini", "getPetById", "--petId", "999", "--output", "text"]); + assert.equal(text.status, 0, text.stderr); + assert.equal((text.stdout as string).trim(), "

not json

"); + + const json = await cli(["mini", "getPetById", "--petId", "999"]); + assert.equal(json.status, 0, json.stderr); + assert.equal(JSON.parse(json.stdout as string).data, "

not json

"); +}); + +test("--schema prints the flattened input schema", async () => { + const r = await cli(["mini", "addPet", "--schema"]); + assert.equal(r.status, 0, r.stderr); + const schema = JSON.parse(r.stdout as string); + assert.equal(schema.properties.name.type, "string"); + assert.deepEqual(schema.properties.kind.enum, ["cat", "dog"]); +}); + +test("snapshot independence: calls work after the origin file disappears", async () => { + fs.rmSync(originSpec); + const r = await cli(["mini", "listPets"]); + assert.equal(r.status, 0, r.stderr); +}); + +test("--refresh re-pulls a file origin (updated spec wins)", async () => { + const updated = JSON.parse(fs.readFileSync(FIXTURE_SPEC, "utf8")); + updated.paths["/refreshed"] = { get: { operationId: "refreshedOp", responses: { "200": { description: "ok" } } } }; + fs.writeFileSync(originSpec, JSON.stringify(updated)); + const stale = await cli(["mini", "-h"]); + assert.equal(stale.status, 0, stale.stderr); + assert.ok(!(stale.stdout as string).includes("refreshedOp")); + const r = await cli(["server", "tools", "mini", "--refresh"]); + assert.equal(r.status, 0, r.stderr); + const names = JSON.parse(r.stdout as string).data.map((t: { name: string }) => t.name); + assert.ok(names.includes("refreshedOp")); +}); + +test("server remove cleans up snapshot and cache", async () => { + const r = await cli(["server", "remove", "mini"]); + assert.equal(r.status, 0, r.stderr); + assert.ok(!fs.existsSync(path.join(path.dirname(configFile), "specs", "mini.json"))); + assert.ok(!fs.existsSync(path.join(path.dirname(configFile), "cache", "mini.tools.json"))); +}); + +test("add rejects swagger 2.0 specs with a clear error", async () => { + const swaggerPath = path.join(tmp, "swagger.json"); + fs.writeFileSync(swaggerPath, JSON.stringify({ swagger: "2.0", info: { title: "x", version: "1" }, paths: {} })); + const r = await cli(["server", "add", "old", "--openapi", swaggerPath]); + assert.equal(r.status, 1); + assert.match(r.stderr as string, /Swagger 2.0/); +}); + +test("add rejects YAML with a conversion hint", async () => { + const yamlPath = path.join(tmp, "spec.yaml"); + fs.writeFileSync(yamlPath, "openapi: 3.0.3\ninfo:\n title: x\n"); + const r = await cli(["server", "add", "y", "--openapi", yamlPath]); + assert.equal(r.status, 1); + assert.match(r.stderr as string, /YAML/); +}); + +test("add rejects --openapi together with --url and with a command", async () => { + const a = await cli(["server", "add", "x1", "--openapi", originSpec, "--url", "http://mcp.example.com"]); + assert.equal(a.status, 1); + assert.match(a.stderr as string, /mutually exclusive/); + const b = await cli(["server", "add", "x2", "--openapi", originSpec, "--", "node", "-v"]); + assert.equal(b.status, 1); + assert.match(b.stderr as string, /mutually exclusive/); +}); \ No newline at end of file diff --git a/test/skill.test.ts b/test/skill.test.ts index 4f36d71..bbcf7d3 100644 --- a/test/skill.test.ts +++ b/test/skill.test.ts @@ -27,6 +27,12 @@ test("skill teaches the core loop commands", () => { assert.match(raw, / -h|--help/); }); +test("skill teaches OpenAPI registration alongside MCP", () => { + const raw = fs.readFileSync(SKILL, "utf8"); + assert.match(raw, /--openapi/); + assert.match(raw, /\$\{ENV_VAR\}/); +}); + test("skill documents the escape hatch and flags", () => { const raw = fs.readFileSync(SKILL, "utf8"); assert.match(raw, /--input/);