Skip to content

Commit 2853389

Browse files
committed
fix(telemetry): record unrecognized callers as unknown instead of human
1 parent b34b967 commit 2853389

7 files changed

Lines changed: 22 additions & 22 deletions

File tree

‎docs/agents/telemetry-change.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -95,17 +95,17 @@ source-config 只用于百炼 / DashScope API 侧消费,不发送到通用网
9595
- `command`、`timestamp`、`durationMs`、`success`
9696
- `cliVersion`、`nodeVersion`、`os`
9797
- `authMethod`
98-
- `callerType`(必填:`agent` 或 `human`);`callerType=agent` 时可选 `callerName`、`skillName`、`skillVersion`
98+
- `callerType`(必填:`agent` 或 `unknown`);`callerType=agent` 时可选 `callerName`、`skillName`、`skillVersion`
9999
- 失败时的 `errorMessage`、`exitCode`、`httpStatus`、`requestId`
100100
- 安全 allowlist 过滤后的 `params`
101101

102-
调用方归因只认本次进程的 `BAILIAN_CALLER_*`(由 `readCaller` 归一化后写入事件),口径对齐 Ark CLI:
102+
调用方归因只认本次进程的 `BAILIAN_CALLER_*`(由 `readCaller` 归一化后写入事件):
103103

104104
- `BAILIAN_CALLER_TYPE` 为 `agent` / `ai_agent`(大小写不敏感)→ `callerType=agent`
105-
- 未设置、空白、显式 `human` 或其他值 → `callerType=human`;人类终端不要手动设这些变量
105+
- 未设置、空白或其他值 → `callerType=unknown`;不要把该字段当成人工
106106
- `callerName` / `skillName` / `skillVersion` 仅在 `agent` 时发送;`kscli` 通常没有 skill 字段
107107

108-
新 CLI 每条进入 middleware 的命令事件都有 `callerType`。旧 CLI 没有该字段。看板只统计带 `callerType` 的事件,不能把字段缺失混进分母。新 CLI + 旧 skill / Agent 漏设变量时,会落进 `human`(与 Ark 相同,靠 skill 强制贴标签收敛)。`BAILIAN_CALLER_*` 是调用方声明,不是 `Identity`,不得用来猜 `bl` / `kscli`。
108+
新 CLI 每条进入 middleware 的命令事件都有 `callerType`。旧 CLI 没有该字段。看板只统计带 `callerType` 的事件,不能把字段缺失混进分母。`unknown` 只表示未识别为 agent,不是人工。新 CLI + 旧 skill / Agent 漏设变量时,会落进 `unknown`。`BAILIAN_CALLER_*` 是调用方声明,不是 `Identity`,不得用来猜 `bl` / `kscli`。
109109

110110
参数默认不上传,只有 `packages/core/src/telemetry/tracker.ts` 的 `PARAM_ALLOWLIST` 中字段会进入事件。不得加入 prompt、凭证、文件路径、URL、账号/租户/工作空间 ID 或其他用户内容。
111111

@@ -158,7 +158,7 @@ AEM 映射:
158158

159159
- [ ] 更新 `TrackingEvent`、`createTrackingEvent()` 与 `buildRemoteAemOptions()` 的字段映射
160160
- [ ] 本地 JSONL 与远端 AEM 必须基于同一结构化事件,不能维护两套字段口径
161-
- [ ] 调用方字段经 `readCaller` → `TrackingDeps.caller` → 事件;未设置默认 `human`,仅显式 Agent 声明为 `agent`
161+
- [ ] 调用方字段经 `readCaller` → `TrackingDeps.caller` → 事件;仅显式 `agent` / `ai_agent` 为 `agent`,其余为 `unknown`
162162
- [ ] 成功与失败均覆盖;遥测异常必须静默且不改变业务退出码
163163
- [ ] runtime 本地语义错误应记录 `exitCode`;新增字段默认随 AEM `ext` 上报,无需占用新的 `c1`—`c5`
164164
- [ ] 检查 `DO_NOT_TRACK=1` 与 `telemetry: false` 两个关闭入口
@@ -177,7 +177,7 @@ vp test packages/core/tests packages/runtime/tests/telemetry-stage.test.ts packa
177177

178178
- ✗ 只看 AEM 命令事件,误以为它能替代网关侧请求渠道统计
179179
- ✗ 把 `authMethod` 当成实际凭证来源;它只是命令声明的鉴权域
180-
- ✗ 把旧 CLI「没有 `callerType`」和「`callerType=human`」混进同一分母宣称人工占比
180+
- ✗ 把旧 CLI「没有 `callerType`」或「`callerType=unknown`」当成人工占比
181181
- ✗ 用 `BAILIAN_CALLER_*` 猜测产品身份,或根据 TTY / 安装目录反推调用方
182182
- ✗ 新增 bypass `fetch` 后漏掉应由网关消费的 source-config,或把它发给 OSS / npm / 第三方下载地址
183183
- ✗ 只改 `bl` 入口,导致 `kscli` 的产品名或版本标签错误

‎packages/core/src/telemetry/caller.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
/** Parse BAILIAN_CALLER_* from the process env. Defaults to human. */
1+
/** Parse BAILIAN_CALLER_*. Only an explicit agent is `agent`; everything else is `unknown`. */
22

33
export type CallerAttribution =
4-
| { callerType: "human" }
4+
| { callerType: "unknown" }
55
| {
66
callerType: "agent";
77
callerName: string;
@@ -36,7 +36,7 @@ function normalizeCallerName(raw: string | undefined): string {
3636
export function readCaller(env: NodeJS.ProcessEnv): CallerAttribution {
3737
const callerType = readTrimmed(env, CALLER_TYPE_ENV)?.toLowerCase();
3838
if (callerType !== "agent" && callerType !== "ai_agent") {
39-
return { callerType: "human" };
39+
return { callerType: "unknown" };
4040
}
4141

4242
const skillNameRaw = readTrimmed(env, SKILL_NAME_ENV)?.toLowerCase();

‎packages/core/src/telemetry/event.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ export interface TrackingEvent {
1515
os: string;
1616
authMethod?: AuthRequirement;
1717
params?: Record<string, unknown>;
18-
/** Always present on new CLI events: `agent` or `human`. */
18+
/** Always present on new CLI events: `agent` or `unknown`. */
1919
callerType: CallerAttribution["callerType"];
2020
callerName?: string;
2121
skillName?: string;

‎packages/core/tests/telemetry-caller.test.ts‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@ test.each([
88
{ label: "illegal false", env: { BAILIAN_CALLER_TYPE: "false" } },
99
{ label: "illegal ai-agent", env: { BAILIAN_CALLER_TYPE: "ai-agent" } },
1010
{ label: "illegal zero", env: { BAILIAN_CALLER_TYPE: "0" } },
11-
])("maps $label to human without agent fields", ({ env }) => {
12-
expect(readCaller(env)).toEqual({ callerType: "human" });
11+
])("maps $label to unknown without agent fields", ({ env }) => {
12+
expect(readCaller(env)).toEqual({ callerType: "unknown" });
1313
});
1414

1515
test.each([
@@ -130,13 +130,13 @@ test("lowercases skillName before validation", () => {
130130
});
131131
});
132132

133-
test("ignores agent detail fields when callerType is human", () => {
133+
test("ignores agent detail fields when callerType is not agent", () => {
134134
expect(
135135
readCaller({
136136
BAILIAN_CALLER_TYPE: "human",
137137
BAILIAN_CALLER_NAME: "cursor",
138138
BAILIAN_SKILL_NAME: "bailian-cli",
139139
BAILIAN_SKILL_VERSION: "2.5.0",
140140
}),
141-
).toEqual({ callerType: "human" });
141+
).toEqual({ callerType: "unknown" });
142142
});

‎packages/core/tests/telemetry-tracker.test.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ const identity: Identity = {
2121
npmPackage: "bailian-cli",
2222
};
2323

24-
const humanCaller: CallerAttribution = { callerType: "human" };
24+
const unknownCaller: CallerAttribution = { callerType: "unknown" };
2525
const agentCaller: CallerAttribution = {
2626
callerType: "agent",
2727
callerName: "cursor",
@@ -40,7 +40,7 @@ test("records BailianError exitCode in local events and AEM ext", async () => {
4040
identity,
4141
settings: { telemetry: true } as Settings,
4242
authMethod: "apiKey",
43-
caller: humanCaller,
43+
caller: unknownCaller,
4444
},
4545
["knowledge", "doc", "delete"],
4646
{},
@@ -57,13 +57,13 @@ test("records BailianError exitCode in local events and AEM ext", async () => {
5757
success: false,
5858
exitCode: 7,
5959
errorMessage: "该操作会永久删除文档。",
60-
callerType: "human",
60+
callerType: "unknown",
6161
});
6262
expect(event.callerName).toBeUndefined();
6363
expect(event.skillName).toBeUndefined();
6464
expect(event.skillVersion).toBeUndefined();
6565
expect(buildRemoteAemOptions(event)).toMatchObject({
66-
ext: expect.objectContaining({ exitCode: 7, callerType: "human" }),
66+
ext: expect.objectContaining({ exitCode: 7, callerType: "unknown" }),
6767
});
6868
});
6969

@@ -85,7 +85,7 @@ test.each([
8585
identity,
8686
settings: { telemetry: true } as Settings,
8787
authMethod: "apiKey",
88-
caller: humanCaller,
88+
caller: unknownCaller,
8989
},
9090
["managed-agent", "webhook", "events"],
9191
{ page, pageSize: 20 },

‎packages/runtime/tests/telemetry-stage.test.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ test("wires process.env through readCaller into trackCommandExecution", async ()
7171
expect(next).toHaveBeenCalledOnce();
7272
});
7373

74-
test("defaults to human when caller env is unset", async () => {
74+
test("defaults to unknown when caller env is unset", async () => {
7575
vi.stubEnv("BAILIAN_CALLER_TYPE", undefined);
7676
vi.stubEnv("BAILIAN_CALLER_NAME", undefined);
7777
vi.stubEnv("BAILIAN_SKILL_NAME", undefined);
@@ -83,5 +83,5 @@ test("defaults to human when caller env is unset", async () => {
8383
const deps = trackerMocks.trackCommandExecution.mock.calls[0]![0] as {
8484
caller: CallerAttribution;
8585
};
86-
expect(deps.caller).toEqual({ callerType: "human" });
86+
expect(deps.caller).toEqual({ callerType: "unknown" });
8787
});

‎skills/bailian-protocol/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ After approval, treat Bailian as selected for the current task. Do not ask again
7474
7575
## Call attribution (mandatory for every Agent `bl` command)
7676
77-
Examples in this protocol, `assets/`, and business skills show bare `bl …`. When an Agent **runs** a command, wrap that single invocation with the prefix below (declared once here). Humans must not set these variables; unset defaults to human.
77+
Examples in this protocol, `assets/`, and business skills show bare `bl …`. When an Agent **runs** a command, wrap that single invocation with the prefix below (declared once here). Humans must not set these variables. Anything other than an explicit agent is recorded as `unknown`.
7878
7979
```bash
8080
BAILIAN_CALLER_TYPE=agent \

0 commit comments

Comments
 (0)