Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions docs/sql-manual/basic-element/session-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,47 @@ SET enable_file_scanner_v2 = false;
SET GLOBAL enable_file_scanner_v2 = false;
```

## Development Version

### `external_scan_consistent_hash_spread_num`

This option is for development builds that support external split spreading. It can spread independently planned queries for the same remote file split across eligible backends, reducing a persistent scan hotspot.

| Item | Value |
| --- | --- |
| Type | Integer |
| Default | `1` |
| Valid values | `0` through `2147483647` |
| Scope | Session; `SET GLOBAL` applies to new sessions |

| Value | Behavior |
| --- | --- |
| `1` (default) | Preserve the original scheduling strategy, configured candidate counts and global split redistribution. It does **not** limit scans to one backend. |
| `0` | Enable automatic spreading across all eligible backends in the query's compute group. Use consistent hashing when `enable_file_cache` or `use_consistent_hash_for_external_scan` is enabled; otherwise use the random strategy. |
| `N > 1` | Spread within at most N distinct consistent-hash candidates, capped by the eligible backend count, when file caching or consistent hashing is enabled. Otherwise preserve round-robin scheduling. |

```sql
-- Enable automatic spreading for this session, without requiring file cache or hash scheduling.
SET external_scan_consistent_hash_spread_num = 0;

-- Limit the hash candidate set. This does not require the cluster to have exactly three backends.
SET use_consistent_hash_for_external_scan = true;
SET external_scan_consistent_hash_spread_num = 3;

-- Preserve the original scheduling.
SET external_scan_consistent_hash_spread_num = 1;
-- Restore the compiled default, also 1.
UNSET VARIABLE external_scan_consistent_hash_spread_num;
```

The spread mode selects the candidate with the least scan weight already assigned by this query's policy; ties are broken uniformly at random. Each split is assigned once. It does not select by live backend CPU usage, and does not guarantee a throughput multiplier. Local preference and mandatory Host constraints take precedence; they may select a backend outside a remote split's hash candidates. File load and schema scans retain their existing behavior.

Spreading disables subsequent global split redistribution, so assignments remain within the split's eligible candidates and locality constraints. With multiple splits, balancing is limited to each split's candidates and may be less even than the original global redistribution.

This option does not enable file caching. When file caching is enabled, `0` can create cached copies on every eligible backend and increase warmup reads and cache storage; explicit N limits remote hash candidates, subject to locality precedence. It does not pre-create cache replicas. Start with `1`; enable spreading when repeated queries concentrate on a saturated backend and other eligible backends have spare capacity. Compare both modes with the same query, cache state and resources.

Persisted global values are retained on upgrade. Changing the compiled default does not reset a saved global value of `0`; use `SET GLOBAL external_scan_consistent_hash_spread_num = 1` if new sessions should preserve original scheduling. Existing sessions must change their session value separately. Negative, non-integer or out-of-range values are rejected.

## What changed in 4.1.4

| Category | Variables |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,47 @@ SET enable_file_scanner_v2 = false;
SET GLOBAL enable_file_scanner_v2 = false;
```

## 开发版本

### `external_scan_consistent_hash_spread_num`

此参数适用于包含外部 split 分散调度功能的开发版本。对于同一个远程文件 split 的多次独立查询,可将扫描工作分散到合法候选 BE,减少持续的扫描热点。

| 项目 | 值 |
| --- | --- |
| 类型 | 整数 |
| 默认值 | `1` |
| 合法范围 | `0` 至 `2147483647` |
| 作用域 | 会话;`SET GLOBAL` 对之后创建的新会话生效 |

| 取值 | 行为 |
| --- | --- |
| `1`(默认) | 保留原调度策略、配置的候选数量及全局 split 再分配。**不表示只使用一个 BE**。 |
| `0` | 开启自动分散,使用查询计算组内全部合法候选 BE。开启 `enable_file_cache` 或 `use_consistent_hash_for_external_scan` 时使用一致性哈希,否则使用随机策略。 |
| `N > 1` | 开启文件缓存或一致性哈希时,在最多 N 个不同的哈希候选中分散,实际数量不超过合法候选 BE 数量;否则保留轮询调度。 |

```sql
-- 本会话开启自动分散,无需先开启文件缓存或哈希调度。
SET external_scan_consistent_hash_spread_num = 0;

-- 限制哈希候选范围,不要求集群恰好有三个 BE。
SET use_consistent_hash_for_external_scan = true;
SET external_scan_consistent_hash_spread_num = 3;

-- 保留原调度。
SET external_scan_consistent_hash_spread_num = 1;
-- 恢复编译默认值,同样为 1。
UNSET VARIABLE external_scan_consistent_hash_spread_num;
```

分散模式优先选择本次查询的调度 policy 中已分配扫描权重最小的候选,权重相同时均匀随机选择。每个 split 仅分配一次;不根据 BE 实时 CPU 选点,也不保证固定吞吐提升倍数。本地优先和强制 Host 约束优先于远程哈希候选范围,可能选择范围外节点。文件导入和 schema 扫描保留原行为。

分散模式关闭后续全局 split 再分配,以保持候选范围和本地约束。多 split 查询仅在各 split 的候选范围内均衡,可能不如原全局再分配均匀。

此参数不会开启文件缓存。文件缓存开启时,`0` 可能在全部合法候选 BE 保存相同数据,增加预热读取和缓存空间;显式 N 可限制远程哈希候选,但本地性仍优先。不会预先创建缓存副本。建议保留默认 `1`;当重复查询集中到已饱和 BE 且其他候选 BE 有可用容量时再开启分散,并使用相同查询、缓存状态和资源进行对比。

升级保留已持久化的全局值。修改编译默认值不会重置已保存的全局 `0`;若需新会话保留原调度,可执行 `SET GLOBAL external_scan_consistent_hash_spread_num = 1`。已有会话需单独修改会话值。负数、非整数和越界值会被拒绝。

## 4.1.4 版本变更概览

| 类别 | 变量 |
Expand Down