Skip to content

feat(plugin): support tencent cloud cls plugin zstd compress - #14005

Open
farmerx wants to merge 1 commit into
apache:masterfrom
farmerx:feature/v2026-10-03-support-clszstd
Open

farmerx wants to merge 1 commit into
apache:masterfrom
farmerx:feature/v2026-10-03-support-clszstd

Conversation

@farmerx

@farmerx farmerx commented Oct 3, 2026

Copy link
Copy Markdown

Description

The tencent-cloud-cls plugin always uploads logs as an uncompressed protobuf
payload, although the CLS structured log API accepts compressed uploads through
the x-cls-compress-type request header (see
上传结构化日志).
Compressing the payload reduces the log write traffic remarkably (about 21x on
realistic access logs), which is the main cost driver for high volume
deployments.

This PR adds optional zstd compression for the upload, mirroring the behaviour
of the official Go SDK (zstdCompress + x-cls-compress-type: zstd).

Changes:

  1. New apisix/utils/zstd.lua: a thin LuaJIT FFI binding over libzstd
    (ZSTD_compressBound, ZSTD_compress, ZSTD_isError,
    ZSTD_getErrorName), with the default compression level 3
    (ZSTD_CLEVEL_DEFAULT, the same as zstd.SpeedDefault used by the Go SDK).
    libzstd is loaded lazily and the result - including the failure - is cached,
    trying libzstd.so.1, zstd and libzstd in order so that environments
    shipping only the versioned soname work too. available() reports whether
    zstd is usable, compress() returns nil, err instead of throwing.
  2. apisix/plugins/tencent-cloud-cls/cls-sdk.lua: _M.new() accepts an extra
    optional compress_type argument (unknown values are rejected);
    send_cls_request() compresses the encoded protobuf when zstd is
    configured and adds the x-cls-compress-type: zstd header. Content-Type,
    the signature and the request URL are unchanged. When compression fails, e.g.
    because libzstd is missing at runtime, the log is uploaded uncompressed and
    an error is logged - a missing native library never causes log loss.
  3. apisix/plugins/tencent-cloud-cls.lua: new compress_type attribute
    ("none" | "zstd", default "none"), passed through to the SDK.
  4. Docs: compress_type and the libzstd requirement are documented in both
    docs/en/latest/plugins/tencent-cloud-cls.md and
    docs/zh/latest/plugins/tencent-cloud-cls.md.
  5. utils/install-dependencies.sh: installs libzstd-dev (apt) and
    libzstd-devel (yum) so that dev/CI environments have libzstd available.

Tests:

  • New t/utils/zstd.t: available(), argument validation, the zstd frame magic
    number 28 b5 2f fd, the compressed size and compression levels 1/3/9 (the
    compression cases are skipped when libzstd is not installed).
  • t/plugin/tencent-cloud-cls.t TEST 23-26: the schema rejects unsupported
    compress types, a route can be created with compress_type: zstd, the
    uploaded body is a zstd frame and carries x-cls-compress-type: zstd, and
    compression failure falls back to an uncompressed upload that is still
    processed successfully.

Notes for reviewers: this introduces a runtime dependency on libzstd (only when
compress_type: zstd is used). Images that do not ship libzstd1 should add
it; without it the plugin keeps working and just uploads uncompressed payloads.

Which issue(s) this PR fixes:

Fixes #

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@farmerx farmerx changed the title 支持cls zstd compress feat(plugin): support tencent cloud cls plugin zstd compress Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant