Skip to content

[Benchmark] RTX 3070 Ti 8 GB + Ryzen 5 7600X — Qwen3.6-35B-A3B abliterated Q5_K_S (0.1.44.1) - #2581

Closed
jkir4n wants to merge 1 commit into
Niko1221:mainfrom
jkir4n:bench/rtx3070ti-qwen36-2026-10-15
Closed

jkir4n wants to merge 1 commit into
Niko1221:mainfrom
jkir4n:bench/rtx3070ti-qwen36-2026-10-15

Conversation

@jkir4n

@jkir4n jkir4n commented Oct 15, 2026 •

Copy link
Copy Markdown

Supersedes #2581 (closed in the 0.2.0-beta tracker reset; branch rebased onto current main).

Area: [Benchmark]

Problem and result

There was no community datapoint for Qwen3.6-35B-A3B on an 8 GiB consumer card. This adds one: a Strata 0.1.44.1 run on an RTX 3070 Ti + Ryzen 5 7600X with a community abliterated Q5_K_S build substituted into the install slot. Before: no result in bench/results/ for this hardware class. After: a result directory in the same shape as the existing community entries.

Related discussion

none

Approach

Two files, both new, no code touched:

  • bench/results/2026-10-15-community-rtx-3070-ti-qwen3-6-35b-a3b-ud-q4-k-xl/README.md — hardware and software, model and configuration, method, results, correctness notes, limits.
  • …/results.json — per-request data from the suite.

Headline numbers (medians, 3 runs): short chat decode 50.6 tok/s (46.6–60.7); 4K fresh prompt 1,189 tok/s prompt / 3.40 s TTFT; 16K fresh prompt 1,100 tok/s prompt / 14.62 s TTFT; 32K-context agent session decode 35.5 tok/s (22.4–47.3) with TTFT 8.28 s (3.31–14.05). VRAM steady at 7,666 of 8,192 MiB; all routed experts run from pageable host RAM at this quant.

Validation

Revision: rebased onto main at 011c9e60 (the 0.2.x line). This PR is documentation-only — no code, no runtime path — so the checks that apply are the added JSON parsing and the file shape.

Passed: python -m json.tool results.json on the added file (valid). The benchmark itself was run with python tools/strata_bench.py against a live HTTP server (model load not timed, one warm-up plus a 1K prompt discarded, unique prompt tags so the conversation cache cannot help). Results parsed from the engine's own timings; per-request data in results.json. A separate 5/5 correctness battery (arithmetic, the five-widgets puzzle, executed fizzbuzz, capital of Australia, exact bullet format) is recorded in the README, not part of the suite.

Failed: none observed.

Not run: no rebuild of the engine (official Windows CUDA 13 release binary used as-is); no CUDA/HIP/SYCL/Metal variants; single machine, single session; no correctness suite integrated into the benchmark.

Limits

Numbers are single-session medians on a box at its RAM edge (free RAM ranged 0.2–3.8 GiB during the runs; chat decode dips to ~14 tok/s when Windows trims model pages), so expect variance with different background load. The model is a community abliterated fine-tune (Q5_K_S, MTP block preserved) substituted into the UD-Q4_K_XL slot — the server label comes from that slot. A second suite run and a 4-client concurrency run on the same day are documented in the README as additional measurements, not part of the primary table.

…_S (0.1.44.1)

Results-only community benchmark; details and limitations in the report README.
@Niko1221

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

@Niko1221 Niko1221 closed this Oct 19, 2026
@jkir4n

jkir4n commented Oct 21, 2026

Copy link
Copy Markdown
Author

/reopen

@github-actions

Copy link
Copy Markdown

Not reopened yet. Still missing:

  • Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] fix decode stop on long prompts".
  • Say in Validation which revision you tested: rebase on main (or 0.2.0-beta) and re-run your checks.

Edit the pull request and comment /reopen again. Details: CONTRIBUTING.md

@jkir4n jkir4n changed the title Community benchmark: RTX 3070 Ti 8 GB + Ryzen 5 7600X - Qwen3.6-35B-A3B abliterated Q5_K_S [Benchmark] RTX 3070 Ti 8 GB + Ryzen 5 7600X — Qwen3.6-35B-A3B abliterated Q5_K_S (0.1.44.1) Oct 21, 2026
@jkir4n

jkir4n commented Oct 21, 2026

Copy link
Copy Markdown
Author

/reopen

@github-actions

Copy link
Copy Markdown

Not reopened yet. Still missing:

  • Say in Validation which revision you tested: rebase on main (or 0.2.0-beta) and re-run your checks.

Edit the pull request and comment /reopen again. Details: CONTRIBUTING.md

@jkir4n

jkir4n commented Oct 21, 2026

Copy link
Copy Markdown
Author

/reopen

@github-actions

Copy link
Copy Markdown

The check passed, but GitHub would not reopen it (Validation Failed: ). If the branch was deleted, restore it or open a new pull request and link this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants